Q: Apache Arrow array builder UnsafeAppend

Question

I am working on the array builder UnsafeAppend api. According to the code in the document.

arrow::Int64Builder builder;
// Make place for 8 values in total
builder.Resize(8);
builder.UnsafeAppend(1);
builder.UnsafeAppend(2);
builder.UnsafeAppend(3);
builder.UnsafeAppendNull();
builder.UnsafeAppend(5);
builder.UnsafeAppend(6);
builder.UnsafeAppend(7);
builder.UnsafeAppend(8);

std::shared_ptr<arrow::Array> array;
arrow::Status st = builder.Finish(&array);

Builder.Resize(i) should make space for i value. After changing i into another value such as 100000000,I should have space for 100000000 values. I got a Segmentation fault from my compiler, its a odd. I did another experiment, resizing the builder to 10. The builder should have 10 space only, but my code can successfully Append even more values than 10 to the builder.

I am a bit confused, isnt the builder should have the exactly space i with the api Resize(i). Does anyone knows any correct ways to the UnsafeAppend API?

int row = 100000000
arrow::StringBuilder b1;
b1.Resize(row);
for(int i=0;i<row;i++)
{
   std::string str = "test";
   b1.UnsafeAppend(str);
}

score 0 · Answer 1 · answered Jun 08 '20 at 21:47

Resize(n) reserves space for n entries, but it doesn't reserve space for character data (which is required by arrow::StringBuilder::UnsafeAppend). For your example, I'd recommend:

int row = 100000000
arrow::StringBuilder b1;

b1.Resize(row);

std::string str = "test";
b1.ReserveData(row * str.size());

for(int i=0;i<row;i++)
{
   b1.UnsafeAppend(str);
}

Q: Apache Arrow array builder UnsafeAppend

1 Answers1