0

I am using MonetDb-Pig layer to load the csv data into Monet db. Internally it is using Binarybulkload commands to load the data but after loading data into table, the csv file values are not not matching with Monet db table values(int ,double).Seems to be data converted into binary format.

How can we get back the actual values in monetdb? .

Table Structure that I am using CREATE TABLE "test" ( "s_suppkey" INT, "s_name" CLOB, "s_address" CLOB, "s_nationkey" INT, "s_phone" CLOB, "s_acctbal" DOUBLE, "s_comment" CLOB );

Load command that I am using COPY BINARY INTO "test" FROM ( '$PATH/part-1/col-0.bulkload', '$PATH/part-1/col-1.bulkload', '$PATH/part-1/col-2.bulkload', '$PATH/part-1/col-3.bulkload', '$PATH/part-1/col-4.bulkload', '$PATH/part-1/col-5.bulkload', '$PATH/part-1/col-6.bulkload' );

RamohanReddy
  • 3
  • 2

2 Answers2

0

The information provided is insufficient to isolate the issue. The most probable issue is a mis-alignment of the number of values in each of the binary columns.

Check the size of the elements in 's_acctbal' input file, to see if it produced Floats instead of Double binary values.

btw. the MonetDB-Pig project is not actively maintained, but we welcome patches.

mkersten
  • 694
  • 3
  • 7
  • i was able lo to load by converting byte Array to LittleEndian. Can you please help how to store timestamps into monet db using copy binary! – RamohanReddy Aug 09 '21 at 08:09
  • @raj in the most recent version of MonetDB (Jul2021) you can load temporal data using the struct definitions listed in the header file copybinary.h – Joeri van Ruth Aug 10 '21 at 08:30
  • @JoerivanRuth, can you please share the struct definitions in the header file 'copybinary.h' – Daksh Sep 02 '21 at 08:48
  • @JoerivanRuth can you please explain in detail how can we load date using struct definition. – RamohanReddy Sep 02 '21 at 09:44
0

please convert byte buffer from BigEndian to LittleEndian, and check

Daksh
  • 148
  • 11
  • Note that in the most recent version of MonetDB (Jul2021), you can explicitly set the endianness of the loaded data: COPY BIG ENDIAN BINARY FROM ... – Joeri van Ruth Aug 10 '21 at 08:28