I'm trying to bulk load some data into monetdb. I followed this example. It works for my test data. But when I use the data from production environment, I get exceptions.
Caused by: java.lang.Exception: 42000!syntax error, unexpected IDENT, expecting DELIMITERS in: "copy into dm.fact_sem_keyword_collection from stdin using delimters"
25005!current transaction is aborted (please ROLLBACK)
at com.lietou.bi.dw.task.common.dataexchange.MonetDBBulkLoadWriter.flush(MonetDBBulkLoadWriter.java:140)
... 12 more
and I found the following log from the debug log.
20150804x_m_0001323100000001255480000000001310100152015-08-05 10:28:44
20150804x_z_0002000000000001000000000000000002015-08-05 10:28:44
20150804xn_cd_01000000000002000000000000000002015-08-05 10:28:44
20150804z_s_0002000000000001000000000000000002015-08-05 10:28:44
RD 1438779576672: read final block: 190 bytes
RX 1438779576672: !42000!syntax error, unexpected IDENT, expecting DELIMITERS in: "copy into dm.fact_sem_keyword_collection from stdin using delimters"
!25005!current transaction is aborted (please ROLLBACK)
RD 1438779576672: inserting prompt
I think there must be some bad data, but I don't know how to find more details to help me locate it.
And there is one more thing. Monetdb seems to read data using a buffer size of 8190 bytes. But right before the exception, there is a write final block of different size. Logs are showed below. What does this mean?
...
TD 1438779576129: write block: 8190 bytes
TX 1438779576129:
...
TD 1438779576137: write block: 8190 bytes
TX 1438779576137:
...
TD 1438779576137: write final block: 5921 bytes
TX 1438779576137:
...
