Cassandra .csv import error:batch too large

Question

I'm trying to import data from a .csv file to Cassandra 3.2.1 via copy command.In the file are only 299 rows with 14 columns. I get the Error:

Failed to import 299 rows: InvalidRequest - code=2200 [Invalid query] message="Batch too large"

I used the following copy comand and tryied to increase the batch size:

copy table (Col1,Col2,...)from 'file.csv' with delimiter =';' and header = true and MAXBATCHSIZE = 5000;

I think 299 rows are not too much to import to cassandra or i am wrong?

Tried to increase the rows in the .csv step by step. 200 rows works fine, 300 not. — Emlon, Apr 14 '16 at 09:00

score 11 · Answer 1 · answered Apr 01 '19 at 20:00

11

Adding the CHUNKSIZE keyword resolved the problem for me.

e.g. copy event_stats_user from '/home/kiren/dumps/event_stats_user.csv ' with CHUNKSIZE=1 ;

answered Apr 01 '19 at 20:00

kirenpillay

339
1
5
7

+1 this solved my problem without altering the cassandra.yaml/scylla.yaml. Im using ScyllaDB by the way. – Karl Anthony Baluyot Nov 20 '19 at 07:18
This Solved the problem +! – userguy Jan 12 '21 at 16:58

score 7 · Answer 2 · answered Apr 14 '16 at 09:23

The error you're encountering is a server-side error message, saying that the size (in term of bytes count) of your batch insert is too large.

This batch size is defined in the cassandra.yaml file:

# Log WARN on any batch size exceeding this value. 5kb per batch by default.
# Caution should be taken on increasing the size of this threshold as it can lead to node instability.
batch_size_warn_threshold_in_kb: 5

# Fail any batch exceeding this value. 50kb (10x warn threshold) by default.
batch_size_fail_threshold_in_kb: 50

If you insert a lot of big columns (in size) you may reach quickly this threshold. Try to reduce MAXBATCHSIZE to 200.

More info on COPY options here

Cassandra .csv import error:batch too large

2 Answers2

Linked