Best way to parse multipart file (CSV) and write in DB

Question

I have a file that is received as a multipart file from an HTTP request. I need to read this file, validate data and write it to a database.

The file looks like

aaaa@aaa.ru;Bogdan

The file contains 500k rows. I am not able to save the file into memory, not even a temp file. I can only process InputStream or multipart file from HTTP request.

I think I need to use multi-threading or maybe Spring Batch, else before write to db I need validate emails and get the bad emails as some Arraylist.

Could you help me with that? What is the best way to do it?

ps. Spring Batch needs to file, but I can't store it in memory.

@Bean
public FlatFileItemReader<Person> personItemReader() {
    FlatFileItemReader<Person> reader = new FlatFileItemReader<>();
    reader.setLinesToSkip(1);
    reader.setResource(new ClassPathResource("/data/person.csv"));

score 0 · Answer 1 · answered Jun 14 '22 at 06:58

0

i do not able to save the file into memory, not even temp file - can only process inputstream or multipart file from Http request

The FlatFileItemReader works with any org.springframework.core.io.Resource implementation. So in your case you can use an instance of org.springframework.core.io.InputStreamResource with the InputStream of your Multipart file.

answered Jun 14 '22 at 06:58

Mahmoud Ben Hassine

28,519
3
32
50

But i cant pass it as parameter to job - thats the problem – JD_UA Jun 14 '22 at 07:10
It should be possible. If you share your web controller where you have a handle to the input stream of the Multipart file, then I will try to help. – Mahmoud Ben Hassine Jun 14 '22 at 09:25

Best way to parse multipart file (CSV) and write in DB

1 Answers1