Stack Overflow
Stack Overflow

Questions tagged [file-processing]

333 questions
0
votes
1 answer

initialise object variable from file paths

servername |-- reports | |-- ABC | | |-- COB02May2017 | | | |-- pnlreport.pdf | | | |-- balancereport.pdf | | |-- COB03May2017 | |-- CustomerB | | |-- COB02May2017 | | |-- COB03May2017 | | | …
M06H
  • 1,675
  • 3
  • 36
  • 76
0
votes
1 answer

How to read text file block by block based on header code in SSIS

I have table which consist of several header code (it always starts with HDR) Every header will have its own BHD(start) and BTR(end) tags I Need to add corresponding header value to each block as a new column. Sample Table:…
vignesh
  • 1,414
  • 5
  • 19
  • 38
0
votes
1 answer

Multicore processing of multiple files and writing to a shared output file

So, I have some 30 files each 1 GB in size and I am reading them sequentially on a MAC with 16 GB RAM and 4 CPU Cores. The processing of each one of them is independent of the others. It is taking almost 2 hours to complete the processing. Each file…
avinash shah
  • 1,425
  • 2
  • 19
  • 19
0
votes
3 answers

Spring Batch - FlatFileItemReader for different delimited files

I have two different files, one delimited by pipe separator "|" and one by comma ",". I am using Spring Batch to process these files using FlatFileItemReader. I dont want to have two readers and two writers for the files. Can I have somehow one…
Bogdan Pușcașu
  • 545
  • 2
  • 7
  • 21
0
votes
0 answers

sftp get or write file on local machine?

I want to download file from remote machine.But its 400MB.I am using get method of JSCH for SFTP. it will take 1-2hour to download? would writing file using BufferedOutputStream would be more faster.
DemoUser
  • 51
  • 3
0
votes
1 answer

File Processing- create new file with additional information

I have a file and inside it has some tables and my question is how can I create a new file but in this file it includes both tables and additional information. I tried in terminal but I could not find it, help me please...
Ekin
  • 37
  • 1
  • 2
  • 9
0
votes
2 answers

How to form triplets from a series

File1 contains a series of comma separated genes: A,B,C,D,E,F Output required is the combinations of triplets that can be formed from the series: ABC BCD CDE DEF The triplets are formed starting with the first three genes in the first triplet.…
Saisha
  • 3
  • 2
0
votes
1 answer

Read excel records of 400k rows in java quickly

I would like to read excel file of 400k rows max using java or spring efficiently.Right now we are using Apache POI to read files and process it took more than 15 minutes . Iam running out of ideas , Can anyone please help me process this huge…
adorearun
  • 151
  • 1
  • 13
0
votes
0 answers

Read csv columns based on value in first column of any row using tinycsvparser

http://bytefish.github.io/TinyCsvParser/sections/quickstart.html how can we identify on the basis of value in first column to run the mapper of any specific class. Like if value in first column is "Person" then it should be mapped with class A. and…
Abhay
  • 59
  • 2
  • 9
0
votes
2 answers

python file based program taking time to execute

I have created below Script processing two files based on user input and generating third result file. Scripts executes properly without any issue but when both file having high count then it is taking time. During my testing i have tested with…
AJNEO999
  • 37
  • 7
0
votes
1 answer

python - Issue in processing files with big size

Basically, I wanted to create a Python script for my daily tasks wherein i wanted to compare two file with any size & wanted to generated 2 new files having matching records & non-matching records from both file. I have written below python script…
AJNEO999
  • 37
  • 7
0
votes
1 answer

Java Regular Expression to read multiple directories in single statement

I wrote a method in Java, postingFile(" C:\Tables\A_env\A_Dif1.csv ") that reads the CSV file and process the data inside it. I got 26 environments, A through Z and I'm searching for a Java regular expression that can read and process data of 26…
Drithin
  • 51
  • 1
  • 4
0
votes
1 answer

Process groups of file pairs from multiple directories

I have some .txt files in dir1: file_name_FOO31101.txt file_name_FOO31102.txt file_name_FOO31103.txt file_name_FOO31104.txt and some related foo.txt files in…
fugu
  • 6,417
  • 5
  • 40
  • 75
0
votes
1 answer

How to split a file in spring batch based on no of lines?

I have a file like this: abc.txt 1 fhdfjdkslk 2 jskslsss 3 iowiwjwm 4 jwkwlwo 5 uuwwwmwaa 6 sbnsbshj 7 yywuiwoiw 8 dhdhgsgs 9 sgsgsgs 10 uwuwqmksam I want to split this file using Spring Batch in a configurable way and read the file in a…
IndrajitDatta
  • 205
  • 4
  • 13
0
votes
1 answer

How to extract one column from multiple files and paste them columwise into a new file

The input filenames look like this: 1012232U1_000n_1012232U1.TXT with n=1...n and contain five colums separated by diffrerent delimeters 200,00; 441,080; 353,651; 404,764;212,94154 201,00; 413,333; 402,073; 422,487;55,48304 202,00; 449,286;…
Corny_cheesecake
  • 1
1 2 3
…
22 23