Stack Overflow
Stack Overflow

Questions tagged [file-processing]

333 questions
2
votes
3 answers

Remove last line from all files of a specific extension

I have several files with same extension .txt in a directory. I want to remove the last line from all .txt files. What I did is find . -type f -name '*.txt' -exec sed '$d' {} \; This prints the desired output from sed to the terminal for every .txt…
flappingwings
  • 65
  • 1
  • 7
2
votes
4 answers

Fastest and best method for importing file contents into a database

I have folders where approx 3000 new csv files come in on a daily basis, each containing between 50 and 2000 lines of information. Currently, there is a process in place which picks these files up one at a time and takes each line one at a time and…
Graeme
  • 1,657
  • 5
  • 28
  • 46
2
votes
0 answers

How do I find a specific line from a large data file using Fortran

I have been working with Fortran for my physics project and I have to read data from a file. I need to find a specific matrix and then print the said matrix onto a different file and diagonalize it. How do I read the matrix that follows a specific…
Kranthi Aditya
  • 21
  • 2
2
votes
3 answers

Higher level file read/write with Objective-C

As Objective-C is superset of C, I guess I can use fopen/fread/fwrite/fprint... just like I do with C. Does that mean that Objective-C doesn't have its unique file processing function?
prosseek
  • 182,215
  • 215
  • 566
  • 871
2
votes
3 answers

.NET Webservice processing large file is having its thread aborted

Let give you the details of this rather odd issue. We have a very large text file to process, it is 780MB and is likily to get bigger. The processing of this file lies in a .NET WebService, a request is sent from a Asp.Net Website client and the…
MrJoeBlow
  • 466
  • 1
  • 5
  • 13
2
votes
3 answers

How can I loop through specific elements of a list in Bash?

I have a list of files stored in a variable obtained by entering files="./*.fasta" I would like to create a for loop that will loop through: the first 200 elements, elements 201-400, elements 401-578, for example. How can I achieve this? I tried…
chrispbio
  • 39
  • 7
2
votes
1 answer

Matlab load function timeout

I would like to increase the matlab load command timeout as I'm accessing a resource over the network (on Windows). The default is 10s as mentioned here. How can this be achieved? related: Matlab read from fifo with fopen timeout
user2305193
  • 2,079
  • 18
  • 39
2
votes
2 answers

How to remove a line after another line in a file with ansible?

My ansible inventory contains a group that has dynamically generated IP addresses. [asg] 10.15.100.13 10.15.100.14 Is it possible to remove the line after [asg]? (i.e. 10.15.100.13) The IP addresses in this group refers to the ec2 instances in an…
Brian
  • 12,145
  • 20
  • 90
  • 153
2
votes
3 answers

Python function to read variable length blocks of data from file while open

I have data files that contain data for many timesteps, with each timestep formatted in a block like this: TIMESTEP PARTICLES 0.00500103 1262 ID GROUP VOLUME MASS PX PY PZ VX VY VZ 651 0 5.23599e-07 0.000397935 -0.084626 -0.0347849…
jpmorr
  • 500
  • 5
  • 25
2
votes
1 answer

Clojure : why does this writer consume so much heap space?

I have a 700 mb XML file that I process from a records tree to an EDN file. After having do all the processing, I finally have a lazy sequence of hashmaps that are not particularely big (at most 10 values). To finish, I want to write it to a file…
Joseph Yourine
  • 1,301
  • 1
  • 8
  • 18
2
votes
3 answers

Unix - show some characters in file by line number

I have a very huge file and need to look at a few characters in the middle of some huge line. Is there a way to show easily characters from n1 position to n2 position in line number l in some file? I think there should be some way to do it with sed,…
Maxim Galushka
  • 397
  • 1
  • 5
  • 22
2
votes
2 answers

MATLAB: Finding files within a directory with specific names?

If I have files with multiple names and I want to select only few from them, how can I do that in MATLAB. For e.g. I have the following file names in a directory 1adl.txt, 2adl.txt... 1adlKey.txt, 2adlKey.txt ... where *adl.txt are the files…
Shehroz
  • 347
  • 2
  • 9
  • 25
2
votes
2 answers

How to search a file for a pattern and get a new file from the match point to end of file?

I need to take a file and find the first occurrence of a literal string pattern as a complete line of the file: Acknowledgments: And then I wish to create a new file from the line of match all the way to the end of the file. I expect perl is a good…
WilliamKF
  • 41,123
  • 68
  • 193
  • 295
2
votes
4 answers

python: text file processing, for loop and read operation conflict

I am learning to code in Python. Now I am experimenting with a file comparison program from here. My code is: #!/usr/bin/python3 def main(): fhand1 = open('mbox.txt') print('file handle for mbox is {}'.format(fhand1)) count = 0 for…
algoProg
  • 718
  • 2
  • 11
  • 27
2
votes
1 answer

Spring Integration File Polling. If moving the file does a AcceptOnceFileListFilter need to be used?

I'm writing a file polling implementation and am trying to determine if I need to use a AcceptOnceFileListFilter. The first step the FileProcessor will perform is to move the file to another directory. Does the poller "batchFilePoller" use multiple…
Michael Freeman
  • 330
  • 1
  • 4
  • 14
1 2 3
…
22 23