Stack Overflow
Stack Overflow

Questions tagged [file-processing]

333 questions
1
vote
1 answer

PowerShell Total File Count And Count Files Newer Than A Date

I want to get total file count and count of files newer than a specific date without having to do 2 separate calls. Is there a way do get both of these counts in 1 call? Inefficient Way: cls $compareDate = (Get-Date).AddDays(-365) $FileShare =…
Doug Punchak
  • 130
  • 3
  • 11
1
vote
1 answer

How to use iterator pattern in java to load file into hashmap in batches

I have a large file containing two million lines . I'm looking to traverse through each line of the file and, process it into a key value pair and store it into a hashmap to make comparisons later on. However, I do not want to have a hashmap with 2…
Devanshi_K
  • 13
  • 3
1
vote
3 answers

How I can improve the performance of this file processing algorithm finding matches in a large textfile?

I have a Textfile containing Gigabytes of integer triples: 357, 1325, 7085 448, 952, 1073 459, 555, 2091 756, 765, 925 765, 925, 3485 792, 1560, 3315 952, 1073, 1105 975, 1073, 1105 990, 1950, 2146 My task is to find quadruples a b c d out of two…
Eldar Sultanow
  • 205
  • 2
  • 9
1
vote
4 answers

How can I properly parse my file? (Using break/continue)

I have the following data that looks like this for example: 34 foo 34 bar 34 qux 62 foo1 62 qux 78 qux These are sorted based on the first column. What I want to do is to process lines that starts with 34, but I also want the file…
neversaint
  • 60,904
  • 137
  • 310
  • 477
1
vote
0 answers

Why is the while loop not executing to compare strings

First, the while loop is not comparing titles in the file and the num_books does not add on top of the file. Second, instead of reading the num_books already in the file, it reads twice every time executed. can someone help? typedef struct book { …
user17591552
  • 19
  • 5
1
vote
1 answer

How to ensure a VERY delayed AJAX call is read properly

I have a File which process many other files and may take upto 30mins to process. I do an AJAX request to the file in the front end. The file outputs to another temporary file regarding the percentage of completion. And when it finishes it outputs a…
footy
  • 5,803
  • 13
  • 48
  • 96
1
vote
1 answer

Reading the contents of a file into a buffer in C

I'm working on a program which reads every integer in csv file and copies it into a buffer so that I can later use it to construct a binary search tree with it. I'll show my code, then I'll explain the issue I'm having: Code - int *createBuffer(int…
Anish Sinha
  • 175
  • 3
  • 13
1
vote
2 answers

Reading and processing a file in two separate threads works twice slower than one thread

I solve a task of counting unique lines in a text file. Each string is one valid ip-address. The file can be of any size (literally, hundreds and thousands of gigabytes are possible). I wrote a simple class that implements a bit array and using it…
chptr-one
  • 600
  • 2
  • 15
1
vote
1 answer

loop through the lines which are specified through a parameter

I have a chunk of code below which reads from a file in s3 and processes all the lines. The file doesn't have any line numbers in it. However, I want the code to process only specific lines of file. How should I approach that? Note: The start line…
Dark Matter
  • 87
  • 2
  • 11
1
vote
2 answers

Need an advice on text-processing flow

I've been a research programmer (MATLAB) for most of my programming career, writing things for only myself that can be run on my own computer. Now, I'd like to be able to have people submit a comma-delimited text file and get processed text files in…
laketuna
  • 3,832
  • 14
  • 59
  • 104
1
vote
1 answer

batch file to copy files from one server to other adding remote directory to filename in process

I've written a batch file to copy files from one server to another, however, i need to be able to rename the file just copied to contain the folder path. The code i have come up with to do the job is: ECHO OFF SETLOCAL EnableDelayedExpansion set…
Jarede
  • 3,310
  • 4
  • 44
  • 68
1
vote
0 answers

Create SPRING BATCH JOBS in RUN TIME and Execute them

I have a requirement where i will take input CSV files from one folder, process them (DB lookup and validation) one after one and generate new output file for each input file. I need to choose the input files at RUN TIME based on the DB query on…
Shubha N
  • 11
  • 2
1
vote
1 answer

Is there a way to process data from a text file containing main headings using a regular expression?

Below is a snippet of the text file format structure Historical Sales for: 12th of October 2019, 11:37 am PRODUCT NAME QUANTITY Coke B 5 Historical Sales for: 21st of October 2019, 8:15 pm PRODUCT NAME QUANTITY Peanuts …
Marome
  • 47
  • 1
  • 11
1
vote
1 answer

Process .dat file as dataframe in Python with irregular spacing

I have a .dat file with values separated by spaces. However, the spaces are irregular (some columns separated by one space, some by eight spaces) and some columns also have strings with spaces in them. For example, the first five lines look like…
Smith
  • 11
  • 2
1
vote
0 answers

Adding integer from a text file, but it prints out something else

I'm trying to insert a data from text file into a linked list, the data contains ID-name 12345678-abcdefgh But i don't know why the ID always print strange numbers, and i can't seem to find any errors in my code. Do anyone have any idea what makes…
Christian Halim
  • 117
  • 7
1 2 3
…
22 23