Stack Overflow
Stack Overflow

Questions tagged [data-processing]

Data Processing concerns the converting of raw data to machine-readable form and its subsequent processing (as storing, updating, rearranging, or printing out) by a computer.

Data Processing concerns the converting of raw data to machine-readable form and its subsequent processing (as storing, updating, rearranging, or printing out) by a computer.

More Info

909 questions
0
votes
2 answers

Looking for well-logged Make output

Shorter question: Make targets have files as dependencies; let's say one example dependency is the file "D." I would like Make to traverse its dependency graph, and for each "D," also depend on success being recorded in a log file of "D's" recipe's…
Alex Nelson
  • 76
  • 6
0
votes
1 answer

Adjust data vector to have certain variance

I have a set of integers x, 0<=x<=255 I need to transform this data in that way that : Average of values in set == 0 Variance == 1 I can meet first condition with: array arr[]; av = average(arr); foreach(x in arr) { x = x - av;} But I do not…
Tomáš Šíma
  • 834
  • 7
  • 26
0
votes
1 answer

Determine coefficients for some function

I have a task that is probably related to data analysis or even neural networks. We have a data source of our partners, job portal. The source values are arrays of different attributes related to the particular employee: His\her gender, Age, Years…
Spaceman
  • 1,185
  • 4
  • 17
  • 31
0
votes
0 answers

Preparation of categorical data for hierarchical clustering

I would like to use R to perform a hierarchical clustering of data that looks like this: L1 L2 L3 W1 p pr r W2 p NA r which is supposed to mean that L2 shares feature W1 with both L1 and L3, while feature W2 is present in L1…
Kamil S.
  • 406
  • 3
  • 10
0
votes
1 answer

Processing a large amount of data using jobs in php

We have a website that has 'projects' on it with a given number of people subscribed to each project. At the 'end' of a project, all the subscribers are gathered and processed. In this case, there are roughly 1,000 subscribers whose data needs to be…
Barry Chapman
  • 6,690
  • 3
  • 36
  • 64
0
votes
4 answers

Fastest Way of Storing Data

I have a server which generates some output, like this: http://192.168.0.1/getJPG=[ID] I have to go through ID 1 to 20M. I see that most of delay is in storing file, currently I do store every request result as a separate file in a folder. in form…
Vahid Farahmand
  • 2,528
  • 2
  • 14
  • 20
0
votes
3 answers

Best to processing large arrays in PHP with my date wise case

I have a large array in PHP, having near around 168000 keys and values. There is date (Y-m-d) and hour in key and numeric value in value. So value is just a numeric. And key is in Y-m-d_H format. Array looks like following: $input =…
user987346
0
votes
0 answers

Reporting across multiple CSV files

This might be a vague question. I am given 4 CSV files with about 500k row in each of them on a daily basis. I need to perform 'join' and 'where' equivalent RDMS operations on them to create daily reports. For example, the work flow could be: Join…
simon604
  • 60
  • 1
  • 6
0
votes
2 answers

Separating strings with commas in Matlab

I'm looking to separate strings in a text file with commas in Matlab, so far I've used "csvwrite" and "dlmwrite". They generally take this form: myFile - input ('Please enter file's directory','s'); readMatrix -…
Tony Hematite
  • 149
  • 3
  • 14
-1
votes
1 answer

PHP bulk processing solution

I'm looking for a PHP component for asynchronous data processing. Basically what I need is to display a page with a progress bar that's refreshed with javascript which displays the progress on some data processing. On the backend you'll define your…
Flupkear
  • 2,135
  • 7
  • 29
  • 32
-1
votes
0 answers

Anyone familiar with a computer language to control ThermoFisher .raw files?

I use ThermoFisher .raw files to explore materials that absorb UV light, and I use a specific program called XCalibur Qual Browser. I called ThermoFisher Customer support and they said that there is no in-house computer language that can be used to…
calderod
  • 1
-1
votes
2 answers

How can I read and write CSV files and process the data into arrays in Java?

I am working on a Java project where I need to handle CSV files. Specifically, I need to read and write CSV files and process the data into arrays for further manipulation. I have researched different libraries and approaches, but I am unsure about…
DamianBautista
  • 1
  • 2
-1
votes
1 answer

extracting values matching timestamps by a new set of timestamps

sample table here i am trying to look up corresponding commodity prices from columns(CU00.SHF,AU00.SHF,SC00.SHF,I8888.DCE C00.DCE), with a new set of timestamps, the dates of which are 32 days later than the dates in column 'history_date'. i tried…
Arthur Zhang
  • 107
  • 8
-1
votes
1 answer

Pandas: Combine consecutive months having same values in other column

I have monthly performance of students for several years for all subjects. DataFrame has following columns: [Name, Subject, Month, Year, Marks] as given in following image 1: Name Month Year Subject Marks 0 A 1 2022 Math 80 1 …
RVD
  • 13
  • 2
-1
votes
1 answer

Time series data

I have an Excel file and there are two columns in it, I want to combine them, but one of them is in datetime form and the other is object (actually time). What I want to do is convert the object one to datetime format.enter image description…
haynick
  • 1
  • 1
1 2 3
…
60 61