Stack Overflow
Stack Overflow

Questions tagged [data-processing]

Data Processing concerns the converting of raw data to machine-readable form and its subsequent processing (as storing, updating, rearranging, or printing out) by a computer.

Data Processing concerns the converting of raw data to machine-readable form and its subsequent processing (as storing, updating, rearranging, or printing out) by a computer.

More Info

909 questions
-2
votes
1 answer

Why does my variable show up as NoneType in Variable Explorer shows up as matrix in command window

So I am working with a dataset that I need to get the sampling frequency in each column this is the function I am using def downsample_happysongs(): n = 8 # number of channels sample_freq = 128 # sample frequency used in dataset data1…
PeruvianNaplord
  • 27
  • 1
  • 5
-2
votes
1 answer

how to edit dataframe based on some condition?

I have two data frames plotted which comes something like this. I want to get a new data frame such that whenever these two curves are crossing each other just put 1 or -1 at those places. Where positive sign could be like when the blue curve…
user13079994
  • 15
  • 4
-2
votes
2 answers

select rows from pandas data frame based on another column % percentage of occurrence

I have a data set like this 1A1HI_R071_PH_INSPECT_VIS_1_2_201231_025816.JPG 1A 1A1HI 1A1JK_R071_PH_INSPECT_VIS_1_2_210115_121554.JPG 1A 1A1JK 1P3G6_R071_PH_INSPECT_VIS_2_2_201231_034741.JPG 1P 1P3G6 1P3GC_R071_PH_INSPECT_VIS_3_2_201107_140047.JPG…
Arjun
  • 49
  • 9
-2
votes
2 answers

Faster processing of large files

//string filename = "123.txt"; foreach (var files in Directory.GetFiles(pathToDir, "*.*", SearchOption.TopDirectoryOnly)) using (var fsIn = new FileStream(files, FileMode.Open, FileAccess.Read)) using (var fsOut = new FileStream($"{files}.crypt",…
ZidoX
  • 122
  • 7
-2
votes
1 answer

Merging 300 variables with Python

Five lines are correct : GO #Fixed EXEC sys.sp_cdc_enable_table #Fixed @source_schema = N'RWAW3', #Fixed @source_name=N'TUY1 #Variable @role_name = NTEDULL #Fixed I have…
majed
  • 1
  • 5
-2
votes
2 answers

Explain 'Multi-col-linearity' in data processing

Can anyone please explain the Multi-col-linearity term from machine learning data processing, in simple words. As the term is very important with respect to data processing and has confusing explanations. So, please explain it in simple words as I…
hacknoid
  • 30
  • 6
-2
votes
3 answers

How to normalize the data in a dataframe in the range [0,1]?

I'm trying to implement a paper where PIMA Indians Diabetes dataset is used. This is the dataset after imputing missing values: Preg Glucose BP SkinThickness Insulin BMI Pedigree Age Outcome 0 1 148.0 72.000000 35.00000 …
user11439817
-2
votes
1 answer

Finding most common value seperated by day

I want to see which category occurs most often each day per participant. There are multiple categories which occur each day and I want a new column which states the category which occured mostly that specific day for a specific participant. I have a…
Marly
  • 3
  • 2
-2
votes
2 answers

How many types of pre- processing techniques have in machine learning before split the train and test the dataset?

How many types of different pre-processing techniques are exists in machine learning for data-sets. I want to define the general template that help me in pre-processing on the different problem. I done some operation on image classification…
Rahul Savariya
  • 1
  • 5
-2
votes
1 answer

pandas: fillna with a value that can indicate it used to be null

i am using a dataset to build a model in machine learning. In the samples, there are 3 categories of labels like "abnormal" "normal" "data lost" . It is the category "data lost" that confuse me. In the samples, this category means that some features…
Kamook
  • 49
  • 9
-2
votes
2 answers

Summing up the total based on the random number of inputs of a column

I need to sum up the "value" column amount for each value of col1 of the File1 and export it to an output file. I'm new in python and need to do it for thousands of records. File1 col1 col2 value 559 1 91987224 2400000000 559 0 …
Rubz
  • 95
  • 8
-2
votes
2 answers

Effect of data type of arguments in function call on processing speed

uint16_t fn1 (uint8_t a, uint8_t b) { return (a+b) } uint32_t fn2 (uint32_t a, uint32_t b) { return (a+b) } Which is more faster on a 8-bit and a 32-bit micro controller? Any equivalent assembly code to demonstrate the difference. For example, …
Chandan Kumar
  • 17
  • 1
  • 5
-2
votes
2 answers

How to read .xls file with multiple header in R / Python for data processing?

I have this file which is specified below, As you can see it consists of many header layers, how can i read this file in R / Python so that i could get it in proper format for processing it?
Nitz
  • 81
  • 1
  • 4
-2
votes
2 answers

How to insert tab in a sequential word in python?

I have a problem with a very large text file which looks like following: A T T A G C A A AT A G C A T TT AG G A G T T A G C A Every character was split by \t,but some characters are connected, I want to add \t to these sequence. What I need is like…
Zaichao Sheng
  • 9
  • 2
-2
votes
1 answer

Script for extracting information of specific pattern from a text file

Hi I am working on a project which deals with large amount of data. I have a text file of around 2 GB with key value pairs and each key has multiple values. I need to extract all the keys in a different file, as I need the keys for testing a…
user2621826
  • 19
  • 2
  • 11
1 2 3
…
60
61