Questions tagged [standardized]

Shifting and rescaling data to assure zero mean and unit variance.

Overview

Specifically, when xi, i =1,..., n is a batch of data, its mean is:

m=∑xi/n

and its variance is:

s2 = ∑(xi−m)2)/ν

where,

v is either n or n-1 (choices vary with application).

Standardization replaces each xi with zi = (xi-m)/s. Do not confuse standardization with normalization.


Tag usage

Questions on tag should be about implementation and programming problems, not about the statistical or theoretical properties of the technique. Consider whether your question might be better suited to Cross Validated, the StackExchange site for statistics, machine learning and data analysis.

145 questions
2
votes
1 answer

How to standardize selected columns in panel data frame

How do I standardize SELECTED columns in a data frame, when the data frame is a panel? Lets say, that I only want to standardize columns X2, X4, X6, X7, and X9 by explicitly selecting them in the sample data frame below, while I account for the…
BeSeLuFri
  • 623
  • 1
  • 5
  • 21
2
votes
2 answers

python pandas standardize column for regression

I have the following df: Date Event_Counts Category_A Category_B 20170401 982457 0 1 20170402 982754 1 0 20170402 875786 0 1 I am preparing the data for a regression…
jeangelj
  • 4,338
  • 16
  • 54
  • 98
2
votes
0 answers

standardization of a tensor in tensorflow

I have the following functions to make standardization/inversion of the input/output to a list of values. Now I want to inverse-apply it to a tensor, so that I can write loss value in "real" units. Obviously, this can't be applied to a tensor…
noname7619
  • 3,370
  • 3
  • 21
  • 26
2
votes
1 answer

Adding Interaction Terms to MATLAB Multiple Regression

I am currently running a multiple linear regression using MATLAB's LinearModel.fit function, and I am bit confused in regards to how to properly add interaction terms to the model by hand. As I am aware, LinearModel.fit does not standardize…
dwm8
  • 309
  • 3
  • 16
2
votes
1 answer

Standardisation in MuMIn package in R

I am using the 'MuMIn' package in R to select models and calculate effect sizes of the input variables (rain, brk, onset, wid). To make my effect size comparable between variables, I standardised them using standardize function in arm package. Here…
user3013423
2
votes
1 answer

R caret glmnet standardize = FALSE

I'm trying to use the caret package to play with alpha levels for a glmnet model. The problem is that the data I'm using is all dummy variables and I don't want glmnet to standardize them. Usually if I was just using glmnet or cv.glmnet on its…
screechOwl
  • 27,310
  • 61
  • 158
  • 267
1
vote
0 answers

Dark photos with standardized pixels between 0 and 1 from a Tensorflow Dataset

In order to create a n. network that processes images I have to standardize the photos present in the tensorflow dataset called "stanford_dogs" relating to photos of different breeds of dogs. I tried different techniques to standardize the images…
1
vote
1 answer

Automate z-score calculation by group

I have the following data frame: df<- splitstackshape::stratified(iris, group="Species", size=1) I want to make a z-score for each species including all of the variables. I can do this manually by finding the SD and mean for each row and using the…
hugh_man
  • 399
  • 1
  • 6
1
vote
0 answers

How to normalize data for one column based on population

I have the dataset below: ## ID DOB sector meters Oct Res_FROM Res_TO Exp_FROM ## 1 20100 1979-08-24 H38 6400 W 1979-08-15 1991-05-15 1979-08-24 ## 2 20101 1980-05-05 B01 1600 NW 1980-05-15 1991-04-15 1980-05-15 ## 3…
barnsm2
  • 185
  • 7
1
vote
1 answer

Calculating several new variables from existing pairs and standardising new variable values against other variables in R

I'd like to create new [word]_c variables from pairs of variables, subtracting variable_b from variable_a, but as there are 50 pairs, it would help to be able to do this without having to write out every name. Once I have the [word]_c columns, I'd…
1
vote
3 answers

Standardizing a set of columns in a pandas dataframe with sklearn

I have a table with four columns: CustomerID, Recency, Frequency and Revenue. I need to standardize (scale) the columns Recency, Frequency and Revenue and save the column CustomerID. I used this code: from sklearn.preprocessing import normalize,…
1
vote
0 answers

Mutual Info scoring (MI Scoring) on standardized features in Python

I'll try not to beat around the bush. How would you approach, if you had imputed a lot of numerical features with k-Nearest Neighbors imputer and then wanted to get some MI scores on all of your features? As you can't store mentioned numerical…
Chris
  • 41
  • 1
1
vote
1 answer

standardize categorical value

Right now I am working on open datasets, one task is to 'standardize' values. We only care about 'age', 'gender', 'race', 'ethnicity', and 'country' attributes. For 'Country', we could have 'united-states', 'united states','usa', 'us' as valid…
Max Wang
  • 21
  • 5
1
vote
2 answers

Standardize features to calculate variance inflation factors

I'm calculating variance inflation factors from patsy import dmatrices from statsmodels.stats.outliers_influence import variance_inflation_factor y, X = dmatrices('A ~ B + C + D + E + F + G, data=df, return_type='dataframe') vif =…
Andy S.
  • 21
  • 1
  • 5
1
vote
0 answers

Python Standardize specific data Sckit-Learn

I want to standardize my dataset to a specific range (5.5,7.5). How can I do this? ı try feature_range parameters,but I could not succeed
1 2
3
9 10