Questions tagged [standardized]

Shifting and rescaling data to assure zero mean and unit variance.

Overview

Specifically, when xi, i =1,..., n is a batch of data, its mean is:

m=∑xi/n

and its variance is:

s2 = ∑(xi−m)2)/ν

where,

v is either n or n-1 (choices vary with application).

Standardization replaces each xi with zi = (xi-m)/s. Do not confuse standardization with normalization.


Tag usage

Questions on tag should be about implementation and programming problems, not about the statistical or theoretical properties of the technique. Consider whether your question might be better suited to Cross Validated, the StackExchange site for statistics, machine learning and data analysis.

145 questions
0
votes
1 answer

How to scale test set based on the mean and std from train set in python?

I read an answer to explain "Why feature scaling only to training set?" " and the answer says "Standardize any test set using the training set means and standard deviations" Therefore, I try to fix my previous incorrect operation. However, I check…
Carlos
  • 167
  • 1
  • 2
  • 14
0
votes
0 answers

R vector calculate incorrect zscore

I have a dataset like the following: > Pricing ID Paper Type Weight Quantity Unit.quote.price Total.quote.price Material.cost 1 1 SRA3 2 1 1 2.600000 2.60000 0.04104 2 2 SRA3 2 1 2 …
0
votes
2 answers

How to obtain standardized betas from a svyglm() regression

I have a svyglm() weighted linear regression model and would like to obtain the standardized betas of the regression coefficients. I have tried to get them with the lm.beta() function, however when I do this the p-values of the coefficients change.…
cholo.trem
  • 314
  • 2
  • 9
0
votes
0 answers

how can I rescale data following a bernoulli distribution

I have read various articles talking about standardization and normalization but none of the offers a concrete example on how to rescale data using their formulas. I would like to transform data as follows; given data = [x1...xn] rescale(data,n)…
MQXXK
  • 1
  • 2
0
votes
1 answer

Feature scaling in an incremental analysis

I'm doing an incremental analysis of my data. The data belongs to 4 age groups (day1, day2, day3 and day4). Before I feed my data to the model, I standardize the features using the standard scaler implementation in sklearn. When I think of it, 3…
0
votes
1 answer

Standardized and simple WordPress pagination options

I want my Wordpress pagination to consistently have 5 numbers in it. I want it to always be: < previous 1 2 3 4 ... 37 next > < previous 5 6 7 8 ... 37 next > While I know there are several ways of developing this pagination and countless different…
Jamie
  • 1,909
  • 3
  • 22
  • 47
0
votes
1 answer

Standardize variables in Python

I have the below code: var_list = ['a', 'b', 'c', 'd', 'd', 'e', 'f', 'g', 'h', 'i'] y_var = 'lp' for x_var in var_list: formula = y_var + ' ~ ' + x_var results = smf.ols(formula, data=df).fit() I would like to standardize the…
Parvesh
  • 13
  • 6
0
votes
1 answer

How can I normalize dataframe in pyspark?

I'm trying to normalize user-item matrix, but I want to use this formula: (df.values-df.values.min())/(df.values.max()-df.values.min()) in Dataframe like this.
bmery77
  • 11
  • 2
0
votes
1 answer

Reverse Z Scores

If you have a data vector you are given that contains only z-scored variables is it possible to reverse the z-score values to get the original measure without using the original measure? In R Program: a = runif(100) az = (a - mean(a))/sd(a) Can you…
ajpat
  • 21
  • 1
  • 8
0
votes
0 answers

How to use PCA with data of values with same units

My data consists of 400000 samples with 3200 values of same units (400000x3200). I know, when the data has values of different size, first you have to standardize or normalize the data, so the scale of each value is the same. But how to calculate…
Len01
  • 31
  • 5
0
votes
3 answers

Different results with PCA and SVD on data with same units

At the beginning, i had 400,000 images that were normalized (gray value increase). After that i did a DFT of each picture and got data of 400000 samples with 3200 absolute fourier-coefficients. Now I would like to do a PCA and SVD. Since my data is…
Len01
  • 31
  • 5
0
votes
1 answer

Standardized Betas for Panel Data in R

I have a panel dataset and I'm running a fixed effects regression. My dependent variable is CDS Spreads and I have 7 independent variables which are macroeconomic variables (GDP, Inflation etc) and then I have ratings data for three agencies which…
Arsh
  • 11
0
votes
0 answers

Normalizing data where one column contains discrete subsets of values (in R)

I understand this may be a rearranging issue on my end, but I just can't visualize how best to do this. I can follow steps included in other normalize questions to use like here and here, but my issue is that my data (pasted a sample below) is…
sleepy
  • 93
  • 9
0
votes
0 answers

Power Function Normal Population

I am looking to calculate the power function from a normal population with mean μ and known variance σ². I'm trying to modify the "help" code: ## Power at mu=105 for H0:mu=100 vs. H1:mu>100 (sigma=15) 20 obs. (alpha=0.05) sigma<-1 c<-0 …
mapleleaf
  • 758
  • 3
  • 8
  • 14
0
votes
1 answer

How to use feature scaling in multivariate time series data?

I am trying to do classification on multivariate time series data. I want to perform feature scaling on this kind of data. Suppose the multivariate time series data looks like this: F1 F2 F3 …