Questions tagged [pearson]

in statistics, Pearson's r, the Pearson product moment correlation coefficient, shows the extent of a linear relationship between two data sets on a scale from -1 to 1.

Overview

Pearson product-moment correlation coefficient is given by the following equation:

enter image description here

where,

pXY = Pearson’s correlation coefficient;
Cov(X,Y) = covariance of random variables X and Y;
Var(X) = variance of random variable X;
Var(Y) = variance of random variable Y;


Tag usage

Questions on tag should be about implementation and programming problems, not about the statistical or theoretical properties of the technique. Consider whether your question might be better suited to Cross Validated, the StackExchange site for statistics, machine learning and data analysis.

155 questions
0
votes
0 answers

python read file by column without loading into memory?

I have a csv file that contains about 400 columns +100.000 lines. I m trying to run MapReduce job in HDinsight Hadoop Cluster. The Logic of my MapReduce is calculating Peason's correlation matrix. The map operation generates every possible pair of…
Anis Tissaoui
  • 834
  • 1
  • 7
  • 26
0
votes
1 answer

Pearson Correlation Coefficient different for different currencies?

I am getting very frustrated with this one problem I have about the pearson correlation coefficient. I have a program that outputs stock values for two tickers for each day for a specified period of time and I also draw a graph of those values. You…
Matheos
  • 207
  • 2
  • 12
0
votes
0 answers

grouped Pearson correlation

I have a big dataframe (shape 100 000*192). I calculated the pearson coefficient for each attribute already. Now I am looking for a way to calculate every group pearson correlations. What I mean is that for now I have if A then B and i want to…
Mayeul sgc
  • 1,964
  • 3
  • 20
  • 35
0
votes
1 answer

Pearson Correlation in SAS

I have a set of data with observations (Joe, Dana, Mark,...) and their respective ratings for a movie ( Batman - 3 Stars, Deadpool - 4 Stars). When I use the proc Corr in SAS only give the correlation between movie and not observations. How do I…
0
votes
0 answers

Recommender using a pearson correlation coefficient

I have a question about using the pearson correlation coefficient in a recommender system. I currently have 3 collections in my database. 1 for users, 1 for restaurants and 1 for reviews. I have written a function which takes 2 user id's and their…
user4189129
0
votes
0 answers

Cortest.mat converting to class "psych,sim"

I've been using the psych package to compare two correlation matrices using the function cortest. Now I want to try the cortest.mat and cortest.jennrich function which require an object of the class phychand sim. I have tried converting mi…
NKGon
  • 55
  • 8
0
votes
1 answer

error while running pearson correlations score in python

I'm new to python (PYTHON 2.7) and I'm trying to run a program that calculates Pearson correlations. The code is from the " collective intelligence" When I import the functions and run the Pearson's correlation I receive this error: >>>…
0
votes
1 answer

Pearson Correlation Zero Values

I am calculating Pearson Correlation. At the end I have the result (correlation1) like below. I wonder why I have 0.0 for all the second coefficient as a result in correlation1. Is there anybody who could explain? Moreover, my correlation code is…
serenade
  • 359
  • 2
  • 5
  • 13
0
votes
1 answer

Compute Pearson correlation avoiding zero values

I have a sparse matrix A(m,n), where n are the variables and m the observations. I want to compute the Pearson correlation among all the n variables. I have some missing observations, e.g. if A(2,3) is not available it means that I don't have such…
Eugenio
  • 3,195
  • 5
  • 33
  • 49
0
votes
1 answer

print pearson correlation residual values

I am able calculate pearson correlation between two list: import scipy from scipy import stats from scipy.stats import pearsonr List1 = [1,2,3,4,5] List2 = [2,3,4,5,6] pearson = scipy.stats.pearsonr(List1,List2) print "pearson correlation: " +…
Mea
  • 1
  • 2
0
votes
1 answer

Calculating running window Spearman correlation and pvalue in R

I wish to calculate a running window Spearman correlation in R. So far I've been using running from gtools package, but I could only get Pearson correlation out of it. I tried to modify the fun parameter but couldn't get it to produce anything but…
Adi Lavy
  • 45
  • 9
0
votes
2 answers

Corrcoef in Matlab is very slow

I have the following code: for k = 1:256 for t = 1:10000 % R matrix buffer = corrcoef(matrixA(:,k),matrixB(:,t)); correlation_matrix(k,t) = buffer (2,1); end end I…
JoeLiBuDa
  • 219
  • 2
  • 10
0
votes
0 answers

Both people rated a product with 0 star

If we have: User 1, rated product A with 0 star. User 2, rated product A with 0 star. What is the Pearson's correlation coefficient or Cosine Similarity between them? According to the formula, it should be 0/0. But what is 0/0? It is not a…
0
votes
0 answers

What's a more "pythonic way" to do this pearson calculation

Alright guys. My professor says that there is a way to do this function without the help of any loops in Python3. I'm not seeing it atm. She recommends using zip, enumerate, readlines, and split(";") (Every review is followed by a ';', if there are…
0
votes
3 answers

Turning columns into lists from csv with Python

So I am generally new to Python, and I've been reading many articles but I am still not sure how to ignore the lines with '#' . I need to: Make the four columns(col2-col5) in this tsv file into separate lists. (How would I chose to ignore the…
liya77
  • 121
  • 1
  • 10