Fast ICA using scikit learn- reconstruction error analysis

Question

I am trying to use fastICA procedure in scikitLearn. For validation purposes I tried to understand the difference between PCA and ICA based signal reconstruction.

The original number of observed signals are 6 and I tried to use 3 reconstruction independent components . The problem is that both ICA and PCA result in the same reconstruction errors no matter what norm I use. Can some one throw light into what is happening here.

The code is below:

 pca = PCA(n_components=3)
 icamodel = FastICA(n_components=3,whiten=True)

 Data = TrainingDataDict[YearSpan][RiskFactorNames]

 PCR_Dict[YearSpan] = pd.DataFrame(pca.fit_transform(Data), 
                                   columns=['PC1','PC2','PC3'],index=Data.index)

 ICR_Dict[YearSpan] = pd.DataFrame(icamodel.fit_transform(Data), 
                                   columns=['IC1','IC2','IC3'],index=Data.index)

'------------------------Inverse Transform of the IC and PCs -----------'

 PCA_New_Data_Df = pd.DataFrame(pca.inverse_transform(PCR_Dict[YearSpan]),
                                   columns =['F1','F2','F3'],index = Data.index)

 ICA_New_Data_Df = pd.DataFrame(icamodel.inverse_transform(ICR_Dict[YearSpan]),
                                   columns =['F1','F2','F3'],index = Data.index)

Below is the way I measure the reconstruction error

'-----------reconstruction errors------------------'
 print 'PCA reconstruction error L2 norm:',np.sqrt((PCA_New_Data_Df - Data).apply(np.square).mean())

 print 'ICA reconstruction error L2 norm:',np.sqrt((ICA_New_Data_Df - Data).apply(np.square).mean())

 print 'PCA reconstruction error L1 norm:',(PCA_New_Data_Df - Data).apply(np.absolute).mean()

 print 'ICA reconstruction error L1 norm:',(ICA_New_Data_Df - Data).apply(np.absolute).mean()

Below are the description of the tails of the PC and ICs

PC Stats :  ('2003', '2005') 
       Kurtosis  Skewness
PCR_1 -0.001075 -0.101006
PCR_2  1.057140  0.316163
PCR_3  1.067471  0.047946 

IC Stats :  ('2003', '2005') 
       Kurtosis  Skewness
ICR_1 -0.221336 -0.204362
ICR_2  1.499278  0.433495
ICR_3  3.654237  0.072480

Below are the results of the reconstruction

PCA reconstruction error L2 norm: 
SPTR        0.000601
SPTRMDCP    0.001503
RU20INTR    0.000788
LBUSTRUU    0.002311
LF98TRUU    0.001811
NDDUEAFE    0.000135
dtype: float64 

ICA reconstruction error L2 norm : 
SPTR        0.000601
SPTRMDCP    0.001503
RU20INTR    0.000788
LBUSTRUU    0.002311
LF98TRUU    0.001811
NDDUEAFE    0.000135

Even the L1 norms are the same. I am a bit confused!

score 0 · Answer 1 · answered Dec 10 '20 at 12:32

Sorry for the late reply, I hope this answer can still help you.

fastICA can be seen as whitening (which can be achieved by PCA) plus an orthogonal rotation (an orthogonal rotation such that the estimated sources are as non-gaussian as possible).

The orthogonal rotation does not affect the reconstruction error of the ICA solution and hence you have the same reconstruction error for PCA and ICA.

Rotating a PCA solution is often used within psychology (for example a Varimax rotation). The orthogonal rotation matrix in fastICA is however estimated with an iterative procedure (fixed-point iteration scheme from Aapo Hyvärinen)

Fast ICA using scikit learn- reconstruction error analysis

1 Answers1