0

I've come across an issue with pandas subtraction that confuses me. I am trying to do subtraction of one column with another. My data that is used are the residuals of arch_model, and ARIMAresult.

Data as follows

Heres the issue. When i perform pandas subtraction, my whole column gets altered to a single value (value of first row) instead of doing what it is supposed to.

residuals.fitted_garch = residuals.arma_resid - residuals.garch_resid

output

            arma_resid  garch_resid  fitted_garch
Gmt time                                         
2010-01-02    0.000012    -0.000004      0.000016
2010-01-03   -0.004926    -0.004943      0.000016
2010-01-04   -0.000392    -0.000408      0.000016
2010-01-05   -0.001888    -0.001905      0.000016
2010-01-06    0.002209     0.002193      0.000016
...                ...          ...           ...
2019-12-27   -0.000043    -0.000059      0.000016
2019-12-28    0.000007    -0.000009      0.000016
2019-12-29   -0.002366    -0.002382      0.000016
2019-12-30   -0.003173    -0.003189      0.000016
2019-12-31   -0.000101    -0.000117      0.000016

Both residuals are pandas series

Would greatly appreciate if you could explain whats going on and provide advice on the fix.

Follow up - Code, dataframe, and result

From the screenshot above, we can see that the result of all the rows in the column is the value of the cell in the first row.

Anthony
  • 25
  • 1
  • 4
  • Hi, have you specified the "axis" parameter correctly? Check the documentation here: https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.subtract.html – GabrielP Sep 06 '21 at 20:01
  • Would greatly appreciate too if you update your post with plain text code as sample please :) – Corralien Sep 06 '21 at 20:01
  • I've uploaded a screenshot which shows the problem. The issue being that the column values are the result of the first cell. This is not what I intended the code to do. I'm trying to get `df.column.C = df.column.A-df.column.B` – Anthony Sep 07 '21 at 13:04

1 Answers1

0

Looking at your picture you posted I think the problem might be coming from iterating through the data. Typically when subtracting one column from another you can just subtract the whole column at once and pandas will know your doing vector subtraction.

residuals['fitted_garch'] = residuals['arma_resid'] - residuals['garch_resid']

Here you don't need to initalize the column with np.nan either and can just create the new "garch_resid" column in place.

bellerb
  • 137
  • 8
  • `residuals.fitted_garch = residuals.arma_resid - residuals.garch_resid` Got the same output where all values in column is the value of the first cell. – Anthony Sep 07 '21 at 16:44