68

When I tried to read a pickle file that saved by a former version of pandas, it yielded an ImportError.

ImportError: No module named 'pandas.core.internals.managers'; 'pandas.core.internals' is not a package

There was no hit on stackoverflow so i would like to share my solution for this particular problem.

Fatih1923
  • 2,665
  • 3
  • 21
  • 28

5 Answers5

71

This error comes off due to encoding of formerly saved pickle file. If you updated pandas to newly amended version, it produces this import error.

Fatih1923
  • 2,665
  • 3
  • 21
  • 28
  • 18
    Update pandas by running `python3 -m pip install --upgrade pandas`. – cjohnson318 Feb 20 '19 at 22:49
  • 1
    Does this seriously mean there is no way to open pickle files generated with pandas 0.23 in pandas 0.24 nor is there a way to convert between the two formats? – Lorenz Walthert Apr 16 '19 at 13:03
  • 3
    @LorenzWalthert Actually, there is a back door. It sounds dummy but it really works. Open your `pickle` file with `pandas 0.23` and save it another format, say `json`. Then, open `json` file with `pandas 0.24` and save `pickle` formatted file. – Fatih1923 Apr 16 '19 at 14:15
  • @Fatih1923 thanks. Yes, that what I thought must be the strategy of last resort -.- – Lorenz Walthert Apr 16 '19 at 15:56
  • 5
    Thanks, I upgraded Pandas: `pip install --upgrade pandas` .. and the problem was solved. – Minions May 02 '19 at 10:12
  • 1
    For me, I'm using the same environment (so versions of both python and pandas are the same), but I'm still getting this error.. – zoekdestep Dec 24 '19 at 14:48
28

I was facing the same error when I was using pandas version 0.23.4.

I have installed pandas 0.24.1 version explicitly by:

pip3 install pandas==0.24.1

This solved my problem(Python version I was using was 3.5)

Chandan
  • 704
  • 7
  • 20
10

I had the same problem, but for me, it seemed to come from the pickle package / interaction with the pandas package.

I had Pandas version 0.23.4. I saved some pickle files with pandas.Dataframe.to_pickle, with python 3.6.6 & Pandas version 0.23.4. Then I upgraded to python 3.7.2 (Pandas version 0.23.4), and was enabled to read thoses pickle files with pandas.Dataframe.read_pickle. Next, I upgraded pandas to pandas 0.24.1, and it worked for me. I can read those files again.

DiCaprio
  • 823
  • 1
  • 7
  • 24
Charles dc
  • 101
  • 2
7

Updating pandas would be the best solution for most cases. However if you have limitations updating your pandas version, and you need to consume pandas objects produced and pickled in a higher version, you can add class location map as below.

from pandas.compat.pickle_compat import _class_locations_map

_class_locations_map.update({
    ('pandas.core.internals.managers', 'BlockManager'): ('pandas.core.internals', 'BlockManager')
})
Gengyu Shi
  • 111
  • 1
  • 5
  • Worked for me when someone created a pickle from pandas 0.25 and I'm locked in with 0.20.3 – mrbTT Jun 07 '21 at 14:13
2

conda update pandas

If you use conda package manager.

Bstampe
  • 689
  • 1
  • 6
  • 16