1

this is my first stack overflow question so please pardon any ignorance about the forum on my part.

I am using python 2.7.6 64 bit and pandas 0.13.1-1 from Enthought Canopy 1.3.0.1715 on a win 7 machine. I have numpy 1.8.0-1 and numexpr 2.2.2-2.

I have inconsistent error behviour running the following on a pandas series of 10,000 numpy.float64 loaded from hdf:

import python
s = pandas.read_hdf(r'C:\test\test.h5', 'test')
s/2.

This gives me inconsistent behaviour, it sometimes works and sometimes throws:

OMP: Error #134: Cannot set thread affinity mask.
OMP: System error #87: The parameter is incorrect.

I have replicated this error on other machines, and the test case is derived from a unit test failure (with the above error) which was replicted on several machines and from a server. This has come up in an upgrade from pandas 0.12 to pandas 0.13.

The following consistantly runs with no error:

import python
s = pandas.read_hdf(r'C:\test\test.h5', 'test')
s.apply(lambda x: x/2.)

and,

import python
s = pandas.read_hdf(r'C:\test\test.h5', 'test')
pandas.computation.expressions.set_use_numexpr(False)
s/2.

Thanks for the help.

user
  • 13
  • 4

1 Answers1

1

This is a very similar problem to as described in this issue, and the linked issue

It seems that only canopy is experiencing these issues. I think it has to do with the canopy numpy MKL build, but that is a guess as I have not been able to reproduce this. So here are some work-arounds:

  • try to upgrade numexpr to 2.4 (current version), or downgrade to 2.1
  • try to use numpy 1.8.1 via canopy
  • try to install numpy/numexpr from the posted binaries here; I don't know exactly how canopy works so not sure if this is possible
  • uninstall numexpr

you can also disable numexpr support via pandas.computation.expressions.set_use_numexpr(False). Note that numexpr is REQUIRED in order to read/use HDF5 files via PyTables. However the expressions disabling of numexpr should just disable it for 'computations' (and not the HDF access).

Jeff
  • 125,376
  • 21
  • 220
  • 187
  • Thanks Jeff, it sounds like this is a question for Enthought. I've been able to work around the problem for now by using `apply`, but am still concerned. – user May 12 '14 at 08:21