Stack Overflow
Stack Overflow

Questions tagged [numexpr]

Numexpr is a fast numerical expression evaluator for NumPy.

Numexpr is a fast numerical expression evaluator for NumPy. With it, expressions that operate on arrays (like 3*a+4*b) are accelerated and use less memory than doing the same calculation in Python.

In addition, its multi-threaded capabilities can make use of all your cores, which may accelerate computations, most specially if they are not memory-bounded (e.g. those using transcendental functions).

Last but not least, numexpr can make use of Intel's VML (Vector Math Library, normally integrated in its Math Kernel Library, or MKL). This allows further acceleration of transcendent expressions.

Numexpr User Guide lists the functionalities supported by it alongwith other useful info related to it.

94 questions
3
votes
2 answers

Why does multi-processing slow down a nested for loop?

I have a lot of very large matrices AFeatures that I am comparing against some other very large matrices BFeatures, both of which have a shape of (878, 2, 4, 15, 17, 512), using the Euclidean distance. I am trying to parallelise this process to…
William Smith
  • 89
  • 1
  • 10
3
votes
1 answer

Trying to install pytables for python3

I use pip python -m pip install tables But then I get this error Collecting tables Using cached tables-3.6.1.tar.gz (4.6 MB) ERROR: Command errored out with exit status 1: command: /Users/collin.dubois/.pyenv/versions/3.9.1/bin/python3 -c…
Collin
  • 41
  • 1
  • 2
3
votes
1 answer

How to fix error while importing Numexpr Python

On Windows 10 Python 3.7.9 (IDLE) I successfully installed "pip install numexpr" but while "import numexpr as ne" I have an error: Traceback (most recent call last): File "", line 267, in import numexpr as ne File…
bluesky
  • 243
  • 2
  • 12
3
votes
1 answer

Python RecursionError : Simple operation crashes with Pandas.eval()

I have just read and drooled with excitement over these newly found optimization functions for my Pandas related needs. According to this book : The DataFrame.eval() method allows much more succinct evaluation of expressions with the columns:…
Imad
  • 2,358
  • 5
  • 26
  • 55
3
votes
1 answer

sympy lambdify with numexpr and sqrt

I'm trying to speed up some numeric code generated by lambdify using numexpr. Unfortunately, the numexpr-based function breaks when using the sqrt function, even though it's one of the supported functions. This reproduces the issue for me: import…
perimosocordiae
  • 17,287
  • 14
  • 60
  • 76
3
votes
1 answer

Creating a callable with numexpr

I'm doing some symbolic math with sympy, then generating a Python lambda function using eval and sympy's lambdastr utility. Here's a simplified example of what I mean: import sympy import numpy as np from sympy.utilities.lambdify import lambdastr #…
perimosocordiae
  • 17,287
  • 14
  • 60
  • 76
3
votes
2 answers

Can not install numexpr (and hence pytables) on windows 7, dll load failing for interpreter.pyd even though it is present

I installed numexpr and pytable using .whl. Installation looked fine but dll import failure keeps coming. Here are the installation details. PS E:\> pip install --use-wheel --no-index --find-links=.\ numexpr-2.4-cp27-none-win32.whl Ignoring indexes:…
Kumar Deepak
  • 473
  • 4
  • 18
3
votes
3 answers

Unable to subtract datetime64 using DataFrame.eval()

Given a DataFrame with a couple of timestamps: In [88]: df.dtypes Out[88]: Time datetime64[ns] uniqstime datetime64[ns] dtype: object If I call eval(), I get a type error: In [91]: df.eval('since = Time -…
chrisaycock
  • 36,470
  • 14
  • 88
  • 125
3
votes
1 answer

How to optimize evaluation of hessian in sympy?

I'm using statsmodels library for generic likelihood models. As I have a quite complicated likelihood function, I used sympy to calculate gradient and hessian for me. This works fine, but it is too slow for my needs, because likelihood function…
Marigold
  • 1,619
  • 1
  • 15
  • 17
2
votes
0 answers

How to speed up df.query

df.query uses numexpr under the hood, but is much slower than pure numexpr Let's say I have a big DataFrame: from random import shuffle import pandas as pd l1=list(range(1000000)) l2=list(range(1000000)) shuffle(l1) shuffle(l2) df =…
Hans
  • 148
  • 2
  • 7
2
votes
1 answer

Usage of numexpr contains() function

I want to check if each element of a Numpy string array contains a given string using numexpr (2.7). I have written: x = np.array(['abc', 'cde']) ne.evaluate("contains(x, 'a')") I get: ValueError: unknown type str96 I also tried to specify a dtype…
M. Page
  • 2,694
  • 2
  • 20
  • 35
2
votes
0 answers

Evaluating einstein summation using numexpr

I am currently working with large numpy array multiplications, using numpy.einsum, and have been facing MemoryError issue. That is why I'm trying to evaluate expressions, wherever possible, with numexpr. As far as I understand: np.einsum('ij,j ->…
konstant
  • 685
  • 1
  • 7
  • 19
2
votes
1 answer

Numexpr detecting number of threads less than number of cores

I am using numexpr for simple array addition on a remote cluster. My computer has 8 cores and the remote cluster has 28 cores. Numexpr documentation says that "During initialization time Numexpr sets this number to the number of detected cores in…
RaVi
  • 41
  • 1
  • 5
2
votes
1 answer

numexpr: temporary variables or repeated sub-expressions?

If the same sub-expression appears in multiple places within one numexpr expression, will it be recalculated multiple times (or is numexpr clever enough to detect this and reuse the result)? Is there any way to declare temporary variables within a…
benjimin
  • 4,043
  • 29
  • 48
2
votes
1 answer

Numexpr/PyTables: how to pass perform multiple condition queries from a list/array?

I am doing a query (i.e. selecting multiple rows based on some condition) using PyTables, with the functions tables.Table.read() and tables.Table.read_where(). This is essentially based on numpy and pandas with…
ShanZhengYang
  • 16,511
  • 49
  • 132
  • 234
1
2
3 4 5 6 7