Stack Overflow
Stack Overflow

Questions tagged [numexpr]

Numexpr is a fast numerical expression evaluator for NumPy.

Numexpr is a fast numerical expression evaluator for NumPy. With it, expressions that operate on arrays (like 3*a+4*b) are accelerated and use less memory than doing the same calculation in Python.

In addition, its multi-threaded capabilities can make use of all your cores, which may accelerate computations, most specially if they are not memory-bounded (e.g. those using transcendental functions).

Last but not least, numexpr can make use of Intel's VML (Vector Math Library, normally integrated in its Math Kernel Library, or MKL). This allows further acceleration of transcendent expressions.

Numexpr User Guide lists the functionalities supported by it alongwith other useful info related to it.

94 questions
1
vote
1 answer

What is the most memory efficient way to combine read_sorted and Expr in pytables?

I am looking for the most memory efficient way to combine reading a Pytables table (columns: x,y,z) in a sorted order(z column has a CSI) and evaluating an expression like x+a*y+b*z where a and b are constant. Up until now my only solution was to…
Ben K.
  • 1,160
  • 6
  • 20
1
vote
1 answer

Find max non-infinity element in pytables CArray

This must be easy, but I'm very new to pytables. My application has dataset sizes so large they cannot be held in memory, thus I use PyTable CArrays. However, I need to find the maximum element in an array that is not infinity. Naively in numpy…
Rich
  • 12,068
  • 9
  • 62
  • 94
1
vote
1 answer

euclidean norm using numexpr

I need to rewrite this code using numexpr, it's calculating euclidean norm matrix of matrix data [rows x cols] and vector [1 x cols]. d = ((data-vec)**2).sum(axis=1) How can it be done? Also maybe there is another even faster method? The problem…
mrgloom
  • 20,061
  • 36
  • 171
  • 301
1
vote
0 answers

Mergesort with pytable

I have two large tables (not fiting in memory) with an id, I'd like to merge (join). Is there a way to do that fastly with pytables ? i thank about full-indexing the two tables and using itersorted with this kind of algorithm :…
julienfr112
  • 2,077
  • 2
  • 25
  • 37
1
vote
0 answers

How to do in-place polar coordinates transform with numpy a la numexpr

Numexpr is a python module that enables jitting some expressions for working with numpy arrays. Now, it would be cool if it allowed me to do: ne.evaluate( """ r = y cita = 2*pi*x/x_span nx = CENTER_X + cos( cita ) * r ny = CENTER_Y…
dsign
  • 12,340
  • 6
  • 59
  • 82
1
vote
1 answer

"Global array" parallel programming on distributed memory clusters with python

I am looking for a python library which extends the functionality of numpy to operations on a distributed memory cluster: i.e. "a parallel programming model in which the programmer views an array as a single global array rather than multiple,…
ARF
  • 7,420
  • 8
  • 45
  • 72
1
vote
1 answer

Automatic float32 promotion in numexpr

Consider the following NumPy array of dtype float32: In [29]: x = numpy.arange(10, dtype=numpy.float32) When I multiply it by 2 using pytables.Expr, I get a float32 array back: In [30]: tables.Expr('x * 2').eval().dtype Out[30]:…
NPE
  • 486,780
  • 108
  • 951
  • 1,012
0
votes
0 answers

numexpr.interpreter module not found error in aws lambda

I created an aws lambda function that uses langchain to use AI. But I encountered a problem when I run the function like: { "errorMessage": "Unable to import module 'lambda_function': No module named 'numexpr.interpreter'", "errorType":…
batuhan
  • 19
  • 1
  • 2
0
votes
1 answer

Why does numexpr.evaluate return an error when using functions from mpmath?

I've just started using numexpr's evaluate function and I've come across an annoying error. I want it to print, per se, sin(10), and it does that just perfectly, but if I do sec(10), I get "TypeError: 'VariableNode' object is not callable" Example…
Remfy
  • 61
  • 4
0
votes
1 answer

pip numexpr installation and spyder kernal issues

I booted up Spyder (Python 3.7) and got this warning: C:\ANACONDA3\lib\site-packages\pandas\compat\_optional.py:138: UserWarning: Pandas requires version '2.7.0' or newer of 'numexpr' (version '2.6.9' currently installed). warnings.warn(msg,…
Tim Kirkwood
  • 598
  • 2
  • 7
  • 18
0
votes
0 answers

Numba much slower than plain python, from Pandas documentation

I am aiming to replicate the results from the enhancing performing guide of the pandas documentation. https://pandas.pydata.org/pandas-docs/stable/user_guide/enhancingperf.html The numba implementation of the example should be really fast, even…
Rens
  • 177
  • 9
0
votes
1 answer

Numexpr in Python doesn't recognise a declared symbol

I'm trying to do some plots of some symbolic data. I have some expression from a regression in the form: expr = …
imwellhowareyou
  • 49
  • 7
0
votes
0 answers

Prevent NumExpr evaluation from breaking my bot (dpy)

I have a bot and I am trying to make a math command, I am using numexprs eval which works great but I have an issue. When a user uses the math command on an insanely high number (f. e. 8287467846 * 72846871642847846) the bot breaks, so I had to…
Kile Alkuri
  • 21
  • 2
0
votes
1 answer

Pandas fails at numexpr check inside exe script made with PyInstaller

I made a Dash web applications from which an .exe was created by using PyInstaller. The app loads without problems but at calling a function it suddenly stops. The problem seems to lie at the fact that numexpr library is not installed (pandas check…
sr_valderrama
  • 1
  • 2
0
votes
1 answer

not able to use all core of CPU in pandas linux

I am trying to utilize all cores of CPU, for NumPy, I am able to use all cores, but when I am performing some operation in pandas, then again it using only one core of CPU. I have tried to set the max thread and other params, but nothin working foe…
Naveen Rishishwar
  • 36
  • 5
1 2 3 4
5
6 7