Stack Overflow
Stack Overflow

Questions tagged [numexpr]

Numexpr is a fast numerical expression evaluator for NumPy.

Numexpr is a fast numerical expression evaluator for NumPy. With it, expressions that operate on arrays (like 3*a+4*b) are accelerated and use less memory than doing the same calculation in Python.

In addition, its multi-threaded capabilities can make use of all your cores, which may accelerate computations, most specially if they are not memory-bounded (e.g. those using transcendental functions).

Last but not least, numexpr can make use of Intel's VML (Vector Math Library, normally integrated in its Math Kernel Library, or MKL). This allows further acceleration of transcendent expressions.

Numexpr User Guide lists the functionalities supported by it alongwith other useful info related to it.

94 questions
1
vote
1 answer

how to utilize all core of cpu in linux for python code execution

I am performing some pandas and NumPy operation, in windows I can see all cores of CPU are running but in Linux environment, everything is running on 1 core. I want to engage all core so perform the operation fast. e.g., def a(): #some…
Naveen Rishishwar
  • 36
  • 5
1
vote
1 answer

multiply dataframes based on timestamp intervals overlap

I have two pandas dataframes, each with two columns: a measurement and a timestamp. I need to multiply the first differences of the measurements, but only if there is a time overlap between the two measurement intervals. How can I do this…
apocalypsis
  • 520
  • 8
  • 19
1
vote
0 answers

Using Numexpr in object method

I am trying to optimize some object oriented numpy code using numexpr but I am not sure if it is possible to do so. I can't find anything in the documentation about it either way. I would like to get the following example to work: import numpy as…
algol
  • 399
  • 3
  • 15
1
vote
1 answer

NumExpr: How to get variables inside expression

I can evaluate an expression in NumExpr like this numexpr.evaluate('17 == b', local_dict={'b': 17}) How can I get list of variable inside and expression like '17 == b' then I will get ['b'] Thank you.
user1501185
  • 101
  • 2
  • 2
1
vote
2 answers

Use object attribute in numexpr expression

I am trying to use an object attribute in a numexpr expression. The most obvious way of doing this: import numpy as np import numexpr as ne class MyClass: def __init__(self): self.a = np.zeros(10) o = MyClass() o.a b =…
Nathan Musoke
  • 166
  • 1
  • 12
1
vote
1 answer

Does numexpr support special bessel functions?

I need to evaluate BesselK function at about ten million points. I know scipy.special supports this as scipy.special.kv(n, x), but I want a faster evaluation as well as a memory efficient evaluation. Ideally numexpr would be good, and I tried…
konstant
  • 685
  • 1
  • 7
  • 19
1
vote
1 answer

What could be the best way to bypass `MemoryError` in this case?

I have two numpy arrays of pretty large size. First is arr1 of size (40, 40, 3580) and second is arr2 of size (3580, 50). What I want to achieve is arr_final = np.sum(arr1[..., None]*arr2, axis = 2) such that the size of arr_final is just (40, 40,…
konstant
  • 685
  • 1
  • 7
  • 19
1
vote
1 answer

Optimising numerical operations in numpy using numexpr

I just started using numexpr, and while the github repository seems to have some basic examples of how to use it, I couldn't clearly understand how those would apply to some complicated cases. Suppose I have a function: def func(x): #x is a numpy…
konstant
  • 685
  • 1
  • 7
  • 19
1
vote
1 answer

How to query python3 dataframe with integer list elements

I have been stuck on a problem for a few days now, and I would be grateful if someone could help. I have a dataframe which has a column filled with integer lists items. For example, a single column dataframe: >>> df=pd.DataFrame({'a':[[1, 2, 3], [2,…
Cédric
  • 11
  • 4
1
vote
1 answer

Speeding up a linear transform using parallel Cython

I need to speed up the calculation of a linear transform which is roughly of the following form: import numpy as…
phlegmax
  • 419
  • 1
  • 4
  • 11
1
vote
1 answer

recurrence in numpy / numexpr python

I have an array x=np.ones(1000) and would like to perform operations on this based on recurrence formulae: x[i]= f(x[i-1], x[i-2]) What is the fastest way to do it in a vectorize way ? I know that xx= ne.evaluate(xx1*xx2) can only manage…
user5497885
1
vote
1 answer

Numexpr doesn't recognize float type (sparse matrix)

I would like to evaluate the performance of numexpr module in python (2.7). For that purpose, I created a random sparse matrix of size (10^5, 10^5). However, the script below throws an error at the expression evaluation step already, saying that it…
El Brutale
  • 93
  • 8
1
vote
0 answers

"DeprecationWarning using `oa_ndim == 0` when `op_axes` is NULL" when performing normalization/computing z-scores on a pandas DataFrame

In order to normalize data in a pandas DataFrame I wrote the following functions: def zscore(data): data = (data - data.mean())/data.std(ddof=0) return data def normalize(x): return (x - x.min(axis=0)) / (x.max(axis=0) -…
Talia
  • 2,947
  • 4
  • 17
  • 28
1
vote
1 answer

pandas 0.13 system error: cannot set thread affinity mask

this is my first stack overflow question so please pardon any ignorance about the forum on my part. I am using python 2.7.6 64 bit and pandas 0.13.1-1 from Enthought Canopy 1.3.0.1715 on a win 7 machine. I have numpy 1.8.0-1 and numexpr 2.2.2-2. I…
user
  • 13
  • 4
1
vote
4 answers

Mapping Boolean formulas to Python set expressions

Say I have a Boolean formula that uses a known set of tokens, e.g.: Boolean operators: and, or, not Grouping operators: (, ) Given a Boolean formula using those tokens, e.g.: F: (A or B) and not(A and C) How can I convert this definition to a…
Amelio Vazquez-Reina
  • 91,494
  • 132
  • 359
  • 564
1 2 3
4
5 6 7