1

I'm currently trying to apply the concept of vectorization using Pandas. I've been successfully able to use crude looping, but on the same code, when I try to vectorize and pass the entire Series to a function I get

ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

My function is rather simple:

def price_function(x):
    if x >= 50:
        return "High"
    else:
        return "low"

And I'm calling it with the price Series of a Dataframe with the following:

listing_price = price_function(listings_dataframe_big['price'])

And the error is being triggered by the following line:

if x >= 50:

Any idea on why is this happening and how to fix this?

Jérôme Richard
  • 41,678
  • 6
  • 29
  • 59
Cláudio Ribeiro
  • 1,509
  • 3
  • 37
  • 65
  • 1
    Since a series has multiple values, then evaluating s > 50 is ambiguous, because part of the series could be True and other parts False. Do you want to check to see if all values are great than 50 or if a single value is greater than 50. use `all` or `any`. – Scott Boston Apr 14 '21 at 20:15
  • 1
    You do `np.where(listings_dataframe_big['price'] >=50, 'high', 'low')`. It is super fast – Prayson W. Daniel Apr 14 '21 at 20:18
  • Note that `apply` and `map` are not vectorizations. `np.where` is vectorized. – tdy Apr 14 '21 at 20:43

2 Answers2

0

Note that apply() and map() are not vectorizations, just loops in disguise.

np.where() is vectorized and thus generally the fastest method:

listing_price = np.where(listings_dataframe_big['price'] >= 50, 'High', 'Low')

But as Scott mentioned, the original issue is that when x is a series, x >= 50 is ambiguous because pandas doesn't know if you want (x >= 50).any() or (x >= 50).all().

tdy
  • 36,675
  • 19
  • 86
  • 83
0

The way to apply a function to all values of a series would be to use map.

listing_price = listings_dataframe_big['price'].map(price_function)

But as already mentioned, np.were is easier and most likely faster.

Wouter
  • 3,201
  • 6
  • 17