Stack Overflow
Stack Overflow

Questions tagged [pandas-apply]

Applies Python functions to rows or columns of a pandas dataframe, which may or may not result in aggregation.

Pandas apply available in DataFrame and Series classes is the equivalent of map in many functional languages like Haskell or Scala. It calls the function given in the argument for each element/row/column (depending on other parameters).

More detailed documentation can be found in:

170 questions
3
votes
2 answers

Pandas groupby function using multiple columns

This is similar to the following, however I wanted to take it one question further: pandas groupby apply on multiple columns to generate a new column I have this dataframe: Group Value Part Ratio 0 A 6373 10 0.637300 1 A …
jerbear
  • 361
  • 5
  • 14
3
votes
1 answer

pandas, keep groupby groups after apply

I would like to use groupby on my dataframe and then chain a series of function calls on each group with apply. As a first prototype, I've set up an example where I convert the entries of my dataframe from string to numeric. The dataframe looks like…
lhk
  • 27,458
  • 30
  • 122
  • 201
3
votes
2 answers

pandas resample apply np.average

I have time series "half hour" data. I need to resample demand to "1 day" using weighted average (using price) during the resample. dft demand price 2012-01-01 00:00:00 30940.500000 42.18 2012-01-01 00:30:00 …
dreab
  • 705
  • 3
  • 12
  • 22
3
votes
1 answer

pandas groupby apply is really slow

When I call df.groupby([...]).apply(lambda x: ...) the performance is horrible. Is there a faster / more direct way to do this simple query? To demonstrate my point, here is some code to set up the DataFrame: import pandas as pd df =…
user5406764
  • 1,627
  • 2
  • 16
  • 23
3
votes
2 answers

Apply lambda on a column in pandas

I have the below data frame ipdb> csv_data country_edited sale_edited date_edited transformation_edited 0 India 403171 21091956 1 1 Bhutan 394096 21091956 2 2 …
Shiva Krishna Bavandla
  • 25,548
  • 75
  • 193
  • 313
2
votes
3 answers

Pandas : Concat rows of a dataframe with same index to form custom string in pairs

Say I have a dataframe df = pd.DataFrame({'colA' : ['ABC', 'JKL', 'STU', '123'], 'colB' : ['DEF', 'MNO', 'VWX', '456'], 'colC' : ['GHI', 'PQR', 'YZ', '789'],}, index = [0,0,1,1]) colA colB colC 0 ABC DEF …
Himanshu Poddar
  • 7,112
  • 10
  • 47
  • 93
2
votes
3 answers

Pandas Groupby and Apply

I am performing a grouby and apply over a dataframe that is returning some strange results, I am using pandas 1.3.1 Here is the code: ddf = pd.DataFrame({ "id": [1,1,1,1,2] }) def do_something(df): return "x" ddf["title"] =…
Ben Muller
  • 221
  • 1
  • 4
  • 10
2
votes
2 answers

Python pandas dataframe apply result of function to multiple columns where NaN

I have a dataframe with three columns and a function that calculates the values of column y and z given the value of column x. I need to only calculate the values if they are missing NaN. def calculate(x): return 1, 2 df =…
Inthu
  • 1,009
  • 1
  • 8
  • 16
2
votes
1 answer

Pass arrays from DatafFame into function with arrays grouped and flattened

I have a dataframe with X position data for hundreds of participants, and three grouping variables (with each participant's X data being 1000 points in length). Preview of dataframe: X Z participantNum obsScenario startPos …
CentauriAurelius
  • 504
  • 3
  • 21
2
votes
1 answer

Faster way of computing the mean with pandas groupy + apply and condensing groups

I want to groupby two values and if the group contains more than one element, return only the first row of the group with the value replaced by the mean for the group. If there is only one element, I want to return directly. My code looks like…
LizzAlice
  • 678
  • 7
  • 18
2
votes
2 answers

Efficient way of row-based calculation in Pandas

I have a dataframe with 2 columns: class (0/1) and time (integer). I need to append a third column which will be the remaining time to get a class 1 row. df = pd.DataFrame([ [1,101], [1,104], [0,107], [0,110], [0,123], [1,156], …
mustafa
  • 3,605
  • 7
  • 34
  • 56
2
votes
0 answers

Break in Apply (Pandas)

Is it possible to include a break in the Pandas apply function? I have a set of very large dataframes that I need to apply a function to as part of an optimization problem. This seems like the best approach but there's significant daylight between…
Kung Fu Howie
  • 21
  • 3
2
votes
2 answers

Match all words from string in another string (words can be in different positions)

I have a list of strings that I have to match with dataframe column. The list looks as follows: list = ['golden village lte', 'pones wcdma', 'coral gbts', 'street view gbts', 'street view wcdma'] The column in the dataframe looks like this: …
jas_0n
  • 93
  • 5
2
votes
2 answers

KeyError with pandas groupby() apply()

Here I'm adding time to data where only date is given. There are 5 minutes between values or 288 values per date. The code works when the input dataframe is 1 day (288 rows) or less, but gives an error when the input is longer. Any idea what I'm…
nice_name
  • 37
  • 2
  • 6
2
votes
2 answers

Apply function to specific selected columns in pandas data frame

I have the following dataframe: # List of Tuples matrix = [([22, 23], [34, 35, 65], [23, 29, 31]), ([33, 34], [31, 44], [11, 16, 18]), ([44, 56, 76], [16, 34, 76], [21, 34]), ([55, 34], [32, 35, 38], [22, 24, 26]), …
sharathchandramandadi
  • 540
  • 3
  • 15
1
2
3
…
11 12