groupby aggregation on day level of date time column in pandas

Question

I have a dataframe as shown below. Which is a doctors booking data.

  Doctor     Appointment              Show
  A          2020-01-18 12:00:00      Yes
  A          2020-01-18 12:30:00      Yes
  A          2020-01-18 13:00:00      No
  A          2020-01-18 13:30:00      Yes
  B          2020-01-18 12:00:00      Yes
  B          2020-01-18 12:30:00      Yes
  B          2020-01-18 13:00:00      No
  B          2020-01-18 13:30:00      Yes
  B          2020-01-18 16:00:00      No
  B          2020-01-18 16:30:00      Yes
  A          2020-01-19 12:00:00      Yes
  A          2020-01-19 12:30:00      Yes
  A          2020-01-19 13:00:00      No
  A          2020-01-19 13:30:00      Yes
  A          2020-01-19 14:00:00      Yes
  A          2020-01-19 14:30:00      No
  A          2020-01-19 16:00:00      No
  A          2020-01-19 16:30:00      Yes
  B          2020-01-19 12:00:00      Yes
  B          2020-01-19 12:30:00      Yes
  B          2020-01-19 13:00:00      No
  B          2020-01-19 13:30:00      Yes
  B          2020-01-19 14:00:00      No
  B          2020-01-19 14:30:00      Yes
  B          2020-01-19 15:00:00      No
  B          2020-01-18 15:30:00      Yes

From the above dataframe I would like to create a function in pandas, which will output the following.

I tried below

def Doctor_date_summary(doctor, date):
   Number of slots = df.groupby([doctor, date] ).sum()

Expected Output:

Doctor_date_summary(Doctor, date)
If Doctor = A, date = 2020-01-19

Number of slots = 8
Number of show up = 5
show up percentage = 62.5

where number of Yes = 5 in show column on that date for that docto

One question - Do you need count all data like in my question and then seelct by date and doctors? Or need select only some values and count like another question? — jezrael, Apr 06 '20 at 11:10

score 1 · Answer 1 · answered Apr 06 '20 at 11:01

You can first create a day column, from here:

df['day'] = df['Appointment'].dt.floor('d')

Then you can use boolean indexing:

def Doctor_date_summary(Doctor, date):
    number_of_show_up = np.sum((df['Doctor']==Doctor) & (df['day']==date) & (df['Show']=='Yes'))
    number_of_slots = np.sum((df['Doctor']==Doctor) & (df['day']==date))

    return number_of_show_up, number_of_slots, 100*number_of_show_up/number_of_slots

Finally:

number_of_show_up, number_of_slots, percentage = Doctor_date_summary('A', '2020-01-19')

print("Number of slots = {}".format(number_of_slots))
print("Number of show up = {}".format(number_of_show_up))
print("show up percentage = {:.1f}".format(percentage))

Number of slots = 8
Number of show up = 5
show up percentage = 62.5

jezrael · Accepted Answer · 2020-04-06T11:20:47.747

You can create each mask separately in function, then chain by & for bitwise AND and sum for count:

df['Appointment'] = pd.to_datetime(df['Appointment'])

def Doctor_date_summary(doctor, date):
    m1 = df['Doctor'] == doctor
    m2 = df['Appointment'].dt.normalize() == date
    m3 = df['Show'] == 'Yes'
    show_up = (m1 & m2 & m3).sum()
    no = (m1 & m2).sum()
    return show_up, no

up, no = Doctor_date_summary('A', '2020-01-19')

Last for output are used f-strings:

print(f"Number of slots = {up}")
print(f"Number of show up = {no}")
print(f"show up percentage = {up/no*100}")
Number of slots = 5
Number of show up = 8
show up percentage = 62.5

groupby aggregation on day level of date time column in pandas

2 Answers2