Split list of datetimes into days

Question

I've got a sorted list of datetimes: (with day gaps)

list_of_dts = [
              datetime.datetime(2012,1,1,0,0,0), 
              datetime.datetime(2012,1,1,1,0,0), 
              datetime.datetime(2012,1,2,0,0,0), 
              datetime.datetime(2012,1,3,0,0,0),
              datetime.datetime(2012,1,5,0,0,0),
              ]

And I'd like to split them in to a list for each day:

result = [
          [datetime.datetime(2012,1,1,0,0,0), datetime.datetime(2012,1,1,1,0,0)],
          [datetime.datetime(2012,1,2,0,0,0)],
          [datetime.datetime(2012,1,3,0,0,0)],
          [], # Empty list for no datetimes on day
          [datetime.datetime(2012,1,5,0,0,0)]
         ]

Algorithmically, it should be possible to achieve at least O(n).

Perhaps something like the following: (This obviously doesn't handle missed days, and drops the last dt, but it's a start)

def dt_to_d(list_of_dts):
    result = []
    start_dt = list_of_dts[0]
    day = [start_dt]
    for i, dt in enumerate(list_of_dts[1:]):
        previous = start_dt if i == 0 else list_of_dts[i-1]
        if dt.day > previous.day or dt.month > previous.month or dt.year > previous.year: 
            # split to new sub-list
            result.append(day)
            day = []
            # Loop for each day gap?
        day.append(dt)
    return result

Thoughts?

use a dict of lists with datetime_value.date() as keys. – Paulo Scardine Jan 24 '12 at 07:11 — Paulo Scardine, Jan 24 '12 at 07:11

Raymond Hettinger · Accepted Answer · 2012-01-24T07:31:48.303

The easiest way to go is to use dict.setdefault to group entries falling on the same day and then loop over the lowest day to the highest:

>>> import datetime
>>> list_of_dts = [
              datetime.datetime(2012,1,1,0,0,0),
              datetime.datetime(2012,1,1,1,0,0),
              datetime.datetime(2012,1,2,0,0,0),
              datetime.datetime(2012,1,3,0,0,0),
              datetime.datetime(2012,1,5,0,0,0),
              ]

>>> days = {}
>>> for dt in list_of_dts:
        days.setdefault(dt.toordinal(), []).append(dt)

>>> [days.get(day, []) for day in range(min(days), max(days)+1)]
[[datetime.datetime(2012, 1, 1, 0, 0), datetime.datetime(2012, 1, 1, 1, 0)], 
 [datetime.datetime(2012, 1, 2, 0, 0)],
 [datetime.datetime(2012, 1, 3, 0, 0)],
 [],
 [datetime.datetime(2012, 1, 5, 0, 0)]]

Another approach for making such groupings is itertools.groupby. It is designed for this kind of work, but it doesn't provide a way to fill-in an empty list for missing days:

>>> import itertools
>>> [list(group) for k, group in itertools.groupby(list_of_dts,
                                                   key=datetime.datetime.toordinal)]
[[datetime.datetime(2012, 1, 1, 0, 0), datetime.datetime(2012, 1, 1, 1, 0)], 
 [datetime.datetime(2012, 1, 2, 0, 0)],
 [datetime.datetime(2012, 1, 3, 0, 0)],
 [datetime.datetime(2012, 1, 5, 0, 0)]]

setdefault and toordinal are nice improvements over my answer. :-) — Paulo Scardine, Jan 24 '12 at 07:36
Even simpler than `setdefault` can be `defaultdict` (from `collections`). — Amber, Jan 24 '12 at 07:49
Beautiful - thanks! I can generate my `days` list at the same time as my `list_of_dts`, so it will be super efficient. — Alex L, Jan 24 '12 at 09:27

score 7 · Answer 2 · answered Jan 24 '12 at 07:12

7

You can use itertools.groupby to easily handle this kind of problems:

import datetime
import itertools

list_of_dts = [
        datetime.datetime(2012,1,1,0,0,0), 
        datetime.datetime(2012,1,1,1,0,0), 
        datetime.datetime(2012,1,2,0,0,0), 
        datetime.datetime(2012,1,3,0,0,0),
        datetime.datetime(2012,1,5,0,0,0),
        ]

print [list(g) for k, g in itertools.groupby(list_of_dts, key=lambda d: d.date())]

answered Jan 24 '12 at 07:12

qiao

17,941
6
57
46

Good to know - thanks! It doesn't handle the requirement of an empty list for missing days, though. – Alex L Jan 24 '12 at 09:28

Paulo Scardine · Answer 3 · 2012-01-24T07:28:12.640

Filling the gaps:

date_dict = {}
for date_value in list_of_dates:
    if date_dict.has_key(date_value.date()):
        date_dict[date_value.date()].append(date_value)
    else:
        date_dict[date_value.date()] = [ date_value ]
sorted_dates = sorted(date_dict.keys())
date = sorted_dates[0]
while date <= sorted_dates[-1]:
    print date_dict.get(date, [])
    date += datetime.timedelta(1)

Results:

[datetime.datetime(2012, 1, 1, 0, 0), datetime.datetime(2012, 1, 1, 1, 0)]
[datetime.datetime(2012, 1, 2, 0, 0)]
[datetime.datetime(2012, 1, 3, 0, 0)]
[]
[datetime.datetime(2012, 1, 5, 0, 0)]

This solution does not requires the original datetime list to be sorted.

Lex · Answer 4 · 2012-02-16T20:40:38.167

1

list_of_dts = [
            datetime.datetime(2012,1,1,0,0,0), 
            datetime.datetime(2012,1,1,1,0,0), 
            datetime.datetime(2012,1,2,0,0,0), 
            datetime.datetime(2012,1,3,0,0,0),
            datetime.datetime(2012,1,5,0,0,0),
            ]

groupedByDay={}
for date in list_of_dts:
    if date.date() in groupedByDay:
        groupedByDay[date.date()].append(date)
    else:
        groupedByDay[date.date()]=[date]

Now you have a dictionary, where the date is the key and the value is a list of similar dates.

and if you are set on having a list instead

result = groupedByDay.values()
result.sort()

now results is a list of lists, where all the dates with the same day are grouped together

edited Feb 16 '12 at 20:40

answered Jan 24 '12 at 07:55

Lex

386
1
4
20

2

`date.date() in groupedByDay.keys()` is O(n). Could you replace it with `date.date() in groupedByDay`? it's O(1). – reclosedev Jan 24 '12 at 08:27
done. Thanks for the tip. I just recently picked up python.. learn something new every day. – Lex Jan 24 '12 at 08:41

Split list of datetimes into days

4 Answers4

Linked