unable to eliminate 'NoneType' object is not iterable error in DataFrame

Question

I'm an old SAS user new to Python. SO has several examples of why this error occurs. Here for example.

I want to eliminate this source of error. I'm working with a large dataframe created with the read_cvs reader.

print(df.size, df.shape, df.ndim)
7202952 (266776, 27) 2

Date and Time are objects:

print(df.Date.dtype)
print(df.Time.dtype)
object
object

So I convert them to datetime:

df['Date'] = pd.to_datetime(df['Date'])
df['Time'] = pd.to_datetime(df['Time'])

Set df.Date as the index:

df.set_index('Date', inplace=True)

Drop na's:

df = df.dropna()
print(df.shape)
(266752, 26)

And verify there are 24 fewer rows than indicated above:

Count the number of accident reports on Dec. 25, 2015 with:

print(len(df.loc['2015-07-04']))
625

Good so far. Now count the number of accident reports between Dec. 25, 2015 and Dec 31, 2015 with:

print(len(df.loc[['2015-12-25','2015-12-26']]))

which returns:

TypeError: 'NoneType' object is not iterable

I've run out of ideas, so I check the index with:

print(df.index)
DatetimeIndex(['2015-01-09', '2015-01-09', '2015-02-23', '2015-02-23',
           '2015-02-23', '2015-02-11', '2015-02-11', '2015-02-23',
           '2015-02-23', '2015-04-18',
           ...
           '2015-08-30', '2015-11-29', '2015-11-29', '2015-11-29',
           '2015-07-26', '2015-07-26', '2015-12-31', '2015-07-28',
           '2015-07-28', '2015-07-15'],
          dtype='datetime64[ns]', name='Date', length=266752, freq=None)

I would like to understand what is the source of this error. Other thoughts are: perhaps sorting df.Date would help, given Date is 'out-of-order' and with multiple rows for each day.

I assumed that dropping the 24 rows would solve the problem. And I also assume that since the DataFrame contains no objects, I would not have any None objects.

df.get_dtype_counts()
datetime64[ns]     1
int64             25
dtype: int64

Where do I go from here?

The dates in your code examples don't seem to be consistent with your explanatory text. — ekhumoro, Oct 14 '16 at 17:45

score 1 · Answer 1 · answered Feb 27 '17 at 15:38

I got the same error with my dataset, but found that when I used pandas.to_datetime to convert the dates I was selecting with, it worked. Here's some example code:

df1 = pd.DataFrame(np.random.randn(6,4), index=pd.date_range(start='2015-01-01', periods=6, freq='D'), columns=list('ABCD'))
len(df1.loc[['2015-01-01', '2015-01-03'],:])

returns an error

len(df1.loc[pd.to_datetime(['2015-01-01', '2015-01-03']),:])

returns 2

unable to eliminate 'NoneType' object is not iterable error in DataFrame

1 Answers1