Take the following example data:
tag = [123,124,125]
rad = [[40.0, 33.0, 23.2], [22.3, 11.6, 20.3], [45.2, 96.2, 77.3]]
I made a pandas dataframe with tag and rad as the keys, where df['rad'][0]=[40.0, 33.0, 23.2]. I need each row value in rad to remain a list.
Now, I'm trying to append this data to an existing dataframe:
store_radial = pd.HDFStore('radials.h5')
df = pd.DataFrame('tag':tag,'radial':[rad]})
store_radial.put('tag', df, append=True, format='t', data_columns=True)
I've seen this method work in other posts here, but when I run it using my data, I get the following error:
TypeError: Cannot serialize the column [radial] because
its data contents are [mixed] object type
The types are the following:
radial object
tag int64
dtype: object
I believe my issue is caused by rad being a list. Is there a way to append to a .h5 file while keeping rad as a list? If there is a better way to do this, I'm all ears. Please let me know if I need to clarify.
Update: I have also tried the following and still get the same error:
df = pd.DataFrame({'radials':[rad_pro], 'tag':tag})
df['radials'].to_hdf('r%d_radials.h5' % run, key='radials', mode='a', append=True, format='table')
From what I understand, the dtype of rad needs to match what the individual values in the list are for this to work. The error is saying that inside radials, the dtypes are mixed. Which I don't quite understand why that is.
Clarification update: To clarify further, I want my dataframe to look like this:
In [15]: df
Out[15]:
radials tag
0 [40.0, 33.0, 23.2] 123
1 [22.3, 11.6, 20.3] 124
2 [45.2, 96.2, 77.3] 125
From there, I want the appended .h5 to keep the format above, where df['radials'][i] is a list.
