0

I have a string with format "[[Integer1, tag1], [Integer2, tag2]]" as values in dask Series and want to use df[col] = df[col].apply(ast.literal_eval) to convert these into normal list values within that dask Series. The len of this list value can vary at different locations in the Series.

But getting the error ValueError('malformed node or string: <_ast.Name object at 0x7f593adfe198>',)

This works with pandas Series but getting this error with dask Series.

goku
  • 156
  • 2
  • 8
  • `"[[Integer1, tag1], [Integer2, tag2]]"` is not a valid python literal. It could be a valid python expression nyouncoyld `eval` if those names were in the global scope. In any case this seems like an XY problem. Why do you have these string to begin with? It looks like someone just dumped the string representation of some nested list of objects and now you are trying to parse that outputs. You should fix the source to use some standard serialization format – juanpa.arrivillaga Sep 08 '20 at 07:42
  • the values in the series were lists in the beginning but to dump it into `parquet` format I had to convert them into strings and when I reload it I need to get this format back as lists. The `len` of these list values can vary and each value is itself a list that contains an integer/float and a string tag. – goku Sep 08 '20 at 08:04
  • Right, you need to use an actual serialization format. Not just dump the string representation and call it a day. That is not meant to be used that way, so don't do it – juanpa.arrivillaga Sep 08 '20 at 08:20
  • Wait, why does you series have lists in it to begin with? There seems to be a fundamental problem with how you are processing this data. You should address that before you lay another layer of hacks on top of it – juanpa.arrivillaga Sep 08 '20 at 08:22
  • I have a bunch of integer values that are required to be stored with a tag in each row. Can you suggest a good way here and a good serialization format to use? – goku Sep 08 '20 at 11:19

0 Answers0