Converting Dask Scalar to integer value (or save it to text file)

Question

I have calculated using dask by

from dask import dataframe
all_data = dataframe.read_csv(path) 
total_sum = all_data.account_balance.sum()

The csv file has a column named account_balance.

The total_sum is a dd.Scalar object, which seems to be difficult to change it to integer. How to get the integer version of it? or save it in a .txt file containing the number is also ok.

I have also tried total_sum.compute().

Thanks.

Once you compute it you can print it and/or save to file. – rpanai Jul 23 '18 at 17:38 — rpanai, Jul 23 '18 at 17:38

score 9 · Answer 1 · answered Jul 23 '18 at 20:42

9

.compute() does indeed bring you a real number, as you can see in this example:

In [18]: import dask.dataframe as dd

In [19]: d = dd.from_pandas(pd.DataFrame({'a': [3,3,3,3]}), npartitions=2)

In [20]: d.a.sum().compute()
Out[20]: 12

answered Jul 23 '18 at 20:42

mdurant

27,272
5
45
74

Thanks, but is there another way? the `total_sum.compute()` takes really long time. – Jul 24 '18 at 05:36
@anbarief1 dask is lazy it means that if you don't compute it you only have the graph for computation. – rpanai Jul 24 '18 at 12:47
It takes a long time because it has to read all of your CSV data! – mdurant Jul 24 '18 at 13:14

Converting Dask Scalar to integer value (or save it to text file)

1 Answers1

Linked

Related