-2

I tried to use dask delayed to improve loops iteration speed, iteration done by map function. The problem is after dd.compute(), the result list is over bracket, so can not get proper dataframe. Anyone have solutions?

def combine(val):
    a=delayed(rss)(val)
    b=delayed(altman)(val)
    df={'Tiker':val,'RS':a,'Alt':b}
    return df

vals=tickers

df=map(combine,vals)
df=dd.compute(df)
df

Output:

([{'Tiker': 'ABDA.JK', 'RS': 0.75, 'Alt': 4.1937988034309255},
  {'Tiker': 'ABMM.JK', 'RS': 1.75, 'Alt': 6320.155816168163},
  {'Tiker': 'ACES.JK', 'RS': 0.44, 'Alt': 7.431649213502305}],)
user19858347
  • 1
  • 1
  • You’re returning a dictionary and just iterating over your arguments using python’s map. Where are you expecting a dataframe to show up? If you just wrap the final result in pd.DataFrame does that do what you were hoping? Or are you hoping to get a dask.dataframe? – Michael Delgado Aug 27 '22 at 20:50
  • Thanks for your comment, actually I need to make iterating with map and using delayed dask for faster result. I hope the result should be wrapped in pd.DataFrame – user19858347 Aug 28 '22 at 02:50
  • Or in other way, coz the list result form is: lst=([{'Tiker': 'ABDA.JK', 'RS': 0.75, 'Alt': 4.1937988034309255}, {'Tiker': 'ABMM.JK', 'RS': 1.75, 'Alt': 6320.155816168163}, {'Tiker': 'ACES.JK', 'RS': 0.44, 'Alt': 7.431649213502305}],) ; you can see there are over bracket, it maybe we could remove the additional bracket to get proper pdDataFrame list. – user19858347 Aug 28 '22 at 02:51
  • So far I got this solution, from itertools import chain def flatten(listOfLists): # "Flatten one level of nesting" return chain.from_iterable(listOfLists) lst=list(flatten(lst)) – user19858347 Aug 28 '22 at 03:16
  • 1
    Sorry, it isn't clear what you are asking/expecting and what problem - if any - you have come across. https://stackoverflow.com/help/how-to-ask – mdurant Aug 28 '22 at 13:28
  • you can initialize a pandas DataFrame with a list of dictionaries - just add `df = pd.DataFrame(df)` at the end of your code. Just calling a variable `df` isn't enough to turn it into a dataframe, and [`dask.delayed`](//docs.dask.org/en/stable/delayed.html) doesn't have anything to do with pandas. To specifically use [`dask.dataframe`](//docs.dask.org/en/stable/10-minutes-to-dask.html#creating-a-dask-object) you'll need to rewrite your code to use that module instead of `delayed` - you shouldn't mix and match dask scheduling paradigms (at least until you really know what you're doing). – Michael Delgado Aug 28 '22 at 18:06
  • 1
    also - please check out the pandas User guide's [intro to data structures](https://pandas.pydata.org/docs/user_guide/dsintro.html). The terminology is important when asking questions and understanding what's going on. There is no such thing as a pandas list. You're currently working with python `list` and `dict` objects - this has nothing to do with pandas. There's no reason this would magically turn into a pandas `DataFrame` unless you explicitly create one - you *can* work with lists and dictionaries to create a pandas object, but this doesn't have anything to do with dask. – Michael Delgado Aug 28 '22 at 18:13
  • Thanks for support, actually I am lack of knowledge of coding. But I need to create my own stocks screener. So come to those problems. By the way, thanks all for support – user19858347 Aug 31 '22 at 03:57

1 Answers1

0

It may be help for beginner like me, we can try trimmed over bracket list by: flatten one level of nesting

def flatten(listOfLists): 
# "Flatten one level of nesting" 
return chain.from_iterable(listOfLists) 
lst=list(flatten(lst))
user19858347
  • 1
  • 1