0

I am trying to use a function within a RPyC threaded server which returns the dict containing file attributes such as location, filename by looping over all the folders within the specified path.

However, when this is returned back to client, the list object (fl) is of type 

<netref class 'rpyc.core.netref.builtins.list'>

which I try to convert to a list using 

ft= list(ft)

but this too converts it to '<class 'list'>' and not 'list' as expected.

I'd like this to be converted to a dataframe but using 'df = pd.DataFrame(fl)' returns an error 'AttributeError: cannot access 'keys'

RPyC server function:

The closest I came to finding a related response was in this post but I still don't know if I understand this right. Is there a way to put this into a dataframe or convert to normal list which can then be converted easily? Any help is appreciated.

Server:

    PATH = r"C:\Temp"
def exposed_fquery():
    fl = []
    for (dpath, dname, fname) in os.walk(PATH):
        for f in fname:
            td = {}
            td['Location'] = dpath
            td['Name'] = f
            fl.append(td)
            print (fl)
    return (fl)

Client:

con = rpyc.connect('localhost',5000)
s = con.root.Server()
filemap = s.fquery(index)
print (type(filemap), "\n", filemap)  
print (type(ft), "\n",ft)
ft= list(ft)
print (type(ft))

Result:

<netref class 'rpyc.core.netref.builtins.list'>
[{'Location': 'C:\\Temp\\', 'Name': 'file1.txt'}, {'Location': 'C:\\Temp\\', 'Name': 'Test.txt'}]
<class 'list'>
ParvBanks
  • 1,316
  • 1
  • 9
  • 15

2 Answers2

0

I'm not sure where exactly your troubles are coming from, but I suspect you've encountered some bad input because with the example you provided it seems to work.

In [7]: ft = [{'Location': 'C:\\Temp\\', 'Name': 'file1.txt'}, {'Location': 'C:\\Te
   ...: mp\\', 'Name': 'Test.txt'}]

In [8]: pd.DataFrame(ft)
Out[8]:
   Location       Name
0  C:\Temp\  file1.txt
1  C:\Temp\   Test.txt

In [9]: pd.DataFrame.from_records(ft)
Out[9]:
   Location       Name
0  C:\Temp\  file1.txt
1  C:\Temp\   Test.txt

As you can see both pd.DataFrame(ft) and pd.DataFrame.from_records(ft) worked.

Also note that when you print the type list, <class 'list'> is what you get:

In [14]: type(ft)
Out[14]: list

In [15]: print(type(ft))
<class 'list'>
Andrew
  • 141
  • 4
  • I know.. that's what's baffling me! I tried it on a number of lists on the console and it works! I noticed that rpyc was saving/ transmitting data differently and thus `` but I didn't know that print (type) would return the `` so I was confused but Thank you for clarifying, that's good to know. However, now I'm even more confused. What does the `'AttributeError: cannot access 'keys'` mean in this case? – ParvBanks Aug 10 '19 at 14:31
  • 1
    Not sure. Can you post the actual input that caused this error? Or if the actual input is too long or private, add the following print statements and show us the output: `print(len(ft))` and `print({type(x) for x in ft})`. – Andrew Aug 10 '19 at 14:44
  • `print (len(ft)) >> 8 print({type(x) for x in ft}) >> {}` Does this mean that the dictionary items will also need to be converted? – ParvBanks Aug 10 '19 at 15:19
  • I tried `f = [dict(x) for x in ft]` to check if it works but it returns another error `ValueError: dictionary update sequence element #0 has length 5; 2 is required`. – ParvBanks Aug 10 '19 at 15:27
  • 1
    A quick google search yielded [this issue](https://github.com/tomerfiliba/rpyc/issues/272) in the RPyC github. The method used there for converting a `'rpyc.core.netref.builtins.dict'` to a native python dict is `{key: ft[key] for key in ft}`, though this is obviously less than ideal. – Andrew Aug 10 '19 at 15:40
  • thanks so much! That fixed the issue! :) Didn't think of checking it on Github. – ParvBanks Aug 10 '19 at 15:51
0

If you want to get real value of dict type, just handle the response as serialization with pickle before sent to RPC client, and you can get the real value of dict type after deserializing with pickle again.

池小波
  • 1