2

I have seen several questions asking how to send a numpy.ndarray through an xml-rpc call. This cannot be done out of the box because, as stated by the xml-rpc docs, there is a fixed set of supported types, basically limited to build-ins and containers of built-ins.

A simple solution is to convert the numpy array to a list using

<array to send>.tolist()

on the sending side, and convert back to a numpy array using

np.array(<list I received>)

on the receiving side. However, suppose I want to be able to support ndarray directly, without explicit flattening/unflattening. Is there an interface I can implement to make this work? For example, can I extend ndarray (I know that's tricky) with certain flattening/unflattening methods which will be registered with xml-rpc and thus used for serialization/de-serialization?

DanielSank
  • 3,303
  • 3
  • 24
  • 42

1 Answers1

1

I don't think there are any hooks for extension.

Rather than flatten/unflatten, you could encode/unencode the object -- for instance using pickle, or base64-encoded pickle.

If you do flatten keep in mind that you need:

<array>.shape
<array>.dtype
<array>.flat

Simple dtypes can be converted back and forth to strings; you may need to consider complex dtypes more carefully if you want to support them.

[EDIT] To answer your question succinctly: No -- no extension hooks are available.

shaunc
  • 5,317
  • 4
  • 43
  • 58
  • Sure, we can always break the array down into manageable parts. However, once we've done that, how can we possibly tell xml-rpc to unpack those parts into a proper ndarray on the receiving end? What I mean is, I can always send a tuple of (myArray.tostring(), myArray.shape, myArray.dtype) across the wire, but the receiver has no idea what to do with it. – DanielSank May 01 '14 at 03:12
  • One way of looking at it is -- even if there were extension hooks, they would have to be installed on both ends. Instead of hooks, it seems that you need to write wrappers, which have a common convention on both ends for how raw objects are annotated. One simple way to do it is that all binary objects are actually pickles: if both ends know this you still can't send everything (eg. not a bound method... unless you extend pickle -- but here there are hooks) but you should be able to send most things. – shaunc May 01 '14 at 15:58
  • Yup. But the thing is, if we're going to do wrappers, at least for ndarray we might as well do the .tolist() and array() that I mentioned in the question. I agree that it would be possible to write a generic flatten/unflatten wrapper to handle more types. – DanielSank May 01 '14 at 21:10
  • The base64 representation will be somewhat more compact... indeed if you really want compact, then look into [blosc](http://blosc.pytables.org/trac). Or if you want readable, then toList() is probably fine (though string or structured dtype could cause problems if you don't save it). – shaunc May 03 '14 at 13:34
  • The point of my question is really whether or not extension hooks exists, and I guess the answer is not. Wrappers it is. – DanielSank May 03 '14 at 17:30
  • shaunc: The only reason I haven't accepted your answer is that it doesn't definitively say whether or not there are extensibility hooks. – DanielSank May 05 '14 at 04:27
  • (Daniel -- did you notice the edit? I even took a quick look at the source code -- no hooks in sight...) – shaunc May 05 '14 at 22:59