9

I use SqlAlchemy to connect to my database backend and make heavy use of multiprocessing in my Python application. I came to a situation which requires to pass an object reference, which is the result of a database query, from one process to another.

This is a problem, because when accessing an attribute of the object, SqlAlchemy trys to reattach the object into the current session of the other process, which fails with an exception, because the object is attached in an other session:

InvalidRequestError: Object '<Field at 0x9af3e4c>' is already attached to session '148848780' (this is '159831148')

What is the way to handle this situation? Is it possible to detach the object from the first session or clone the object without the ORM related stuff?

Manuel Faux
  • 2,317
  • 5
  • 24
  • 35
  • 2
    Just a suggestion: have you tried merging the object into the new session? http://www.sqlalchemy.org/docs/orm/session.html#merging – Maxim Sloyko Jan 13 '11 at 13:05
  • session.expunge(obj) makes sqlalchemy forget about obj. However, if you change obj afterwards, changes are not commited to the database any more. maksymko's suggestion is certainly better. – Simon Jan 13 '11 at 13:09
  • Is the attribute on the object another database stored object? For example is it a foreign key or a blob? – Ken Kinder Jan 13 '11 at 16:13
  • ''merge'' looks well to me, I'll try this one out. I only need read-access to the attributes, which are int's and str's. – Manuel Faux Jan 13 '11 at 18:18

1 Answers1

15

This is a bad idea (tm).

You shouldn't share a stateful object between processes like this (I know it's tempting) because all kinds of bad things can happen since lock primitives are not intended to work across multiple python runtimes.

I suggest taking the attributes you need out of that object, jamming them into a dict and sending it across processes using multprocessing Pipes:

http://docs.python.org/library/multiprocessing.html#pipes-and-queues

Rafael Ferreira
  • 1,260
  • 8
  • 11