Using del properly when referencing external resources

Question

I am dealing with an external library that does not handle its resources in a pythonic way. The library is natively a C++ library with a Python wrapper around it. Whenever a library function returns a C++ object, what I actually get is a string handle to it. The object itself persists in memory until I explicitly call a delete() function with that handle as the argument. I tried to introduce my own wrapper around the handle to make sure it is deleted when the wrapper goes out of scope. Something like this:

#import some_module as sm
class MyWrapper(str):
  def __new__(*args, **kwargs):
    # do something
  def __del__(self):
    sm.delete(self)

This works most of the time but on script completion I often get an exception that says sm has no attribute delete. It looks like the module object gets destroyed before my wrapper around the handle. Is there a way to make sure my __del__ is called before the module is unloaded?

How is your wrapper class used? It may make more sense to use `__exit__` instead of `__del__`, then use `with` to handle the lifetime. — Carcigenicate, May 28 '21 at 16:58
`__del__` does not make guarantees as to when it will be called. In python, it's generally better to do resource management explicitly, since the lifetime of objects is not determined by their lexical scope. So basically, context management — Mad Physicist, May 28 '21 at 17:01
@Carcigenicate: I thought about that but I typically have many handles with overlapping use — BlindDriver, May 28 '21 at 17:14
Wouldn't your current implementation have the same issue though? You're destroying the underlying object as soon as a one of the wrappers wrapping a handle is destroyed. — Carcigenicate, May 28 '21 at 17:16

score 2 · Answer 1 · answered May 28 '21 at 17:15

Interpreter shutdown may clear modules in arbitrary order. Store any required cleanup functions locally in __del__ by binding them as defaults:

class MyWrapper(str):
  def __new__(*args, **kwargs):
    # do something
  def __del__(self, _delete=sm.delete):
    _delete(self)

Since defaults are bound on definition and remain until the function itself is deleted, _delete will keep the original sm.delete available as long as the function/method lives.

Note that __del__ may be called at an arbitrary time, including never. It is generally a good idea to use it as a last resort: provide other means to explicitly release the object (e.g. support with or a .close()/.delete() method) and write __del__ in such a way that it does nothing if the object was released already.

Using __del__ properly when referencing external resources

1 Answers1

Using del properly when referencing external resources