0

I am working on a project in which I am using the API of some third party closed-source Django application in order to write a plugin that extends its functionality.

The plugin consists of defining resource classes by inheriting from pre-made existing ones (defined in the Django application) and then writing the required logic to create those new resources and afterwards updating them on a regular basis

The straight forward way to create the custom resource classes is by using the class keyword as in the following code snippet that defines a new resource called Volume

from chroma_core.lib.storage_plugin.api import attributes, identifiers, plugin, relations, resources, statistics
class Volume(resources.PhysicalDisk):
    name = attributes.String()
    storagePool = attributes.String()
    wwn = attributes.String()
    health = attributes.String()

    class Meta:
        identifier = identifiers.ScopedId('serialNumber')

the instantiation of object resources from the custom resource defined above is done by using a method from the given API that has the following signature: 

Plugin.update_or_create(ResourceClass, parents=[], **attrs) which is defined by the API

Doing things in this way, everything is good and there are no problems at all.

However, for many reasons and requirement constraints I have to create the custom resources on the fly, so I moved to use the "type" built-in function as folowes:

Meta = type('Meta', (), {'identifier': identifiers.ScopedId('serialNumber')})
superclasses = (resources.PhysicalDisk, )
attrs = {'name': attributes.String(), 'storagePool': attributes.String(), 'wwn': attributes.String(), 'health': attributes.String(), 'Meta': Meta}

Volume = type('Volume', superclasses, attrs)

unfortunately, when creating new objects from the custom resource "Volume" defined on the fly I get an error:

[2014-07-21 09:30:26,916: WARNING/scan_daemon] Backtrace: Traceback (most recent call last):

  File "/usr/share/chroma-manager/chroma_core/services/plugin_runner/scan_daemon.py", line 192, in run

  File "/usr/share/chroma-manager/chroma_core/services/plugin_runner/scan_daemon.py", line 222, in _scan_loop

  File "/usr/share/chroma-manager/chroma_core/lib/storage_plugin/base_plugin.py", line 193, in do_initial_scan

  File "/usr/share/chroma-manager/chroma_core/lib/storage_plugin/base_plugin.py", line 210, in _initial_populate

  File "/usr/share/chroma-manager/chroma_core/services/plugin_runner/resource_manager.py", line 327, in session_open

  File "/usr/lib/python2.6/site-packages/django/db/transaction.py", line 224, in inner
    return func(*args, **kwargs)

  File "/usr/share/chroma-manager/chroma_core/services/plugin_runner/resource_manager.py", line 1361, in _persist_new_resources

  File "/usr/share/chroma-manager/chroma_core/services/plugin_runner/resource_manager.py", line 1340, in order_by_references

  File "/usr/share/chroma-manager/chroma_core/lib/storage_plugin/manager.py", line 229, in get_plugin_resource_class

PluginNotFound: PluginNotFound: Plugin chroma_core.lib.storage_plugin.base_resource not found (not one of ['pluginmngr', 'linux_network', 'linux'])

I did a lot of debugging, and by inspecting the pluginmngr.py module with inspect.getmembers I figured out a very odd behavior that I do not understand: here below is the inspecting code:

module_obj = sys.modules[__name__]
inspect.getmembers(module_obj)

when the Volume resource is defined statically by the class keyword then inspection shows that Volume is reported as expected 

'Volume': <class 'pluginmngr.Volume'>

while when creating the Volume on the fly using the "type" built-in function as showed above, then inspecting the module shows that Volume is reported as

 'Volume': <class 'chroma_core.lib.storage_plugin.base_resource.Volume'>

hence, the custom resource Volume oddly enough was associated to some strange module that belongs to the django application (as stated before, it is closed source and I do not have access to the source code of that module).

for the sake of trying to understand better what's going on, I removed the inheritance from Volume = type('Volume', superclasses, attrs) and created the volume just as Volume = type('Volume',(), attrs) then inspecting the module in this case shows that Volume is correctly associated to pluginmngr (pluginmngr.Volume) as one expects and not to chroma_core.lib.storage_plugin.base_resource

However, inheriting from some built-in resource class is compulsory and the problem is still there

so to summarize, could anyone explain why defining the Volume resource class in two different ways firstly by the class keyword, and secondly by the type built-in function (that should be perfectly interchangeable) leads in the first case to pluginmngr.Volume and to chroma_core.lib.storage_plugin.base_resource.Volume in the second case?

best regards

wissam
  • 1
  • 1
  • Have you tried to set the Volume.__module__ directly? – mdurant Jul 21 '14 at 14:24
  • thanks @mdurant for answering! Yes, trying to enforce the module does not resolve the issue but changes slightly the error message as follows: When enforcing the module then by inspecting it, Volume is reported as 'Volume', rather than as 'Volume', and the error message changes to PluginNotFound: PluginNotFound: Plugin not found (not one of ['pluginmngr', 'linux_network', 'linux']) even though /opt/wis/DxLustre/plugin/src/pluginmngr.py exists – wissam Jul 21 '14 at 17:54
  • It sounds like the proprietary code does some source-code introspection on the class upon loading, so you need the class to belong in a module on disc. Are there many classes in a session, or might it be feasible to create a file with a class definition dynamically and import that? – mdurant Jul 21 '14 at 18:10
  • @mdurant, I managed to move to code that creates the Volume class dynamically to a separate module and then import that module in the pluginmngr.py, unfortunately I get the same exact error (PluginNotFound: PluginNotFound: Plugin chroma_core.lib.storage_plugin.base_resource not found (not one of ['pluginmngr', 'linux_network', 'linux'])). Volume is still reported ad class 'chroma_core.lib.storage_plugin.base_resource.Volume – wissam Jul 22 '14 at 07:28
  • I guess I'm running out of ideas. You could dynamically produce code like your working class definition, and import that. Or perhaps you could define in code a bare class with the correct inheritance, and set all of its attributes at run time. – mdurant Jul 22 '14 at 14:55

0 Answers0