Perl: Dynamic module loading, object inheritance and "common helper files"

Question

In a nutshell I try to model a network topology using objects for every instance in the network. Additionally I got a top-level manager class responsible for, well, managing these objects and performing integrity checks. The filestructure looks like this (I left out most of the object-files as they are all structured pretty equal):

Manager.pm
Constants.pm
Classes/
  +- Machine.pm
  +- Node.pm
  +- Object.pm
  +- Switch.pm

Coming from quite a few years in OOP, I'm a fan of code reuse etc. so I set up inheritance between thos objects, the inheritance tree (in this example) looks like this:

Switch  -+-> Node -+-> Object
Machine -+

All those objects are structured like this:

package Switch;
use parent qw(Node);

sub buildFromXML {
  ...
}
sub new {
  ...
}

# additonal methods

Now the interesting part:

Question 1

How can I ensure correct loading of all those objects without typing out the names statically? The underlying problem is: If I just require "$_" foreach glob("./Classes/*"); I get many "Subroutine new redefined at" errors. I also played around with use parent qw(-norequire Object), Module::Find and some other @INC modifications in various combinations, to make it short: It didn't work. Currently I'm statically importing all used classes, they auto-import their parent classes.
So basically what I'm asking: What is the (perl-)correct way of doing this?
And advanced: It would be very helpful to be able to create a more complex folder structure (as there will be quite a few objects) and still have inheritance + "autoloading"

Question 2 - SOLVED

How can I "share my imports"? I use several libraries (my own, containing some helper functions, LibXML, Scalar::Util, etc.) and I want to share them amongst my objects. (The reasoning behind that is, I may need to add another common library to all objects and chances are high that there will be well above 100 objects - no fun editing all of them manually and doing that with a regex / script would theoretically work but that doesn't seem like the cleanest solution available)
What I tried:

import everything in Manager.pm -> Works inside the Manager package - gives me errors like "undefined subroutine &Switch::trace called"
Create a include.pl file and do/require/use it inside every object - gives me the same errors.
Some more stuff I sadly don't remember

include.pl basically would look like that:

use lib_perl;
use Scalar::Util qw(blessed);
use XML::LibXML;
use Data::Dumper;
use Error::TryCatch;
...

Again I ask: What's the correct way to do it? Am I using the right approach and just failing at the execution or should I change my structure completely?
It doesn't matter that much why my current code doesn't work that well, providing a correct, clean approach for those problems would be enough by far :)

EDIT: Totally forgot perl version -_- Sidenote: I can't upgrade perl, as I need libraries that are stuck with 5.8 :/

C:\> perl -version
This is perl, v5.8.8 built for MSWin32-x86-multi-thread
(with 50 registered patches, see perl -V for more detail)

Copyright 1987-2006, Larry Wall

Binary build 820 [274739] provided by ActiveState http://www.ActiveState.com
Built Jan 23 2007 15:57:46

I was just passing (don't have time to wade through that lot, sorry) and I noticed you have `package Switch;` and you are on 5.8. There was a core module called `Switch` in 5.8 (Damian's ill-fated "case" statement), so that's a bad idea. — cdarke, Sep 14 '12 at 09:56
Internally I use SwitchDescription as package name, I chose the short name only for the sake of readability, so that shouldn't be a problem. But thanks anyway :) — incaseoftrouble, Sep 14 '12 at 10:07

score 3 · Accepted Answer · edited Jun 20 '20 at 09:12

This is just a partial answer to question 2, sharing imports.

Loading a module (via use) does two things:

Compiling the module and installing the contents in the namespace hierarchy (which is shared). See perldoc -f require.
Calling the import sub on each loaded module. This loads some subs or constants etc. into the namespace of the caller. This is a process that the Exporter class largely hides from view. This part is important to use subs etc. without their full name, e.g. max instead of List::Util::max. See perldoc -f use.

Lets view following three modules: A, B and User.

{
   package A;
   use List::Util qw(max);
   # can use List::Util::max
   # can use max
}
{
   package User;
   # can use List::Util::max -> it is already loaded
   # cannot use max, this name is not defined in this namespace
}

Package B defines a sub load that loads a predefined list of modules and subs into the callers namespace:

{
   package B;
   sub load {
     my $package = (caller())[0]; # caller is a built-in, fetches package name

     eval qq{package $package;} . <<'FINIS' ;
       use List::Util qw(max);
       # add further modules here to load
       # you can place arbitrarily complex code in this eval string
       # to execute it in all modules that call this sub.
       # (e.g. testing and registering)
       # However, this is orthogonal to OOP.
FINIS

     if ($@) {
       # Do error handling
     }
   }
}

Inside the eval'd string, we temporarily switch into the callers package and then load the specified module. This means that the User package code now looks like this:

{
   package User;
   B::load();
   # can use List::Util::max
   # can use max
}

However, you have to make sure the load sub is already loaded itself. use B if in doubt. It might be best to execute B::load() in the BEGIN phase, before the rest of the module is compiled:

{
  package User;
  BEGIN {use B; B::load()}
  # ...
}

is equivalent to

{
  package User;
  use B;
  use List::Util qw(max);
  # ...
}

TIMTOWTDI. Although I find evaling code quite messy and dangerous, it is the way I'd pursue in this scenario (rather than doing files, which is similar but has different side effects). Manually messing with typeglobs in the package namespace is hell in comparision, and copy-pasting a list of module names is like going back to the days when there wasn't even C's preprocessor.

Edit: `Import::Into`

… is a CPAN module providing this functionality via an interesting method interface. Using this module, we would redefine our B package the following way:

{
  package B;
  use List::Util;   # you have to 'use' or 'require' this first, before using 'load'.
  use Import::Into; # has to be installed from CPAN first
  sub load {
    my $package = caller;
    List::Util->import::into($package, qw(max));
    # should work too: strict->import::into($package);
    # ...
  }
}

This module hides all the dirty work (evaling) from view and does method call resolution gymnastics to allow importing pragmas into other namespaces.

Thanks for the answer! I also thought of something like that, but as you said, `eval`ing **is** kinda messy and I'd rather only use it as a "last resort" solution. BTW what may be worth noting in your first snippet: if `package User` has a `use A` directive in it, you should be able to call `A::max` inside the `User` package. This train of thought helped me understanding the principles of `use` and `require` a lot — incaseoftrouble, Sep 14 '12 at 10:34
@Megge The problem is that the `package` function doesn't take an arbitrary expression, so the namespace has to be hardcoded. So without `evaling` it is difficult (impossible?) to switch into a namespace specified at runtime. `import`ing happens only into the current package :( And yes, `A::max` is available, but using that doesn't feel right. — amon, Sep 14 '12 at 10:41
The thing is I have the impression that there should be a halfway decent approach in a "new" language like perl. Maybe there is a solution / a module that uses a completely different approach. I'll just wait if anyone comes up with a solution over the weekend, if not I'll use yours. — incaseoftrouble, Sep 14 '12 at 10:56
@Megge Actually, there is a module that does this: [`Import::Into`](http://search.cpan.org/~mstrout/Import-Into-1.001000/lib/Import/Into.pm). Suprise, it uses `eval` too, and just provides syntactic sugar using some *really* funky code … (Perl—a "new" language? born 1987 — only 4 years younger than C++ —reborn 1994 as Perl 5, it has a longer history than either Python or Java) — amon, Sep 14 '12 at 11:25
Good thing about that sugar is, it (should) work and I have one problem less to care about :) Does "really funky" mean "should not use" or would you recommend it? ("New" in a sense that it's not something like BASIC, Fortran etc. and it has OOP. Perl feels so intuitive regarding so many aspects that it really seems strange that such a common task does not have a solid solution. (On the other hand I had my first encounter with perl only 4 weeks ago, so the "intuitive" feeling might fade soon ;) ) — incaseoftrouble, Sep 14 '12 at 11:45
"Funky" means: I didn't understand the code on first sight. But now that I do, it is absolutely beautiful and it does have a few edges over my approach. I am adding a short usage section to my post in a minute. — amon, Sep 14 '12 at 14:13
Thanks alot! I guess Question #2 is answered. I guess if I mark your answer as solution, the whole question gets marked as solved, so I'll accept your answer when everything is solved. — incaseoftrouble, Sep 15 '12 at 07:04

score 0 · Answer 2 · answered Dec 01 '13 at 21:08

Addendum to Import::Into Solution

I found a scenario that seems to require eval() from within the Import::Into solution. In this scenario, mod User is effectively among the uses from package B. This may be a common scenario for people using Import::Into.

Specifics:

I created module uses_exporter with separate subs for importing different groups of modules, e.g. load_generic() and load_list_utils().
The uses in load_list_utils() are to public mods like List::MoreUtils, AND to a module of my own, list_utils_again. That local module also calls load_list_utils(). The call fails if load_list_utils() uses list_utils_again.
My solution was to put the use to list_utils_again into an eval which does not excecute when $target eq 'list_utils_again'

score 0 · Answer 3 · answered Dec 01 '13 at 21:17

The correct idiomatic Perl way to do this is not to always load a bunch a modules whether used or not; it is to have every file use those modules it directly (not indirectly) needs.

If it turns out that every file uses the same set of modules, you might make things simpler by having a single dedicated module to use all those in that common set.

Perl: Dynamic module loading, object inheritance and "common helper files"

Question 1

Question 2 - SOLVED

3 Answers3

Edit: Import::Into

Edit: `Import::Into`