Dynamically loaded BPL's sharing code / passing objects

Question

I was toying around with the idea of using dynamically loading BPL's and passing object instances from the main app to a method in a BPL. This poses a problem units between used by the application and by the BPL.

I wrote a small little prototype which did this and was curious how Delphi internally manages differences between classes defined in the app vs. the BPL.

For example, say a basic Widget class like:

TmyWidget = class
private
  fId:Integer;
  fDescription:String;
public
  procedure DoSomething1();
end;

Now the app and the BPL are built using the unit containing TmyWidget class. Later, something changes in TMyWidget and the app is rebuilt, but the BPL is not (or vice-versa.) I added another method DoSomething2() and created an instance of TmyWidget in the app and passed it to the BPL for processing and in the basic example, it worked. But it's obviously fraught with potential problems.

If another dynamically loaded BPL also uses TmyWidget then things get even more interesting. It seems to work, but it definitely doesn't feel ideal.

The main question is - how does one typically pass objects to and from the main application and DLLs or BPLs? I've never attempted it before and likely for a good reason, but I've got this idea that lends itself to this approach...

I'd imagine that the best approach is to serialize the object and pass those bytes over and deserialize it in the DLL/BPL with this process being mindful of potential version differences between the host and the dynamically loaded module but I was hoping the new SimpleSharedMem option might bring this new functionality without the overhead of serialization, but it seems to be not very useful unless you are strict in keeping the app and dll rebuilt on any shared code changes...but in this prototype, the app would stay fairly constant and the dynamically loaded modules would be changing frequently with functionality being added to TmyWidget. (The server app serves as the factory for building TmyWidget's based on client requests and the app would pass instances to the various modules for processing.)

Surely the correct solution is to have only a single instance of each unit. Why do you have the same unit twice? Can't you organise the code to avoid that, like nature intended? — David Heffernan, Mar 09 '11 at 16:12
Yes, it could (should) be reorganized - it started by playing around with an idea that turned into loading BPL's and these BPL's, which started out as DLL's with some shared code, weren't segmented and for some reason it worked, and I was kinda wondering why. — Darian Miller, Mar 09 '11 at 17:29

score 10 · Accepted Answer · edited Mar 09 '11 at 16:45

10

...was curious how Delphi internally manages differences between classes defined in the app vs. the BPL

Delphi manages this by not allowing it. You can't have a unit with the same name in multiple packages at the same time: if you do, you get an error message saying something similar to Package XYZ already contains ABC (haven't seen that in a while...). Since the type name includes the unit name, you can't have the same type in two different packages. Unless it's a Interface defined by it's GUID, but that's a different story.

... how does one typically pass objects to and from the main application and DLLs or BPLs?

You don't pass objects to DLL, that's not a good idea. When you need to pass objects to a BPL, make sure the base class for that BPL is defined into an 3rd BPL.

Example. Polymorphic behavior for your TmyWidget is probably defined using some virtual methods. Make sure you have a TmyWidgetBase class that defines all of those virtual methods, derive all your TmyWidget's from that base class and pass around objects with the type TmyWidgetBase. Make sure the TmyWidgetBase class is in it's own Package.

When I attempted doing this I ended up with an tiny "bootstrap" exe and lot's of BPL's. Essentially all the logic was in BPL's, to facilitate passing objects around.

edited Mar 09 '11 at 16:45

David Heffernan

601,492
42
1,072
1,490

answered Mar 09 '11 at 16:26

Cosmin Prund

25,498
2
60
104

Note - I didn't get the "package already contains xx" error with two dynamically loaded BPLs that included the same TmyWidget unit in each (dynanically loaded using LoadPackage at runtime) It was partially confusing why it didn't error..the help says it cannot happen. – Darian Miller Mar 09 '11 at 17:20
@Darian, unfortunately I can't test this right now. Maybe Delphi doesn't do the tests for duplicate unit names if there's no "require" condition between the packages. – Cosmin Prund Mar 09 '11 at 18:45
I get the error if I build the app with runtime packages, elsewise no error. Then it's just acting like a normal DLL with initializations/finalizations being called. – Darian Miller Mar 09 '11 at 19:08
When you build the Exe with runtime packages you create the "require" relation I mentioned. Packages actually have the "requires" keyword in the DPK file, exe's don't, but it's the same thing. If you do that and Delphi doesn't immediately complain you'll end up with at least a few problems: The "is" and "as" operators aren't going to work properly because every package contains it's own copy of the class info structure.You'll be shipping an unnecessary large exe because it also contains it's own copy of the RTL and VCL (and you'll need to ship the BPL versions of those for the package's need). – Cosmin Prund Mar 09 '11 at 19:38
@Darian, did you try loading two packages both containing the same unit into your exe at runtime? That's what I wanted to test, can't do that right now because I've got some problems with missing run-time packages and I don't have the time (nor desire) to fix it. – Cosmin Prund Mar 09 '11 at 19:43
Yeah, it's getting way past funky-time...a little late-night idea gone amok. If everything is basically in runtime packages then it gets super funky when of the base packages need updated and there are a handful of dependant BPLs loaded that need to be swapped out while the server is live servicing client requests. Back to blackbox DLL's and off of this dynamic runtime package loading circular logic spree.. Thanks for the lifeline! – Darian Miller Mar 09 '11 at 19:50
And yes, I loaded two packages both containing the same unit at runtime with LoadPackage. Even the main app included the unit to make it a happy 3 copies of the class definition loaded with a potential nightmare of all 3 having different implementations of the class. Delphi allowed it, and it 'worked' even when I extended the class in one of the BPL's and then again in the main app, but it's certainly a nightmare waiting to happen to try it in anything other than a test app. – Darian Miller Mar 09 '11 at 19:55
You should never ever use LoadPackage with an application that wasn't compiled with at least the rtl package as runtime package. – Thorsten Engler Mar 10 '11 at 03:32
@Thorsten - can you add that to the Wiki? :) I've never used LoadPackage before...was just chasing a wild idea. – Darian Miller Mar 10 '11 at 04:22
1

@Darian, packages work very well when you want to be able to pass objects between modules (exe/dll's). But it's important that you only pass types across module boundaries which have been defined in a package that is being required by both modules (for packages that's the requires clause, for exe's and dll's that's the runtime packages that you listed in the compiler options). – Thorsten Engler Mar 10 '11 at 08:40
1

*Duplicate unit checks are bypassed if the AValidatePackage function returns "True." Be warned that this may cause strange and unpredictable behaviour if two packages are loaded that contain the same units and types.* - from SysUtils.pas. Didn't see this anywhere in the documentation. – Kenneth Cochran Mar 10 '11 at 16:33
@CodelElegance...thanks, I did see that comment in the source after the fact. Basically, Delphi allows it to happen, but they don't guarantee what's going to happen when you do it. I'm staying away from it due to updates to more than one BPL at a time on the fly. – Darian Miller Mar 10 '11 at 16:48
If you find the limitations of a standard dll to be too cumbersome you should investigate Com programing. Delphi has excellent support for it and it affords you some of the same benefits as using bpls. – Kenneth Cochran Mar 10 '11 at 17:58

score 7 · Answer 2 · edited Jun 20 '20 at 09:12

One of the projects I've worked on has successfully used a large number of runtime packages for more than a decade now so I'll share a few of my experiences dealing with packages.

As Cosmin pointed out different packages cannot contain the same units. If you use implicit linking, by adding a package to the requires clause of another package or by adding a package to the the Runtime packages list in Project Options the compiler will do the work for you and report one of the following error messages:

E2199: Packages '%s' and '%s' both contain unit '%s' (if your compiling project that depends on two packages containing the same unit)

E2200: Package '%s' already contains unit '%s' (if your compiling a package that contains a unit this is already contained in one of the packages it depends on)

If you are using explicit linking, using LoadPackage, a check will normally be attempted a runtime (though it can be circumvented) and raise an:

EPackageError: Cannot load package '%s.' It contains unit '%s', which is also contained in package '%s'

Resolving these errors isn't really all that difficult.

If you have two packages that both need to use a unit just let one of them contain the unit and the other one require the first. package dependency graph

If you have two packages that need to use each other's contained unit you'll have to move those units to an new package that both can depend on. dependency graph

Implicitly linked packages have the advantage that you can directly access class definitions as though they were statically linked. Just add a unit to the uses clause of the unit you need to use it in. The compiler and the runtime environment take care of resolving everything.

Explicitly linked packages will need to rely on class registration in the initialization section.

Dynamically loaded BPL's sharing code / passing objects

2 Answers2