0

I'm using Delphi 7 XMLDoc unit for building a protocol definition on XML base, that is the protocol is defined in an XML file and some objects are get information from this XML for breaking-down and constructing ISO8583 messages delivered on TCP/IP. I widely make use of IXMLDocument and IXMLNode interfaces. Everything is fine, no memory leaks (using FASTMM) at point of termination. But during its running the software slowly consumes up memory. The memory is somehow released when freeing the object - derived from TXMLDocument - I'm using for XML access. I analysed and found that there is no memory problem if I did not use IXMLDocument and IXMLNode interfaces but of course I have to.

I have an object (TXMLTree) deriving from TXMLDocmument:

TXMLTree = class(TXMLDocument, IXMLDocument)
...
end;

I have another object (TDPR) and in its constructor I create TXMLTree and get the interface (fIDocument) plus an interface to the document's starting point (fIDocStart) as well: 

TDPR = class(TObject)
...
  fIDocument:IXMLDocument;
  fIDocStart:IXMLNode;
...
end;

constructor TDPR.Create(aFilename: string);
begin
  inherited; 
...
  fTree := TXMLTree.Create(aFilename);
  fTree.GetInterface(IXMLDocument, fIDocument);

  fIDocStart := fIDocument.DocumentElement;
...
end;

Usually I use fIDocument and fIDocStartas a starting point to reach the XML doc, writing code like this: 

node := fIDocument.DocumentElement.ChildNodes[aName].ChildNodes.FindNode(aFieldName);

or

node := fIDocStart.ChildNodes[aName].ChildNodes.FindNode(aFieldName);

Only one such line is enough to slowly increase memory usage, but if I perform it in a for loop of e.g. ten millions there is not more effect because - I guess - only one reference is used and released all the way in the loop.

Now let's have a procedure Foo where I get the document starting point:

procedure TDPR.Foo;
var lNode:IXMLNode;
begin
  lNode := IDocument.DocumentElement;
end;

If I call this procedure it increases memory usage a bit, but call it periodically and it is only question of time to run out of memory.
But: if I use fIDocStart which already stores the needed interface - so no need to call DocumentElement again and again - there is no memory problem. 

procedure TDPR.Foo;
var lNode:IXMLNode;
begin
  lNode := fIDocStart;
end;

So it seems that every call that gets and returns an interface object (IXMLDocument IXMLNode IXMLNodeList and so on) allocate some memory and does not frees it while reference objects are properly released. Even if we ask for the same node. Of course I cannot store all possible nodes just to avoid using up memory.

In TDPR's destructor I free TXMLTree by setting the correspondent interface to nil and no memory leaks is detected at termination. Reference counters seems to be ok.

destructor TDPR.Destroy;
begin
...
  fIDocStart := nil;
  fIDocument := nil;

...
  inherited; 
end;

And it is nothing to do with large XML files. The problem is the same whith XML like this:

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl"?>
<Base24-POS>
   <Instructions>
      <Parameters Receive="1"/>
   </Instructions>
</Base24-POS>

I know it is obscure and I only ask to get the chance that somebody has already bumped into the same problem and may found a solution. (BTW I remember using quite the same code in WindowsXP with no trouble. Now it is Windows7)

peter
  • 51
  • 4
  • And check that you are not storing interface references. – Runner Oct 25 '13 at 14:01
  • If I might store interface references that would cause memory leaks by not allowing the object to be freed. But why it would increase memory usage to the ultimate? BTW at program termination there is no memory leaks detected. – peter Oct 25 '13 at 14:46
  • Sorry guys, I forgot to mention that I extensively use `multithreading` and never the same thread gets the `xml` document interfaces. Now it seems that if I create and get interfaces from the same thread there is no memory 'leak'. May anybody know the reason for this? And that would be a hell to synchronize all running threads to e.g. the main thread when it needs an `xml` interface. Any other suggestion? – peter Oct 27 '13 at 17:06
  • I tried to solve this problem of multithreaded access by the use of `CoinitializeEx`'s parameter `COINIT_MULTITHREADED` but no use. – peter Oct 27 '13 at 19:26
  • The code you showed sofar does not pose any problem. Please try to make a complete example (SCCE) where you can replicate this behavior. Chances are big something is wrong with your threading code, not the XML related code... – whosrdaddy Oct 27 '13 at 21:53
  • There is not much code to show. I create an `TXMLDocument` object, and manipulate it via `IXMLDocument` and `IXMLNode` interfaces. If it all happens in a single thread, everything is ok, but in a multithreading approach, when one thread creates the `TXMLDocument` object and the others manipulates it by interfaces, it uses more and more memory. `CoInitializeEx(nil, COINIT_MULTITHREADED)` is called in every thread but in vain. – peter Oct 28 '13 at 15:44
  • Can you create/post a complete example that exhibits this behavior? I want to verify this in higher Delphi versions. – whosrdaddy Oct 28 '13 at 21:11
  • Sure. I have a small test application dedicated for this problem. Can I upload .pas file or do I have to edit it? – peter Oct 29 '13 at 10:05
  • I thought about it, and because the test application is really a Delphi `Form` with `buttons` and `timers` it would be much easier for you if I could send you the whole staff (e.g. .dfm, .pas, .res). Is it violates the rules of `stackowerflow` to get your mail address? – peter Oct 29 '13 at 14:01
  • @whosrdaddy I opend a new question dedicated to this multithreading problem where I put the whole code of a working Delphi form. – peter Oct 30 '13 at 04:08

1 Answers1

0

I found that the problem is somehow related to multithreading access to COM object (XMLDocument is COM). When creating TXMLDocument and manipulating it via interfaces such as IXMLDocument and IXMLNode on a single thread there was no memory "leak". But using different threads I could not eliminate the problem. I used CoInitializeEx with parameter COINIT_MULTITHREADED. It seems that every thread allocates some memory when getting an interface and does not free it, and every thread allocates it once - at least for a certain interface, so one thread does not cause visible memory leak. But dynamically created threads are all failed to free that memory and eventually consumes up process memory. Until now I have found no solution so I considered cleaning memory periodically with SetProcessWorkingSet(ProcessHandle, -1, -1) and that symptomatic treatment solved the problem until better solution is reached. I also opend a new question Memory consumption when using Delphi7 COM interfaces in a multithreaded way dedicated to this problem.

Community
  • 1
  • 1
peter
  • 51
  • 4