3

I have a problem similar to this, although hopefully I've narrowed it down a little.

I am currently testing the publishing part of my Tridion 2011 installation (Conent Manager and Content Delivery on separate servers). When I try to publish many pages at once (1000 in my case) the process dllhst3g.exe *32 starts to acquire a large amount of memory. This is understandable as there is a lot of work to do but it never gives it back. This is causing the content manager to run slowly and eventually crash.

I was getting out of memory exceptions being thrown by the Tridion Content Manager when the dllhst3g.exe *32 process had around 3.6GB of memory (I assume this is due to it being a 32 bit process). To prevent these I limited the SDL Tridion Content Manager COM+ Application to an arbitrary 500Mb per process which gets forcibly terminated 15 minutes after the limit is reached. This has stopped the out of memory exceptions for now but I still have performance issues and a potential to run out of memory if more than my servers 8GB is allocated in the 15 minute window. More about this here

I have ruled out the underlying PageTemplate code as the associated page template has no code associated. These pages are blank.

The issue also seems to be much more prominent when using the Core Service API. Code I am using is

using (var client = new Tridion2011CoreService.CoreServiceClient())
{
    foreach (var id in ids) // ids is a collection of 500 page ids
    {
        // publishing to staging and live
        var targets = new string[] { "tcm:0-7-65538", "tcm:0-8-65538" };

        var publishInstructionData = new PublishInstructionData();
        publishInstructionData.ResolveInstruction = new ResolveInstructionData();
        publishInstructionData.RenderInstruction = new RenderInstructionData();

        var readOptions = new ReadOptions();

        client.Publish(new string[] { id }, publishInstructionData, targets, PublishPriority.Normal, readOptions);
    }
}

(I realise I could send all the ids through with one call to publish but then I hit a message limit error and as far as I am aware the result of multiple calls and single calls is effectively the same)

Any ideas?

(let me know if I have missed out any details and I'll update the question accordingly)

Forgot to add. The Content Manager is installed on a virtualised Windows Server 2008 with 4 CPUs, 8GB RAM and 50GB disk space. The Content Delivery is installed on a separate server with the same specs.

Community
  • 1
  • 1
Kevin Brydon
  • 12,524
  • 8
  • 46
  • 76
  • I see you are publishing items from within templates. This could easily lead to an huge number of items in the publish queue. Could it be that you are simply asking too much of the publisher? And have you looked at the number of threads for rendering? – Quirijn Oct 04 '12 at 10:00
  • Thanks for your response. What do you mean by "publishing items from within templates"? Each page is assigned the same blank page template i.e. no code. The page has no components. The html pages that are output are blank files. I realise that publishing 1000 pages is maybe a fairly heavy task but soemthing I would think an CMS system should be able to handle. I think the main issue is that it seems the memory being allocated is not being released. – Kevin Brydon Oct 04 '12 at 10:41
  • Sorry, I misread your question. I thought that snippet of CoreService code came from your template itself. – Quirijn Oct 04 '12 at 11:09
  • Ah right. The code is from an external win forms application running on my PC that calls the Core Service API – Kevin Brydon Oct 04 '12 at 11:34
  • 1
    You state "as far as I am aware the result of multiple calls and single calls is effectively the same". Let me make you aware then: if you pass all ids in a single call, you're creating a single publish transaction with a 1000 items. As it is now, you're creating 1000 publish transactions each with a single item. The resulting behavior will be completely different. That said: I don't know why you'd run out of memory, but I also don't think limiting and killing processes like you do is a solution. Stop doing that and use performance monitor would be my suggestion. – Frank van Puffelen Oct 04 '12 at 12:36
  • I have tried publishing 10 pages one-by-one (calling `Publish(...)` 10 times) I end up with 10 entries in the Publishing Queue each with their own Transaction ID. I have also tried publishing the same 10 pages as a single transaction (calling `Publish(...)` once) and I again end up with 10 entries in the Publishing Queue each with their own unique Transaction ID. Could you explain how these are different? – Kevin Brydon Oct 04 '12 at 13:17
  • Hmmm... I thought the latter should result in a single publish transaction. But that might indeed only happen if you handle the page-explosion in a custom resolver. :-/ – Frank van Puffelen Oct 04 '12 at 18:41
  • No problem. I've sent a request to customer support and will update this when/if an answer is found. – Kevin Brydon Oct 04 '12 at 20:57

2 Answers2

1

After contacting SDL customer support I was told that the only solution is to recycle the dllhost process when it takes up a pre-determined amount of memory.

To do this for SDL Tridion Content Manager 2011 running on Windows Server 2008

  1. Open Component Services (either search for it or Control Panel -> Administrative Tools)
  2. Expand the tree Component Services -> Computers -> My Computer -> COM+ Applications
  3. Right click SDL Tridion Content Manager and select Properties
  4. Select the Pooling & Recycling tab
    • Set Pool Size to 1
    • Set Lifetime limit to 0
    • Set Memory Limit to 524288 (512 MB) or choose your own limit (I'd go for a value less than 1GB)
    • Set Call Limit to 0
    • Set Activation Limit to 0
  5. If you can, restart your system. Otherwise, restart all the Tridion Services (Control Panel -> Administrative Tools -> Services and restart everything that starts with "Tridion")

Maybe the "real" fix is but a patch away...

Kevin Brydon
  • 12,524
  • 8
  • 46
  • 76
0

Try monitoring the transport package root folder. This would be set in SDL Tridion MMC snap in. Default value would be C:\temp in CMS server. Please check the work area folder of transport which would be set in cd_transport config xml in Tridion installed folder\Config. Also check for the deployer incoming and work folders.Large volume of files accumlating in these folders may cause in poor performance and try cleaning up these folders if you have errored out folders or files to prevent Tridion from retrying the failed transactions. Also if your deployer is runnning as a website-httpupload.asp, try verifying the application domain recycle settings of deployer just to make sure you're not resetting appdomain every few minutes.

You could also try looking in to related items of your publish items.If you have lot of versions or unwanted related items, you could think of custom resolving of publish items to avoid unwanted components with a particular schema etc.Nuno has provided a very good article in below link. Thanks to Nuno.. http://nunolinhares.blogspot.nl/2011/10/tridion-publisher-and-custom-resolvers.html

Krishnakumar
  • 243
  • 1
  • 11