BizTalk Custom Pipeline Component System.OutOfMemoryException

Question

I'm currently developing a BizTalk Custom Send Pipeline that accepts an xml file and converts it to Excel. Unfortunately, after deploying the pipeline, I'm receiving a System.OutOfMemoryException. I've included the code for Execute Method of IComponent interface. All suggestions are welcome.

public Microsoft.BizTalk.Message.Interop.IBaseMessage Execute(IPipelineContext pContext, Microsoft.BizTalk.Message.Interop.IBaseMessage inmsg)
    {

        MemoryStream outMemStream = new MemoryStream();
        try
        {

            if (inmsg.BodyPart.Data != null)
            {
                // Read the source message coming from the messaging engine and convert it to memory stream 
                byte[] buffer = new byte[16 * 1024];
                using (MemoryStream ms = new MemoryStream())
                {
                    int read;
                    while ((read = inmsg.BodyPart.Data.Read(buffer, 0, buffer.Length)) > 0)
                    {
                        ms.Write(buffer, 0, read);
                    }
                    buffer = ms.ToArray();
                }

                if (buffer != null)
                {
                    var binaryWriter = new BinaryWriter(outMemStream);
                    binaryWriter.Write(buffer);
                }

                OpenXMLOffice oOffice = new OpenXMLOffice();
                outMemStream.Position = 0;
                oOffice.XMLToExcel(outMemStream, TemporaryFileLocation);
                inmsg.BodyPart.Data.Position = 0;
                inmsg.BodyPart.Data = outMemStream;
                pContext.ResourceTracker.AddResource(outMemStream);
            }

            return inmsg;
        }
        catch (Exception ex)
        {
            throw new ApplicationException(String.Format("Error converting XML to Excel:{0} - Stack Trace: {1}", ex.Message, ex.StackTrace));
        }
    }

Here is the most recent error received:

Log Name:      Application
Source:        BizTalk Server
Date:          2/14/2012 9:29:00 AM
Event ID:      5754
Task Category: BizTalk Server
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      IASDev-PC
Description:
A message sent to adapter "FILE" on send port "ExcelSendPort" with URI "C:\SeleneFTPFile\Excel\%MessageID%.xml" is suspended. 
 Error details: There was a failure executing the send pipeline: "IAS.SeleneFTPFile.ExcelEncodePipeline, IAS.SeleneFTPFile, Version=1.0.0.0, Culture=neutral, PublicKeyToken=2add433e7764165f" Source: "Excel File Encoder" Send Port: "ExcelSendPort" URI: "C:\SeleneFTPFile\Excel\%MessageID%.xml" Reason: Error converting XML to Excel:Exception of type 'System.OutOfMemoryException' was thrown. - Stack Trace:    at System.IO.MemoryStream.set_Capacity(Int32 value)
   at System.IO.MemoryStream.EnsureCapacity(Int32 value)
   at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at IAS.SeleneFTPFile.Components.ExcelPipeline.EncodeExcel.Execute(IPipelineContext pContext, IBaseMessage inmsg)  
 MessageId:  {ED37CDD1-EF0C-46E7-9519-061AF3D4F8A4}
 InstanceID: {B0E448B3-3DAD-4E52-8F87-07C5D5AA5224}

One thing possibly - BinaryWriter is also IDisposable - http://stackoverflow.com/questions/1079434/do-we-need-to-close-a-c-sharp-binarywriter-or-binaryreader-in-a-using-block — StuartLC, Feb 14 '12 at 07:59
@nonnb so should I add the BinaryWriter to the PipelineContext's ResourceTracker? — dustyhoppe, Feb 14 '12 at 16:09
you've scoped the binaryWriter very tightly (good) but by explicitly disposing you may free the binarywriter earlier than GC gets around to collecting it. I doubt that this is the main cause of the OutofMemoryException though. — StuartLC, Feb 15 '12 at 06:51

score 1 · Answer 1 · edited Oct 02 '12 at 07:47

1

Even I face the same issue. I can find that the reason is the stream position does not advance and remains at 0 for the inmsg.BodyPart.Data stream, even after the below statement executes:

read = inmsg.BodyPart.Data.Read(buffer, 0, buffer.Length)

edited Oct 02 '12 at 07:47

j0k

22,600
28
79
90

answered Sep 29 '12 at 22:03

Rahul Vardarajan

11
1

score 1 · Answer 2 · edited May 23 '17 at 12:20

1

You could try pre-allocating the MemoryStream buffer (as suggested here). The error message states that it can't allocate enough memory for the line when performing the buffered write (ms.Write(buffer, 0, read);).

using (MemoryStream ms = new MemoryStream(buffer.Length))

The other issue you have is that your buffer could overflow - producing this exact error when the MemoryStream is written to (ms.Write(buffer, 0, read)).

byte[] buffer = new byte[2 * 1024 * 1024]; // try increasing to 2MB buffer

edited May 23 '17 at 12:20

Community

1
1

answered Feb 15 '12 at 15:06

SliverNinja - MSFT

31,051
11
110
173

@SilverNinja, I modified my MemoryStream by pre-allocating it using the buffer. Unfortunately, I'm still receiving the same error. Thanks for the suggestion though. – dustyhoppe Feb 15 '12 at 17:38
You need to increase the size of your buffer. The `byte[]` you created is limited to 16K...try doubling it or multiplying it by another 1024 to give you 15MB instead. You need to determine the size of the incoming payload to predict what size buffer you should use. – SliverNinja - MSFT Feb 15 '12 at 17:51

BizTalk Custom Pipeline Component System.OutOfMemoryException

2 Answers2