Using SSIS Script task to read XML from variable object LoadXml fails

Question

I have a SQL statement (SQL server 2014) that builds out XML formatted data. I am pulling this into an SSIS (2013) object variable. If I put the SQL into a data flow task to go into a text file, it works. BUT, no encoding header. Because of the size of the return data, converting to a NVARCHAR is not an option - data would get cut off.

So, instead I have a sql task, same query: ResultSet = XML; Variable datatype = Object (it fails if I try to string)

So, after much time here - I found all sorts of great stuff to use XmlWriter and XmlWriterSettings. Awesome, great. BUT -- I can't seem to get the script task (C#) to read (or format/define to) the XML object correctly. I keep getting a "DTS Script Task has encountered an exception in user code"

I set a break point at the LoadXML statement - verified the variable does have the XML data - tho interestingly enough it gets surrounded with ... . In debug mode the specific error is "Data at the root level is invalid. Line 1, position 1."

It throws the exception at the LoadXML statement -- what am I missing? Do I need to define the XML layout (fields)? The commented out test string works w/o issue. no fail. header written.

Any ideas?

Update: it also fails with a string version of my test data, without the root tag.

public void Main()
        {

            Variables varCollection = null;
            Dts.VariableDispenser.LockForWrite("User::PCC_XML");
            Dts.VariableDispenser.GetVariables(ref varCollection);

            XmlDocument xdoc = new XmlDocument();

            XmlWriterSettings settings = new XmlWriterSettings
            {
                Encoding = Encoding.UTF8,
                ConformanceLevel = ConformanceLevel.Document,
                OmitXmlDeclaration = false,
                CloseOutput = true,
                Indent = true,
                IndentChars = "  ",
                NewLineHandling = NewLineHandling.Replace
            };    

            xdoc.LoadXml(varCollection["User::PCC_XML"].Value.ToString());

            //xdoc.LoadXml("<xml><foo></foo></xml>");

            using ( StreamWriter sw = File.CreateText("C:\\test_xml.xml") )
            using ( XmlWriter writer = XmlWriter.Create(sw, settings))
            {
                xdoc.WriteContentTo(writer);
                writer.Close();
            }    

            Dts.TaskResult = (int)ScriptResults.Success;
        }

a skinnied down version of my XML is (including the ROOT that got added)

<ROOT>
    <Universal_letters>
        <Universal_letter>
            <docID>123456</docID>
            <Letter_Code>123</Letter_Code>
            <Callback_Phone>1-888-111-111</Callback_Phone>
            <Delivery_Methods>
                <Delivery_Method />
                <Print_Queue />
            </Delivery_Methods>
            <Requested_Date>2016-07-28</Requested_Date>
            <Consumer_First_Name>Bubba</Consumer_First_Name>
            <Consumer_Last_Name>Bubbster</Consumer_Last_Name>
            <Consumer_Address_1>123 Street Way</Consumer_Address_1>
        </Universal_letter>
    </Universal_letters>
</ROOT>

Sorry, I am not overly familiar with xml so I approached this though SQL. Below is some TSQL that will break down the data into 4000 character outputs. Utilizing this logic you could do this in a loop in the control flow of the ssis package or do something similar in c#. Sorry, this is not very readable as a comment. declare @v varchar(max) = '' declare @c int; set @count = len(@v)/4000 if (len(@v)%4000 > 0) set @c = @c + 1 declare @i int = @c while (@count > 0) begin select substring(@v,0+(4000*(@i-@c)),4000+(4000*(@i-@c))) set @c = @c - 1 end — Andrew O'Brien, Jul 28 '16 at 19:57

score 0 · Answer 1 · answered Jul 29 '16 at 12:32

Have you tried removing the spacing between XML elements or checked for any special characters. Your XML is fine and the C# Code in Script Task is also doing it's job. I recreated the same scenario as mentioned in your query. So, I think that the issue really is with the actual XML that is being generated at runtime and set as input to the PCC_XML variable. There might be something fishy in the way it may be getting written to the variable. Some spacing or Special characters may cause the same to happen. Please mark it as answered if it helps you.

score 0 · Answer 2 · edited May 23 '17 at 11:44

I couldn't get my XML sample to work as Mohd suggested. Today, however, I found this http://www.sql-server-performance.com/2013/export-to-xml-using-ssis/ (hich used a string as the result set, and a script task to convert & export the XML.) combined with https://stackoverflow.com/a/14840090/4509043 - for encoding. Got me where I needed to be.

Interestingly enough, the c# script task didn't truncate any of my big XML data - unlike an NVARCHAR(MAX). So that's awesome! It's also worth mentioning that OLE DB connections will not work, it must be ADO.NET.

Final Solution: User::PCC_XML is a string datatype. The SQL task has single row results - XML type.

My final - working C# code is:

before namespace declare - with other namespaces

using System.Xml; //added - for xml
using System.Text; //added - for Encoding
using System.IO; // added - for file writing

main proc

public void Main()
        {

            XmlDocument xdoc = new XmlDocument();

            // will ensure that XML exported will have opening tag
            // <?xml version="1.0" encoding="UTF-8"?>
            XmlWriterSettings settings = new XmlWriterSettings
            {
                Encoding = Encoding.UTF8,
                ConformanceLevel = ConformanceLevel.Document,
                OmitXmlDeclaration = false,
                CloseOutput = true,
                Indent = true,
                IndentChars = "  ",
                NewLineHandling = NewLineHandling.Replace
            };

            // strip out the <root> </root> tags on insert into string
            string strXML = Dts.Variables["User::PCC_XML"].Value.ToString().Replace("<ROOT>", "").Replace("</ROOT>", "");

            xdoc.LoadXml(strXML.ToString());

            // test destination
            using ( StreamWriter sw = File.CreateText("C:\\test_xml.xml") )
            using ( XmlWriter writer = XmlWriter.Create(sw, settings))
            {
                xdoc.WriteContentTo(writer);
                writer.Close();
            }

            Dts.TaskResult = (int)ScriptResults.Success;
        }

Using SSIS Script task to read XML from variable object LoadXml fails

2 Answers2