Marklogic loading data using information studio flows (Invalid entity reference)

Question

I'm trying to load data using information studio flows. I have 1 xml document (1.65 GB). When I load it I got this error XDMP-DOCENTITYREF Invalid entity reference. I don't know how to solve it.

Answer 1

In XML, an entity reference is something like & -- the & and the ; is a code that refers to a character (in this case, the ampersand). The "Invalid entity reference" says that MarkLogic's parser doesn't recognize what's in there. It might be simply an unescaped "&" in the text, or it could be in the right form with a bad value. Check the error log for clues as to what you're looking for, or where it is in your source document.

While we're at it, you probably don't want to load a 1.65 GB document all as one thing. Is there content a level down that would make for good individual documents? (For instance, an RSS feed will be one XML document with multiple entries.) If you can split that document, you're much more likely to get good behavior.

To do that splitting, try using MarkLogic Content Pump's document splitting capability. MLCP was built for this type of work.