XQuery file returning invalid entity reference using special characters

Question

I have the following query in a MarkLogic XQuery file, and I am seeing the following error message returned

XDMP-ENTITYREF: (err:XPST0003) Invalid entity reference " " . See the MarkLogic server error log for further detail.

The following is the code I am using in the XQuery file.

xquery version "1.0-ml";

declare variable $query := 

  cts:or-query
  ((
    cts:element-word-query(xs:QName("lines"),"l&amp;l"),
    cts:element-word-query(xs:QName("lines"),"pool &amp; cue"),
    cts:element-word-query(xs:QName("lines"),"look")
  ));

declare function local:do-query(){
  element xml {
    for $i in cts:uris( (), (), $query)
    let $item := doc($i)

    return
      element item {
        element title { $item/title/string() }
    }
  }
};

local:do-query()

Obviously the 2x tags i am looking for are l&l and pool & cue. I have also looked into the repair-full suggestion in another question posted, but couldn't figure out how that fits into this query. If I removed the ones with special characters, it works as expected.

Any ideas?

This code runs just fine in QConsole, so perhaps there is a different issue. How are you invoking the code, and are you sure you are invoking above code and not something else? Could it be a data issue instead? — grtjn, Apr 07 '17 at 18:28
Yes the code runs fine in qconsole agreed, but when uploading said code into a .xqy file and trying to run it in a browser, it's there where the error message is returned. Expected behaviour is to see the xml results as observed via the console. — Key, Apr 07 '17 at 18:55
How did you upload the .xqy file, and did you check with for instance QConsole Explore feature on your modules database whether the .xqy file once loaded into the database looks correct? — grtjn, Apr 09 '17 at 17:52
I'm using 'xdmp:document-insert' to insert the file to the modules database. Great point regarding how it shows up in the .xqy file. Both element word queries are showing up as 'l & l' and 'pool & cue', which isn't the expected behaviour either. — Key, Apr 10 '17 at 08:41
Yes, it is less complicated to load .xqy files from disk when inserting or uploading them. There are tools that can help with deployment, and you can use rest apis for that too. — grtjn, Apr 10 '17 at 09:58
Why would the document-insert automatically convert those encoded values back to the original special characters? i'm assuming that's what the issue is. Is there anyway my query can be slightly adapted to get it working, otherwise i'll look into inserting the files using curl — Key, Apr 10 '17 at 10:16
I have to make a guess on how you actually do it, but I guess you use QConsole to do the doc-insert of the module. But a value like `&` within a query, is `&` as string. You QC code is read as query, put persisted as string. You would need to escape the amp to `&` to make it work. Inserting from disk skips the parsing of amp in query context, causing less mistakes.. — grtjn, Apr 10 '17 at 11:42
Feel free to write that as an answer for the tick mark, works exactly how i wanted thanks :). Will be looking into uploading via curl for future reference. — Key, Apr 10 '17 at 15:47

score 2 · Accepted Answer · answered Apr 10 '17 at 17:38

Based on the additional info in the comments to the question, this is not an issue with the execution of the code, but rather with deployment of the code.

This happens often if you insert code using QConsole, or some other ways in which you evaluate XQuery code. The & get interpreted, and translated to the & character it represents. If you then write that into a .xqy file into some Modules database, it does not get escaped back into & again, since XQuery files are stored as plain text in MarkLogic, and & doesn't get escaped in plain text.

A better way to deploy code is by uploading or inserting from disk. That way characters like &, >, and { inside XML won't get interpreted, but preserved and inserted as is. There are tools like ml-gradle and Roxy that make deploying MarkLogic code very easy. Consider using these. Alternatively you could also look into using Curl against the Management REST api.

If you want to use QConsole after all, escape characters like & twice. E.g. & becomes &amp;, and < becomes &lt;.

HTH!

XQuery file returning invalid entity reference using special characters

1 Answers1