2

I have a basic query:

/*[fn:name()="something"]

(1) Marklogic 7 is taking multiple seconds, is there an index that I can add to make this query faster?

(2) Which in memory limits should be increased to improve performance?

(3) Are there other ways to improve the performance with a different query but get exactly the same result?

Aslan Brooke
  • 497
  • 2
  • 8

3 Answers3

5

Try using fn:node-name instead. I believe that's optimized. You'll need to handle namespaces properly, and that's part of why it can be optimized while fn:name can't.

 /*[fn:node-name()=fn:QName("","something")]
mblakele
  • 7,782
  • 27
  • 45
2

The following two xPaths should be the exact same:

/theNameOfMyElement

and

/*[fn:name()="theNameOfMyElement"]

The latter is adding an unnecessary and costly qualifier. First off, the * has to search for everything, not just elements. Several other problems exist with that approach.

If my first query is still taking a long time, use cts:search, which is much faster as it searches against indexes. The queries above can be written like this:

cts:search(/theNameOfMyElement, ())

Where the second parameter (empty sequence) can be a qualifying cts:query.

If namespaces are giving you fits, you can just do:

/*:theNameOfMyElement

/*[fn:name()="something"] seems like very bad practice to me. Use /something instead.

EDIT

After seeing that the other answer got accepted, I've been trying to think of what scenario you must be trying to solve if his solution worked and mine didn't. I'm still very certain that there is a faster way by just using xPath the way it was designed to work.

After some thought, I've decided your "real" scenario must either involve a dynamic element name or you may be trying to see if the element name matches one of a sequence of names.

I have drawn up a sample with it's output provided below, that demonstrates how you could still use both without using the qualifier based on fn:node-name

let $xml as element(sample) := <sample>
    <wrapper>
      <product>
        <entry>
          <red>$1.00</red>
          <yellow>$3.00</yellow>
          <blue>$4.50</blue>
        </entry>
      </product>
    </wrapper>
  </sample>
let $type as xs:string := "product"
return $xml/wrapper/xdmp:unpath($type)/entry/(red|yellow)

(: Returns 
  <red>$1.00</red>
  <yellow>$3.00</yellow> 
:)
CtheGood
  • 999
  • 8
  • 17
  • At scale, this comes down to how much of the work can be done by index lookups and how many documents have to be fetched and filtered. Doing more work in index lookups avoids time spent in document fetches and filtering. MarkLogic automatically indexes elements by QName, not by string name. This is why `/*[fn:node-name()=fn:QName("","fubar")]` can be fast, but `/*[fn:name()="fubar"]` is likely to be slow — and so is `/*:fubar`. I'm not sure about something like `document()/xdmp:unpath('fubar')` — but it is unlikely to be any faster than `fn:node-name`. – mblakele Feb 27 '16 at 16:50
  • The 2nd format seems to be a useful construct if you are accessing nodes programmatically. e.g. if you are passing the node name to a function as an xs:string you need the form /*[fn:name() = "blah"] option – chriskelly Feb 28 '16 at 20:07
  • 1
    Accessing nodes programmatically is great. But use `fn:node-name` not `fn:name`. This isn't just because of performance. Code that passes a node name as a string is highly suspect and probably buggy. It'll break when inputs have namespace declarations that violate its expectations, and node-processing code can't do much to control input namespace declarations. To be safe, use a QName with `fn:node-name`. Unlike a string, this is namespace-aware. – mblakele Feb 29 '16 at 05:29
1

In addition to the other good suggestions, consider applying pagination. MarkLogic can identify interesting content from indexes fast, but pulling up actual content from disk is relatively slow. And sending over all results at once could mean trying to hold all results (potentially billions) in memory before sending a reply over the wire. Pagination allows pulling up results in batches, which keeps memory usage low, and potentially allows parallelization as well.

HTH!

grtjn
  • 20,254
  • 1
  • 24
  • 35