How to have sorted results when using Data Movement API?

Question

I'm using marklogic 9 and data movement api to export the results of a search. I create a query like:

StructuredQueryDefinition query = ...
String sortOptionsXml = "<sort-order  direction=\"" + sortDirection + "\">" + "<path-index>" + sortIndex + "</path-index>" + "</sort-order>";
String queryAsXml = "<search xmlns=\"http://marklogic.com/appservices/search\">" + query.serialize()
                + "<options><search-option>filtered</search-option>\n" + sortOptions + "</options>" + "</search>"
RawCombinedQueryDefinition combinedQueryDefinition = queryManager
            .newRawCombinedQueryDefinition(new StringHandle().with(queryAsXml).withFormat(Format.XML));

and then I send this query to a QueryBatcher, created like:

QueryBatcher batcher = databaseClient.newDataMovementManager().newQueryBatcher(query)

and start the batcher as:

batcher.withBatchSize(500)
        .withThreadCount(8)
        .onUrisReady(
                new QueryBatchListener() {
                    @Override
                    public void processEvent(QueryBatch queryBatch) {
                        LOGGER.info(String.join(",", queryBatch.getItems()));
                    }
                }
        )
        .onQueryFailure(Exception::printStackTrace);
JobTicket jobTicket = dataMovementManager.startJob(batcher);
batcher.awaitCompletion();
dataMovementManager.stopJob(jobTicket);

However, if I look the returned items, they are not sorted by the sort options I used, not even inside a batch. Is it possible to get the URIs sorted?

score 0 · Accepted Answer · answered May 02 '18 at 11:59

Answering myself: it is not possible. I created a ticket on marklogic github project (https://github.com/marklogic/java-client-api/issues/916) and the final conclusion was (https://github.com/marklogic/java-client-api/issues/916#issuecomment-385773027):

DMSDK splits the huge tasks into small chunks by directing the query to each forest and getting the URIs from each forest in batches. Even if we pass in the sort options, we would get an ordering per forest and not on the whole. Since we can't get a global ordering, we chose to drop the options and it doesn't make sense to have an ordering at the forest level.

How to have sorted results when using Data Movement API?

1 Answers1