I'm loading JSON documents using mlcp from a CSV into my db. I want to add a property to all those files and later be able to search the documents based on the property value. How can I do that using transformations? Using xdmp.documentAddProperties I'm able to apply it to manual documents using their URIs.
Similar query, how do I add another column during ingestion?
MLCP transforms don't have built-in features to help with that, but you can just use xdmp.documentAddProperties inside the MLCP transform. That should normally work just fine.
It is recommended however to put metadata inside the main document, mostly for performance reasons. A very common pattern we use for this, which is also used by the Data Hub Framework, and Entity Services, is to wrap the entire document in an envelope, and create dedicated 'sections' for provenance header info, canonical model instance, embedded triples, and attachments. Something like:
{
envelope: {
header: {},
triples: {},
instance: {},
attachments: {}
}
}
It is not a very rigid pattern, so you can bend it to your needs if you feel the need, but the above has become a very common pattern by now.
All in all, it might even be worth looking into using Data Hub Framework for your data ingestion, rather than using MLCP directly. You can find all you need about this DHF here:
https://marklogic.github.io/marklogic-data-hub/
HTH!
