1

I have a xml file to ingest in MarkLogic database where a new XML field has to be added . And the requirement is to add that XML field only during the mlcp import. Is this possible in MarkLogic using xquery?

XML file now -

<name>rashmita</name>
<employeeType>regular</employeeType>

XML to be changed -

<name>rashmita</name>
<employeeType>regular</employeeType>
<role>developer</role>
Mads Hansen
  • 63,927
  • 12
  • 112
  • 147
Rashmita Purkayastha
  • 457
  • 1
  • 8

1 Answers1

1

Yes, it is possible to modify the payload on ingest with MLCP.

Transforming Content During Ingestion

You can create an XQuery or Server-Side JavaScript function and install it on MarkLogic Server to transform and enrich content before inserting it into the database. Your function runs on MarkLogic Server.

Function Signature

A custom transformation is an XQuery function module that conforms to the following interface. Your function receives a single input document, described by $content, and can generate zero, one, or many output documents.

declare function yourNamespace:transform(
  $content as map:map,
  $context as map:map)
as map:map*

So, for your example (assuming that the actual docs are well-formed XML) could look something like:

module namespace example = "http://marklogic.com/example";
declare function example:transform(
  $content as map:map,
  $context as map:map
) as map:map*
{
  let $doc := map:get($content, "value")
  let $root := $doc/*
  let $_ :=
    if ($root)
    then 
      map:put($content, "value", 
        document { element {$root/name()} {$root/@*, $root/*, <role>developer</role>} })
    else ()
  return $content
};

Using a Custom Transformation

Once you install a custom transformation function on MarkLogic Server, you can apply it to your mlcp import or copy job using the following options:

  • transform_module - The path to the module containing your transformation. Required.
  • transform_namespace - The namespace of your transformation function. If omitted, no namespace is assumed.

An example invocation setting those parameters:

mlcp.sh import -mode local -host mlhost -port 8000 -username user -password password -input_file_path /space/mlcp-test/data -transform_module /example/mlcp-transform.xqy -transform_namespace "http://marklogic.com/example"

Mads Hansen
  • 63,927
  • 12
  • 112
  • 147
  • I have used this. It gives me this error . Any idea on why this gives me this endtag open error ? Is it because of the xml ? The error I gave below in the comment. – Rashmita Purkayastha Jan 11 '23 at 09:38
  • 23/01/11 09:35:52 ERROR contentpump.TransformWriter: Batch 83965077.0: Document failed permanently: /example/xml/employee.xml 23/01/11 09:35:52 WARN contentpump.TransformWriter: XDMP-UNEXPECTED: (err:XPST0003) Unexpected token syntax error, unexpected $end, expecting EndTagOpen_ – Rashmita Purkayastha Jan 11 '23 at 09:38
  • my example was incomplete. You would also need to declare the `example` namespace for that transform module. You can use any namespace and namespace-prefix, just need to be consistent in how it's declared and referenced (and the transform module has to have it declared - sorry about that!) – Mads Hansen Jan 11 '23 at 12:57
  • Not sure if you mean by this - xquery version "1.0-ml"; xdmp:document-load("http://marklogic.com/example", /example/xquery/addelement.xq none {xdmp:default-permissions()} ) – Rashmita Purkayastha Jan 11 '23 at 13:57
  • I have added this xquery beforehand – Rashmita Purkayastha Jan 11 '23 at 13:58
  • And using this for mlcp - import -host localhost -port 8000 -username admin -password ***** -input_file_path /example/xml -transform_module /example/xquery/addelement.xq -transform_namespace "http://marklogic.com/example" – Rashmita Purkayastha Jan 11 '23 at 14:31