0

I have one directory having multiple batch files and each batch file have several documents. This directory will be updated continuously with new batches.

How we can schedule a job which will execute and load batch data on daily basis ? Is Data Movement Software Development Kit provide configuration to schedule daily jobs or just start and stop jobs ?

Thanks Girish

grtjn
  • 20,254
  • 1
  • 24
  • 35
Girish Kumar Agrawal
  • 1
  • 1

1 Answers1

1

Provided the directory is accessible from MarkLogic, MarkLogic could pull the batches in with a Scheduled Task. This section in the Admin Guide tells more about scheduling:

http://docs.marklogic.com/guide/admin/scheduling_tasks

On the other hand, the writing to, and reading from disk sounds unnecessary. Why doesn't the process that writes the batches push the information directly into MarkLogic? DMSDK can help with that, if your source system is written in Java. DMSDK does not help with configuring Scheduled Tasks, but it can help a lot with batch updates to transform and push information into MarkLogic. More about it can be found in the Java Guide:

http://docs.marklogic.com/guide/java/data-movement

HTH!

grtjn
  • 20,254
  • 1
  • 24
  • 35
  • Thnaks! The process to write batch is not in java so we can't directly push into Marklogic. As per documentation DMSDK is combination of MLCP + CROB2 but i am not able to find example to use MLCP features in DMSDK. Could you please provide example to push one csv file with DMSDK. I want to utilize all options of mlcp. Thanks for your support. – Girish Kumar Agrawal Aug 02 '18 at 04:25
  • I just want to execute input and harmonize flow of DHF with DMSDK. – Girish Kumar Agrawal Aug 02 '18 at 05:43
  • DMSDK does not use MLCP nor CORB2. Instead it makes use of REST transforms. To split and ingest a CSV using DMSDK, you'd have to split the CSV yourself, and then ingest the parts. – grtjn Aug 09 '18 at 09:30
  • I don't think you can launch DHF input and harmonize flows using DMSDK. DHF might use DMSDK under its covers to execute some tasks, but not sure. There are examples of running DHF flows from Gradle, not sure if that would be of help? – grtjn Aug 09 '18 at 09:32
  • Consider posting a new, and rephrased question on stackoverflow. You are talking about scheduling, which sounds like MarkLogic Scheduled Tasks, rather than DHF Jobs. – grtjn Aug 09 '18 at 09:33
  • Thanks for your support ! – Girish Kumar Agrawal Aug 10 '18 at 11:14