0

I have a large ND-JSON file in SFTP (~20K lines). Is there a way to generate sub files out of this (~500 lines each) and place in another folder in SFTP?

Does Mule 4 has the capability to split a large file and write in SFTP? Or is there a need for a Java component?

Please advise.

Triumph Spitfire
  • 663
  • 15
  • 38

1 Answers1

2

If the input file is parsed as NDJSON, you can use the DataWeave function divideBy() to separate the array read from the file into subarrays of n elements.

Example:

%dw 2.0
output application/java
import * from dw::core::Arrays
---
payload divideBy 500

Then you should be able to use a to process each segment and output an NDJSON file inside.

aled
  • 21,330
  • 3
  • 27
  • 34