"Read multiple file from different location simultaneously and merge them into one payload"

Question

I am reading the multiple files from different folder and merging them into one but not able to merge into one file.

I am using composite source where I added two file connector then I am logging the payload into logger. payload I am getting one by one. How can I get the one payload combination of the two different payloads or multiple files input?

<flow name="file2Flow">
    <composite-source doc:name="Copy_of_Composite Source">
        <file:inbound-endpoint path="src/main/resources/input1" responseTimeout="10000" doc:name="File"/>
        <file:inbound-endpoint path="src/main/resources/input2" responseTimeout="10000" doc:name="File"/>
    </composite-source>
    <file:file-to-string-transformer doc:name="File to String"/>
    <logger message="#[payload]" level="INFO" doc:name="Logger"/>
</flow>

also I am trying this but not getting output

<flow name="file2file2Flow">
       <http:listener config-ref="HTTP_Listener_Configuration" path="/files" doc:name="HTTP"/>
       <scatter-gather doc:name="Scatter-Gather">
          <file:outbound-endpoint path="src/main/resources/input1" responseTimeout="10000" doc:name="File"/>
          <file:outbound-endpoint path="src/main/resources/input1" responseTimeout="10000" doc:name="File"/>
    </scatter-gather>
    <dw:transform-message doc:name="Transform Message">
         <dw:set-payload><![CDATA[%dw 1.0
       %output application/json
      ---
    {
       post1: payload[0],
       post2: payload[1]
       }]]>     
    </dw:set-payload>
        </dw:transform-message>
        <logger message="#[payload]" level="INFO" doc:name="Logger"/>
    </flow>

score 0 · Answer 1 · answered May 11 '19 at 13:31

file:inbound-endpoint will poll one directory, so if you need different directories that won't work.

composite-source allows it, but they wont be available in the same payload.

file:outbound-endpoint is for writing files only.

In Mule 3, you can achieve this though through a combination of a poll to trigger the flow, scatter-gather to route to multiple processors and the mule requester module to read files mid flow.

Mule Requester Module: https://www.mulesoft.com/exchange/68ef9520-24e9-4cf2-b2f5-620025690913/requester-module/

Rough example:

<flow name="dw-testFlow">
        <poll doc:name="Poll" frequency="10000">
            <logger level="INFO" doc:name="Logger" />
        </poll>
        <scatter-gather doc:name="Scatter-Gather">
            <mulerequester:request config-ref="muleRequesterConfig" resource="myFileEndpoint" doc:name="Mule Requester" />
            <mulerequester:request config-ref="muleRequesterConfig" resource="myFileEndpoint" doc:name="Mule Requester" />
        </scatter-gather>
    </flow>

"Read multiple file from different location simultaneously and merge them into one payload"

1 Answers1