2

I'm using WSO2 ESB to process files: an inbound-endpoint with polling, detect a file and pass it to a sequence. That sequence read the file and process records (writing to database).

My question is about processing multiple files at once. For now I have this configuration:

<?xml version="1.0" encoding="UTF-8"?>
<inboundEndpoint name="AssuresContratsGtIEP" onError="TechnicalExceptionSEQ" protocol="file" sequence="AssuresContratsGtSEQ" suspend="false" xmlns="http://ws.apache.org/ns/synapse">
    <parameters>
        <parameter name="interval">5000</parameter>
        <parameter name="sequential">true</parameter>
        <parameter name="coordination">true</parameter>
        <parameter name="transport.vfs.FileProcessCount">2</parameter>
        <parameter name="transport.vfs.ContentType">application/binary;charset=UTF-8</parameter>
        <parameter name="transport.vfs.LockReleaseSameNode">false</parameter>
        <parameter name="transport.vfs.AutoLockRelease">false</parameter>
        <parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
        <parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
        <parameter name="transport.vfs.FileURI">/home/istya/Flux/REF_PERS/Input/</parameter>
        <parameter name="transport.vfs.MoveAfterFailure">/home/istya/Flux/REF_PERS/Failure/</parameter>
        <parameter name="transport.vfs.DistributedLock">false</parameter>
        <parameter name="transport.vfs.FileNamePattern">.*ASSURES_CONTRATS_GT.*.csv</parameter>
        <parameter name="transport.vfs.MoveAfterProcess">/home/istya/Flux/REF_PERS/Process/</parameter>
        <parameter name="transport.vfs.Locking">disable</parameter>
        <parameter name="transport.vfs.FileSortAttribute">none</parameter>
        <parameter name="transport.vfs.FileSortAscending">true</parameter>
        <parameter name="transport.vfs.CreateFolder">true</parameter>
        <parameter name="transport.vfs.Streaming">true</parameter>
        <parameter name="transport.vfs.Build">false</parameter>
    </parameters>
</inboundEndpoint>

When I change the sequential parameter to "false", the program crashes: first mediator on the sequence says that the file does not exist (although the file is still in the polling folder...).

Does anyone knows how to process multiple files with the same inbound and sequences in WSO2 ESB ? Is it possible or do we need to install a cluster of ESB ?

Thanks for your support ! Julien

Julien GRESSE
  • 354
  • 1
  • 13
  • Did you find a solution to this? I'm facing the same issue – Pavan Andhukuri Oct 22 '20 at 04:57
  • Hi Pavan, no I don't have an explanation for this bug. I was able to solve this by using the Message Broker and create a list of files to process, sequentially though... – Julien GRESSE Oct 23 '20 at 11:50
  • What did you end up doing? As a work around? – Pavan Andhukuri Oct 23 '20 at 12:06
  • We used WSO2 Message Broker: for every file to process, we send a message to the broker. We developed a JMS client that checks broker's queue every minute to see if there is a message (a file to process). That way we can create a queue of files to process. But they will process sequentially. But if you have multiple WSO2 ESB you can process multiple files at once. – Julien GRESSE Oct 24 '20 at 16:49

0 Answers0