0

We are currently working on application running over Mule CE run-time. Reading a file from shared mount folder is fine on local machine. But in production cluster environment, we have 2 nodes each polling the same location. There is a possibility where both nodes pick the same file at same time.

I know Mule EE run-time handles this situation brilliantly, but we are not in a state to upgrade from CE to EE for now. That's why I need some help around it.

I tried using Quartz, but according to our analysis, in CE run-time Quartz scheduler can cause the same problem. Scheduler can advise both nodes to pick the same file at same time.

Then I tried implementing Custom file message receiver, please review the setting below:

Listener:

<file:inbound-endpoint path="${file_to_mq_wholefile.path}" 
        moveToPattern="#[message.inboundProperties['originalFilename']+'_'+message.rootId+'_'+org.mule.util.DateUtils.getTimeStamp('yyyy_MM_dd-HH_mm_ss.SSSSSS')].backup" 
        moveToDirectory="${file_to_mq_wholefile.move.to.directory}" 
        responseTimeout="10000" doc:name="File" 
        connector-ref="FileInboundGeneralConnector" 
        transformer-refs="RemoveArchiveInboundFileTransformer">
    </file:inbound-endpoint>

Mule Configuration:

<file:connector name="FileInboundGeneralConnector" autoDelete="true" streaming="false" validateConnections="false" doc:name="File"> 
    <service-overrides messageReceiver="com.company.adaptors.components.CustomFileMessageReceiver"/>  
</file:connector>

JavaCode:

public class CustomFileMessageReceiver extends FileMessageReceiver {

public CustomFileMessageReceiver(Connector connector, FlowConstruct flowConstruct, InboundEndpoint endpoint, String readDir, String moveDir, String moveToPattern, long frequency) throws CreateException {
    super(connector, flowConstruct, endpoint, readDir, moveDir, moveToPattern, frequency);
}

protected boolean attemptFileLock(File sourceFile) throws MuleException {
    System.out.println("************");
    return true;
}

}

  1. I am not sure how to proceed ahead from here.
  2. Please can anybody suggest any other alternatives for this solution

I have review other stackoverflow links (ex: Using Quartz with Mule in Clustered Environment) but, nothing reached to a final solution...

Sambit Swain
  • 131
  • 1
  • 13

2 Answers2

0

There are 2 options I can think of:

  1. You can check this solution and see if it works for your use case: https://github.com/arbuzworks/fulcrum-cluster-mule
  2. Implement custom file locking solution by using a common storage like Redis for all nodes.
Emin Can Sümer
  • 523
  • 3
  • 14
  • Hi @Emin, thanks for your suggestion. We tried this approach but couldn't succeed 100%. At some point it failed again. We are now creating the flow with initial state = STOPPED, and using internal hazelcast app through a Java API. This api with help of hz grabs each node instance in a loop and changes the flow initial state from STOPPED to ACTIVE – Sambit Swain Aug 16 '20 at 17:02
0

In mule4 connectors, we have an option to use a lock. It ensures that no other process can read the file one process started processing it. More information can be found here.

https://docs.mulesoft.com/file-connector/1.3/file-read

Tushar Koley
  • 164
  • 3