I'm trying to create a Job which is suppose to :
- Get me the maxID in a MongoDB collection
- Give me all documents from another collection verifying id < maxID
In the first transformation I retrieve the maxID and set the variable SetMaxTransformation
In the second one I use a "Get Variable" Step which I link to a write log to make sure the maxID is read correctly in the new transformation.
Additionally I place my MongoDB Input step connected to my database and use the following aggregation query: [{$match :{'Param.button_state' : "+1", '_id':{ $lt: "${maxID}"} }}, {$sort : {_id :1} }, {$limit : 10 }]
Mongo query
Here is the transformation structure :
enter image description here
Here is the problem, the variable is read by the "Get variable" but not by the "MongoDB Input Step" since there's no output from the previously mentioned step following the Job execution. When I hard code the value of maxID in the aggregation query, the output is not null
Here is the job structure : Job
Thanks for your time, Babacar