JsonPath not working properly in Spring Cloud Data Flow filter component

Question

I'm trying to write a simple SCDF flow that reads from Kafka, filters the messages by presence of specific value and pushes data into Mongo. As part of this i had to wrote following #jsonPath

 #jsonPath(payload,'$[?(@.metadata!=null)].metadata[?(@.trigger-routing!=
 null)].trigger-routing') == {'1'}

And i wrote an example test that will run the SPeL and verify what it returns(Note: I'm intetionally using @EnableIntegration to wire the same profile of SPeL functionality as SCDF, at least that's my theory)

@SpringBootTest(classes = SpelTst.TestConfiguration.class)
public class SpelTst {

    @Configuration
    @EnableIntegration
    public static class TestConfiguration {

    }

    @Autowired
    IntegrationEvaluationContextFactoryBean factory;

    @Test
    public void test() throws JsonProcessingException {
        final StandardEvaluationContext context = factory.getObject();
        ExpressionParser parser = new SpelExpressionParser();
        Expression exp = parser.parseExpression("#jsonPath(payload,'$[?(@.metadata!=null)].metadata[?(@.trigger-routing!= null)].trigger-routing') == {'1'}");

        final PixelHitMessage sampleOne = new PixelHitMessage()
                .setMetadata(ImmutableMap.of("trigger-routing", "1"))
                .toChildType();

        final PixelHitMessage sampleTwo = new PixelHitMessage()
                .setMetadata(ImmutableMap.of("trigger-routing", ""))
                .toChildType();

        final PixelHitMessage sampleThree = new PixelHitMessage()
                .setMetadata(Collections.emptyMap())
                .toChildType();

        final PixelHitMessage sampleFour = new PixelHitMessage()
                .toChildType();


        System.out.println(resolve(context, exp, sampleOne));
        System.out.println(resolve(context, exp, sampleTwo));
        System.out.println(resolve(context, exp, sampleThree));
        System.out.println(resolve(context, exp, sampleFour));
    }

    private static Object resolve(StandardEvaluationContext context, Expression exp, PixelHitMessage sampleOne) throws JsonProcessingException {
        final ObjectMapper mapper = new ObjectMapper();
        final String payload = mapper.writerFor(PixelHitMessage.class).writeValueAsString(sampleOne);
        System.out.println(payload);

        final Message<String> kafkaMessage = MessageBuilder.withPayload(payload).build();

        context.setRootObject(kafkaMessage);

        return exp.getValue(context, Object.class);
    }

}

When i run this i get a a following output

{"timestamp":"2020-06-26T19:31:38.013Z","level":"INFO","thread":"main","logger":"SpelTst","message":"Started SpelTst in 1.706 seconds (JVM running for 4.352)","context":"default"}

{"eventId":null,"postTime":null,"headers":null,"metadata":{"trigger-routing":"1"}}
true
{"eventId":null,"postTime":null,"headers":null,"metadata":{"trigger-routing":""}}
false
{"eventId":null,"postTime":null,"headers":null,"metadata":{}}
false
{"eventId":null,"postTime":null,"headers":null,"metadata":null}
false

The above is the exact behaviour that i'm seeking to achieve.

But when i use the same SPeL in a filter component in SCDF, i'm getting following exception

Caused by: com.jayway.jsonpath.PathNotFoundException: No results for path: $['metadata']['trigger-routing']

Example of a message that should return false

{"eventId":"acb0afce-7782-4dc6-af09-4d6878fa8fd3","postTime":1593201189799,"headers":{"accept":"*/*","host":"localhost:7071","user-agent":"insomnia/2020.2.2"},"metadata":{}}

Example of a message that should return true

{"eventId":"045698d4-d4dc-41b0-8bab-7c07ad58970a","postTime":1593201492866,"headers":{"accept":"*/*","host":"localhost:7071","user-agent":"insomnia/2020.2.2"},"metadata":{"trigger-routing":"1"}}

In SCDF, the SPeL is working only for the positive scenario, the absence of any of the data on the path causes the above exception. I was thinking about using the Option.DEFAULT_PATH_LEAF_TO_NULL for the JsonPath, but as far as i know there is no way to specify it trough Spring properties(i checked the code for the JsonPathUtils, and they are calling the version of JsonPath logic that uses default context without with default(empty) config.

I also verified whether the filter expression is correctly deployed(screen of the K8 configuration for the pod that is running the filter application), and it appears to be correct.

What version of `json-path` is being used?And also is there more stacktrace for the exception? — Kavithakaran Kanapathippillai, Jul 12 '20 at 10:22
Could you try to filter your JSON file with other tool, for instance, jq (https://stedolan.github.io/jq/). And compare? Also, are you sure you are testing both solution with same dataset? — davidmpaz, Jul 01 '22 at 05:23

Santanu · Answer 1 · 2022-08-22T07:20:55.913

This kind of solution can be used.

    <dependency>
        <groupId>com.jayway.jsonpath</groupId>
        <artifactId>json-path</artifactId>
        <version>2.7.0</version>
    </dependency>

Then the validation could be

    DocumentContext documentContext = JsonPath.parse(messagePayload);
    Map<String, Object> metadata = documentContext.read("$.metadata");

Imports are, for DocumentContext and JsonPath

import com.jayway.jsonpath.DocumentContext;
import com.jayway.jsonpath.JsonPath;

Now for the true scenario

System.out.println(metadata);

Output will be: {trigger-routing=1}

And for the false scenario

System.out.println(metadata);

Output will be: {}

Other JSON Path syntaxes are - https://support.smartbear.com/alertsite/docs/monitors/api/endpoint/jsonpath.html

JsonPath not working properly in Spring Cloud Data Flow filter component

1 Answers1