How to apply python on BatchStage Hazelcast Jet

Question

What I am trying to do is, I am fetching the data from different source and doing grouping and join on them and after that I am getting result in BatchStage and now I have to perform python code on this function but I am not able to do it.

Here is my below code:

BatchStage<List<Map<String, Object>>> bd = AggregateData.aggregate(data, Object);
bd1 = bd1.filter(k -> {
                        // some filters
                    });
// Now here after this I want to execute python code:
bd1.apply(mapUsingPython(new PythonServiceConfig()
                                     .setBaseDir("D:/")
                                     .setHandlerModule("take_sqrt")))
                             .setLocalParallelism(1)
                             .writeTo(Sinks.logger());

But it is now allowing as I am new to this I am not getting exact syntax, please help me out

score 3 · Answer 1 · answered Jul 23 '21 at 08:46

3

mapUsingPython transforms BatchStage<String> to BatchStage<String>, so you first have to explicitly convert your List<Map<String, Object>> to string representation.

answered Jul 23 '21 at 08:46

Marko Topolnik

195,646
29
319
436

Thank you for the response, but is there is any other way to send List directly to it? – user3458271 Jul 23 '21 at 09:29

How to apply python on BatchStage Hazelcast Jet

1 Answers1