0

I'm trying to build a project using an Apache Flink Stateful Function in Python, but I can't seem to get it to work. What I've narrowed the issue down to is that it seems when I send the request to my stateful function through my protobuf schema, the serializer is unable to serialize my message into the class I'm expecting. Here's what I'm trying to do:

import json
from statefun import StatefulFunctions, RequestReplyHandler
from jobs.session_event_pb2 import Event

functions = StatefulFunctions()


@functions.bind("namespace/funcname")
def funcname(context, session: Event):
    print("hello world")


handler = RequestReplyHandler(functions)

if __name__ == '__main__':
    inputFile = open("my_file.json", "r")
    for line in inputFile:
        data = json.loads(line).get('properties')
        if data is not None and data.get('prop1') is not None and data.get('prop2') is not None:
            request = Event()
            request.prop1 = data["prop1"]
            request.prop2 = data["prop2"]
            request = request.SerializeToString()
            handler(request)

Here's my Protobuf schema:

syntax = "proto3";

package mypackage;

message Event {
    string prop1 = 1;
    string prop2 = 2;
}

What am I doing wrong here?

Ertai87
  • 1,156
  • 1
  • 15
  • 26

1 Answers1

0

That's because the RequestReply handler does not take direct protobuf messages. The Flink runtime sends a type called ToFunction and receives a response of type FromFunction. This payload contains your caller messages along with persisted values and other meta information.

If you can't to invoke the functions directly, such as in a test, I would encourage you to do that and not use the handler at all.

Seth
  • 345
  • 1
  • 4
  • Thanks! So then why, in their sample starter code ( https://ci.apache.org/projects/flink/flink-statefun-docs-master/getting-started/python_walkthrough.html ) can they send the HTTP request body from Flask directly into the function? What is different about an HTTP request body? – Ertai87 Aug 18 '20 at 18:10
  • You can call it manually if you use the correct type, "ToFunction" and "FromFunction" which is what is coming through flask https://github.com/apache/flink-statefun/blob/56ef7e5bec2c20e11c518d406a0b4c2290c50c64/statefun-python-sdk/statefun/request_reply.py#L28-L29 – Seth Aug 19 '20 at 16:07
  • So the Flask framework uses those ToFunction and FromFunction classes by default? – Ertai87 Aug 19 '20 at 16:10
  • The RequestReplyHandler does – Seth Sep 04 '20 at 13:20