0

[Python - Beam SDK]

I would like to be able to test timing issues in integration tests, so I want to build a generator system that pipes in messages into my Beam application with timestamps I specify. My current idea is to an application write to ports (one for each stream I want to listen to), and then have the Beam application listen to these ports (the same way it would listen to a Kafka topic).

Does Beam already have the ability to listen to a port as a source? And if not, what would be the best way to make this?

Akshay Balwally
  • 1

1 Answers1

0

My experience with the [Java SDK]

In the way i see it a SocketIIO.read() implementation should just emit String (or byte[]) back. So creating a PTransform that open the port and then on the expand() method create a loop to just read the String (or byte[]) and then emit the values into the pipeline (returning a PColletion).

Hope it gives you an idea.

jcgarciam
  • 392
  • 5
  • 12