2

I am trying to run TensorFlow Transform, using Python, Apache Flink as the Beam Runner. I noticed that Beam does not have AWS S3 as the io connector, and would like to know any work around for this.

Here is the list of supported io connectors, but Python+S3 is not even in the roadmap.

I can think of two work around:

  1. mount the S3 bucket as a local drive to a EC2 instance
  2. Write my own Python S3 connector using their guide.

I want to know if there are other creative (easy) way out.

Thanks!

Happy Gene
  • 502
  • 3
  • 7

0 Answers0