3

I am struggling to understand how to create a pipeline which involves reading and manipulating a video/image file. Looking into the documentation, which is quite "essential" without examples and few comments, maybe the beam.io package should help, i.e. class LocalFileSystem

However I have no idea how to use it to create a working pipeline which reads and apply some transform (i.e. frame extraction with ffmpeg)

I am using using python, however if java is more documented i can switch.

Any example? any help? Thanks in advance

lordcenzin
  • 339
  • 3
  • 15
  • You probably need to write a subclass of `FileBasedSource` with `splitable=False` and implement your own `read_records`. https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/filebasedsource.py#L201 – greeness Jun 17 '18 at 13:04

1 Answers1

2

IMHO, you can specifying ffmpeg in the workers in order to use it for image/video processing. To upload the specified resources instead of default ones to the workers, use filesToStage pipeline option. To use this option, you should use Java SDK since it is not available in Python.

See this SO question for more details about use ffmpeg in pipeline and this question to have a overview about the process.

enle lin
  • 1,664
  • 8
  • 14