Can Flink output be sinked to a NFS or GPFS file system?

Question

Flink has a RollingFileSink which per the documentation works with HDFS (Hadoop). Are there similar classes which can be used to sink data to file systems such as NFS, ext4 or GPFS?

https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/connectors/filesystem_sink.html

Do you use flink distributed / multiple machines? If you are, using a non distributed filesystem might not be your best option. — Richard Deurwaarder, Apr 02 '18 at 08:49

score 2 · Accepted Answer · answered Apr 02 '18 at 13:34

2

The RollingSink class works with all Hadoop-compatible file systems (it simply uses Hadoop's FileSystem abstraction).

The list of compatible file systems includes POSIX-compliant file systems (file://, which covers NFS, ext4, ...) and GPFS as well.

answered Apr 02 '18 at 13:34

Stephan Ewen

2,311
1
17
14

Can Flink output be sinked to a NFS or GPFS file system?

1 Answers1