Does Apache Apex rely on HDFS or does it have its own file system?

Question

I understand that Apache Apex runs on Hadoop and YARN. Does it utilize HDFS for persistence and replication to protect against data loss? or does it have its own?

score 2 · Answer 1 · answered Apr 13 '16 at 17:40

Apache Apex uses checkpointing of operator state for fault tolerance. Apex uses HDFS to write these checkpoints for recovery. However, the store for checkpointing is configurable. Apex also has an implementation to checkpoint to Apache Geode. Apex also uses HDFS to upload artifacts such application package containing the application jar, its dependencies and configurations etc that are needed to launch the application.

score 1 · Accepted Answer · answered Apr 12 '16 at 08:20

1

Apache Apex does not have it's own file system. Streaming applications written with Apex will, by default, use HDFS for checkpointing, persistence and for saving application specific data.

answered Apr 12 '16 at 08:20

PradeepKumbhar

3,361
1
18
31

Does Apache Apex rely on HDFS or does it have its own file system?

2 Answers2