is OpenMPI supported on distributed file systems like HDFS(Hadoop Distributed File System)?
Asked
Active
Viewed 547 times
1 Answers
2
Hadoop is not implemented using MPI because
- MPI is a message passing interface whereas MapReduce is a tool initially developed by Google for running Big Data applications.
- One of the main feature available in Hadoop/Mapreduce is fault tolerance. But it is not supported in MPI.
Refer
Why isn't Hadoop implemented using MPI?
MPI and MapReduce
Let we come to your question,
An article says that,
The prospect of running MPI jobs under YARN is an enticing proposition. As mentioned, the vast stores of data now living in HDFS can be accessed by non-MapReduce applications like those written in MPI. (Please note, however, that the intimate interface of MapReduce to HDFS is part of the MapReduce framework and is not automatic in other frameworks and must be managed by the user’s application.)
The prospect of running Open MPI under YARN has been investigated by Ralph H. Castain of the Open MPI team.
you can refer the below link for more information,
-
So does this mean we can access data present on HDFS by OpenMPI application? – rishi007bansod Jul 29 '15 at 15:36
-
Yes. we can access by MPI application. – Kumar Jul 29 '15 at 16:45