5

I'm new on Hadoop world, and I need install mesos with Hadoop HDFS to make a fault-tolerant distributed file system, but all installation references include necessary components for my scenario as for example: MapReduce.

Do you have any idea or references about this?

enrique-carbonell
  • 5,836
  • 3
  • 30
  • 44

1 Answers1

7

Absolutely possible. Don't think Hadoop as an installable program, it's just composed by a bunch of java processes running on different nodes inside a cluster.

If you use hadoop tar ball, you can just run NameNode and DataNodes processes if you only want HDFS.

If you use other hadoop distros (HDP for instance), I think HDFS and mapreduce come from different rpm packages, but it does harm to install both rpm packages. Again just run NameNode and DataNodes if you only need HDFS.

zhutoulala
  • 4,792
  • 2
  • 21
  • 34
  • ok, thks @zhutoulala...do you know some tutorial to make this? – enrique-carbonell Jan 21 '15 at 20:28
  • http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/ is one of the most popular ones – zhutoulala Jan 21 '15 at 20:48
  • 1
    I have one instance of postgres and I need that their data file it's store on HDFS, it's possible? I'm not sure because, when I was reading your link and other docs, all show that the way of read/write on HDFS it's using hadoop command. The read/write operation with HDFS not it's transparent to the user? – enrique-carbonell Jan 21 '15 at 21:57
  • well, this is a different question:) You can also use HDFS api or webhdfs web service to get/put data into HDFS. But if you ask is using HDFS for your database's storage, that's a totally different story – zhutoulala Jan 21 '15 at 23:11
  • ok, thanks @zhutoulala, yes I was suppose that it's possible the use of HDFS for my database storage, but now, after some time reading docs, I think that this idea it's wrong or not available at this moment. – enrique-carbonell Jan 22 '15 at 13:25
  • @kikicarbonell based on this article [link](https://www.enterprisedb.com/blog/hadoop-postgres-%E2%80%93-bridging-gap) it looks as if using FDW (Foreign Data Wrappers) it would be possible to store data in HDFS and access it via postgres. Not sure of the effectiveness / limitations of this method, but looks interesting. – Voltaire Aug 22 '17 at 10:56