0

I am trying to develop Chef cookbooks to Deploy and Manage Hadoop Cluster (specifically MapR).

As per my understanding Chef is a good tool to install hadoop cluster in an automated way. But I am not sure how Chef can help to manage/change configuration of Hadoop cluster after installation and can build cluster to the state where cluster goes down completely (in case of Disaster Recovery).

Pradeep Bhadani
  • 4,435
  • 6
  • 29
  • 48
  • Is the [hadoop cookbook](https://github.com/continuuity/hadoop_cookbook) not corresponding to your needs ? – Tensibai Sep 08 '14 at 11:54
  • hadoop cookbook is for Apache hadoop flavor. i am working on MapR hadoop – Pradeep Bhadani Sep 08 '14 at 14:55
  • 1
    And this one https://github.com/boorad/mapr_cookbooks ot this one https://github.com/vmware-serengeti/serengeti-pantry/tree/master/cookbooks/mapr ? Googling around this will give you some insight on how to make your own I think. – Tensibai Sep 08 '14 at 14:56
  • i have seen these cookbook.....it seems that these cookbook are meant for installing hadoop...not managing lifecycle of hadoop.. – Pradeep Bhadani Sep 08 '14 at 16:15
  • https://github.com/vmware-serengeti/serengeti-pantry <= if you read carrefully, the whole configuration will be managed by chef. So any change would be done by updating attributes in the cookbook and running chef again. More precisely read [HERE](http://www.projectserengeti.org/) for the whole backend or thoose cookbooks – Tensibai Sep 08 '14 at 16:19
  • thanks for pointing out...but i dont see it can create mapr volumes and manage them or keep track of mapr volumes...my question is - is it feasible to maintain these kind kind of operation with Chef – Pradeep Bhadani Sep 08 '14 at 16:25
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/60834/discussion-between-tensibai-and-user1158351). – Tensibai Sep 08 '14 at 16:30

1 Answers1

0

From my experience with Hadoop , Chef/Puppet/Ansible or any other configuration management tool is good for installing/deploying Hadoop Distribution (Cloudera, HDP or MapR) but it is best to manage Hadoop configuration using their CM tools (like Cloudera Manager for CDH distro , Ambari for HDP and MapR CM for Mapr)

Pradeep Bhadani
  • 4,435
  • 6
  • 29
  • 48