2

I am attempting to convert a bash script to a java program. Within this script I run the start scripts for Hadoop, Zookeeper, and Accumulo: 

/hadoop/bin/start_all.sh  
/zookeeper/bin/zkServer.sh start  
/accumulo/bin/start_all.sh

This is simple to do in a script. And if the programs are already running I can call these startup scripts again no issue and the programs will simply output that they are already running and their pids.

I'm trying to figure out if there is a way to do this within a java program. Is there some hidden command in the Hadoop/ZooKeeper/Accumulo API where I can run Class.run(configs) and it'll start or try to start Hadoop/ZooKeeper/Accumulo?

My next step is that I can probably use jsch to run ssh commands, but that seems like I'm not really leaving the bash script behind.

Edit: executing hadoop example jar files from java In this question the asker is executing the start commands using Runtime. Is this an appropriate way to start Hadoop? I'd rather use the native Hadoop API if there are commands to use there.

Community
  • 1
  • 1
S.Huston
  • 254
  • 6
  • 18
  • I can't help but wonder why... mind explaining? – Donald Miner Sep 16 '13 at 01:58
  • @DonaldMiner I'm writing a program to make starting these programs easy for the non-technical. – S.Huston Sep 17 '13 at 20:53
  • Cloudera manager/Ambari? Puppet? Something seems off here... nontechnical people probably shouldn't be starting and stoping the hadoop services. – Donald Miner Sep 18 '13 at 00:23
  • @DonaldMiner It's for a single-node instance enclosed demo more than anything. Eventually the management will probably be run by one of the services you mentioned. Thanks for the sanity check. :) – S.Huston Sep 18 '13 at 15:50

1 Answers1

1

There isn't any specific API to start the Hadoop services or Zookeeper services in my view. Take for an instance the class org.apache.hadoop.hdfs.server.namenode.NameNode

Though you can use the main() in the above class to start the service.

From the script, Hadoop uses something like the following to start the services calling the main() function of the different classes with arguments.

nohup $_JAVA_EXEC -Dproc_$COMMAND $JAVA_HEAP_MAX $HADOOP_OPTS -classpath "$CLASSPATH" $CLASS "$@" > "$_HADOOP_DAEMON_OUT" 2>&1 < /dev/null

Where, CLASS = org.apache.hadoop.hdfs.server.namenode.NameNode Others are self explanatory.

SSaikia_JtheRocker
  • 5,053
  • 1
  • 22
  • 41
  • Actually, I wanted a solution that worked from a Java Program, I already had code that runs in a script fine. This doesn't look like something you'd throw into a Java program... My apologies for withdrawing after accepting. I read your solution incorrectly. – S.Huston Sep 17 '13 at 20:50
  • Well, I don't mind but the thing is you don't have specific APIs as far as I know, as I have already mentioned. My intention was to show how Hadoop actually starts up the namenode. So, if Hadoop itself is uisng the above Class to start that, there may not be other ways. – SSaikia_JtheRocker Sep 18 '13 at 05:25
  • I also did mention of main() method, just high lighted it for you. From any other Java class you can all that, but the thing is you'll need the arguments. I might be wrong here. – SSaikia_JtheRocker Sep 18 '13 at 08:03
  • 1
    You're correct. I think I'm going to use RuntimeException to start these processes in my main(), using the typical terminal commands, as I can wait for the processes to finish. Thanks for the update. – S.Huston Sep 18 '13 at 15:48
  • 1
    Just added some more information. I might have not stated it properly that the main() function is inside org.apache.hadoop.hdfs.server.namenode.NameNode class, which Hadoop uses to start the namenode service passing the arguments. – SSaikia_JtheRocker Sep 18 '13 at 17:59