AWS EMR - install HUE using Java SDK

Question

I am trying to set up a StepConfig to install and run HUE on my cluster. I am creating steps in a following way:

private StepConfig newInstallHueStep() {
    return new StepConfig()
            .withName("Install Hue")
            .withActionOnFailure(getDefaultActionOnFailure())
            .withHadoopJarStep(stepFactory.newScriptRunnerStep("s3://us-west-1.elasticmapreduce/libs/hue/install-hue"));
}

and

private StepConfig newRunHueStep() {
    return new StepConfig()
            .withName("Run Hue")
            .withActionOnFailure(getDefaultActionOnFailure())
            .withHadoopJarStep(stepFactory.newScriptRunnerStep("s3://us-west-1.elasticmapreduce/libs/hue/run-hue"));
}

The result is that step Install Hue is successful and Run Hue is failing.

enter image description here

The stderr log:

=== Uninstalling hbase
=== Saved registry at /var/lib/hue/app.reg
=== Saved /usr/lib/hue/build/env/lib/python2.6/site-packages/hue.pth
=== Uninstalling security
=== Saved registry at /var/lib/hue/app.reg
=== Saved /usr/lib/hue/build/env/lib/python2.6/site-packages/hue.pth
=== Uninstalling search
=== Saved registry at /var/lib/hue/app.reg
=== Saved /usr/lib/hue/build/env/lib/python2.6/site-packages/hue.pth
=== Uninstalling sqoop
=== Saved registry at /var/lib/hue/app.reg
=== Saved /usr/lib/hue/build/env/lib/python2.6/site-packages/hue.pth
=== Uninstalling zookeeper
=== Saved registry at /var/lib/hue/app.reg
=== Saved /usr/lib/hue/build/env/lib/python2.6/site-packages/hue.pth
=== Uninstalling rdbms
=== Saved registry at /var/lib/hue/app.reg
=== Saved /usr/lib/hue/build/env/lib/python2.6/site-packages/hue.pth
=== Uninstalling spark
=== Saved registry at /var/lib/hue/app.reg
=== Saved /usr/lib/hue/build/env/lib/python2.6/site-packages/hue.pth
=== Uninstalling impala
=== Saved registry at /var/lib/hue/app.reg
=== Saved /usr/lib/hue/build/env/lib/python2.6/site-packages/hue.pth
=== Saved /usr/lib/hue/build/env/lib/python2.6/site-packages/hue.pth
Running '/usr/lib/hue/build/env/bin/hue syncdb --noinput' with None
Running '/usr/lib/hue/build/env/bin/hue migrate --merge' with None
Traceback (most recent call last):
  File "/usr/lib/hue/build/env/bin/hue", line 9, in <module>
    load_entry_point('desktop==3.7.1-amzn-2', 'console_scripts', 'hue')()
  File "/usr/lib/hue/desktop/core/src/desktop/manage_entry.py", line 60, in entry
    execute_manager(settings)
  File "/usr/lib/hue/build/env/lib/python2.6/site-packages/Django-1.4.5-py2.6.egg/django/core/management/__init__.py", line 459, in execute_manager
    utility.execute()
  File "/usr/lib/hue/build/env/lib/python2.6/site-packages/Django-1.4.5-py2.6.egg/django/core/management/__init__.py", line 382, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/lib/hue/build/env/lib/python2.6/site-packages/Django-1.4.5-py2.6.egg/django/core/management/base.py", line 196, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/lib/hue/build/env/lib/python2.6/site-packages/Django-1.4.5-py2.6.egg/django/core/management/base.py", line 232, in execute
    output = self.handle(*args, **options)
  File "/usr/lib/hue/build/env/lib/python2.6/site-packages/Django-1.4.5-py2.6.egg/django/core/management/base.py", line 371, in handle
    return self.handle_noargs(**options)
  File "/usr/lib/hue/desktop/core/src/desktop/management/commands/install_all_examples.py", line 89, in handle_noargs
    raise exception
hadoop.fs.exceptions.WebHdfsException: SecurityException: Failed to obtain user group information: org.apache.hadoop.security.authorize.AuthorizationException: User: hue is not allowed to impersonate hue (error 403)
Command exiting with ret '255'

I've also tried with single step Run Hue then the stderr was:

Command exiting with ret '255'

Hue needs to be added as a proxy user, e.g. http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cdh_ig_cdh_hue_configure.html?scroll=topic_15_4#topic_15_4_1_unique_1 — Romain, May 29 '15 at 03:57

score 1 · Accepted Answer · answered May 29 '15 at 08:57

The right way of installing Hue with Java sdk is to post a Install Hue step as a bootstrap action and Run Hue as a job flow step.

RunJobFlowRequest runJobFlowRequest = new RunJobFlowRequest("Main", jobFlowInstancesConfig)
            .withName("SDK_filtering" + System.currentTimeMillis())
            .withAmiVersion("3.7.0")
            .withVisibleToAllUsers(true)
            .withServiceRole("EMR_DefaultRole")
            .withJobFlowRole("EMR_EC2_DefaultRole")
            .withBootstrapActions(
                    new BootstrapActionConfig("Install Hue",
                            new ScriptBootstrapActionConfig("s3://us-west-1.elasticmapreduce/libs/hue/install-hue", null)))
            .withSteps(
                    newEnableDebugStep(),
                    newInstallHiveStep(),
                    newInstallPig(),
                    newRunHueStep()
            );

AWS EMR - install HUE using Java SDK

1 Answers1