0

I have a secured hadoop cluster and I want to run hive action using oozie rest api.I was able to run Hive Action using workflow.xml via Oozie rest api.But if I try to run the same hive action without using workflow.xml (passing configuration in payload) it fails due to credentials.Is there any way using which I can pass this credentials in payload of Oozie rest api.

Sachin Janani
  • 1,310
  • 1
  • 17
  • 33

1 Answers1

0

Step 1: upload the workflow.xml (or whatever.0F253C.xml) script, including parameters in the credentials definition, to HDFS using either WebHDFS or HttpFS REST API

    <credentials>
        <credential name='HiveMetastore' type='hcat'>
            <property>
              <name>hcat.metastore.uri</name>
              <value>${hiveMetastoreURI}</value>
            </property>
            <property>
              <name>hcat.metastore.principal</name>
              <value>${hivePrincipal}</value>
            </property>
        </credential>
  </credentials>

...

<action name="DummyHive" cred="HiveMetastore">
    <hive xmlns="uri:oozie:hive-action:0.3">

...

Step 2: submit a workflow with that XML script plus actual values for parameters

oozie.wf.application.path=/user/johndoe/hic_sunt_leones/whatever.0F253C.xml
hiveMetastoreURI=thrift://dummy.nsa.gov:9083
hivePrincipal=hive/_HOST@HADOOP.NSA.GOV

...

Samson Scharfrichter
  • 8,884
  • 1
  • 17
  • 36
  • Thanks @Samson for the reply.But my question was how to submit credentials in payload without workflow.xml.As there are two ways to execute hive action using oozie rest api 1) Using workflow.xml 2) By passing all configurations as a part of payload.So in second method how to pass credentials (metastore principal) of hive metastore – Sachin Janani Sep 15 '15 at 12:08
  • Sorry, I don't know anything about "passing all config as part of the payload". Can you point me to the documentation? – Samson Scharfrichter Sep 15 '15 at 14:46
  • Please follow this guide first 2 lines in this section:https://oozie.apache.org/docs/4.0.0/WebServicesAPI.html#Proxy_Hive_Job_Submission – Sachin Janani Sep 15 '15 at 14:49
  • OK, you mean the "Proxy Hive Job Submission" feature - sounds like an ugly shortcut with limited functionality. Never used these "proxy" things. Sorry :-/ – Samson Scharfrichter Sep 15 '15 at 14:52
  • No problem.Thanks for your interest.But there can be valid scenario where remote user need to pass the workflow configuration as payload – Sachin Janani Sep 15 '15 at 14:55
  • Did you check how the Hue GUI submits an Oozie workflow, and whether it involves a temp HDFS file for the XML - or not? I'm afraid that you can't be smarter than these guys... – Samson Scharfrichter Sep 15 '15 at 18:53
  • Good point samson I have not checked hue but I have viewed source code for the oozie which handles this credentials part, it seems that something is missing there,but let me check the hue as well. – Sachin Janani Sep 16 '15 at 03:58