Stack Overflow
Stack Overflow

Questions tagged [oozie]

Oozie is a workflow/coordination system to manage Hadoop Map Reduce jobs

Oozie is a workflow scheduler system to manage Apache Hadoop jobs.

Oozie Workflow jobs are Directed Acyclical Graphs (DAGs) of actions.

Oozie Coordinator jobs are recurrent Oozie Workflow jobs triggered by time (frequency) and data availabilty.

Oozie is integrated with the rest of the Hadoop stack supporting several types of Hadoop jobs out of the box (such as Java map-reduce, Streaming map-reduce, Pig, Hive, Sqoop and Distcp) as well as system specific jobs (such as Java programs and shell scripts).

Oozie is a scalable, reliable and extensible system.

References

Related Tags

1929 questions
9
votes
4 answers

Stop Oozie workflow execution

Yesterday I kicked off an oozie workflow. It started two jobs that stalled all day. I killed them this morning, having made a change that I now want to test. After killing the two jobs it's like the workflow became unstuck and is now proceeding. …
digitaljoel
  • 26,265
  • 15
  • 89
  • 115
9
votes
1 answer

How oozie handle dependencies?

I have several questions about oozie 2.3 share libraries: Currently, I defined the share libraries in our coordinator.properties: oozie.use.system.libpath=true oozie.libpath= Here are my questions: When share libraries are copied to…
Terminal User
  • 873
  • 3
  • 13
  • 21
8
votes
4 answers

Hive Internal Error: java.lang.ClassNotFoundException(org.apache.atlas.hive.hook.HiveHook)

I am running a hive query throwh oozie using hue.. I am creating a table through hue-oozie work flow... My job is failing but when I check in hive the table is created. Log shows below error: 16157 [main] INFO …
Amaresh
  • 3,231
  • 7
  • 37
  • 60
8
votes
2 answers

Oozie command to rerun workflow with skip nodes not working

There is a couple of options while re-running a workflow via Oozie command line. oozie.wf.rerun.failnodes oozie.wf.rerun.skip.nodes Option 1 works fine, however, while re-running workflow with option 2, it throws error E0404. oozie job -oozie…
Vishal Joshi
  • 161
  • 1
  • 2
  • 6
8
votes
2 answers

Oozie shell action not running as submitting user

I've written an Oozie workflow that runs a BASH shell script to do some hive queries and perform some actions on the results. The script runs but throws a permission error when accessing some of the HDFS data. The user that submitted the Oozie…
Blake
  • 83
  • 6
8
votes
4 answers

Oozie SSH Action

Oozie SSH Action Issue: Issue: We are trying to run few commands on a particular host machine of our cluster. We chose SSH Action for the same. We have been facing this SSH issue for some time now. What might be the real issue here? Please point me…
Kasa
  • 81
  • 2
  • 3
7
votes
2 answers

Oozie fs:exists with variables

I'm struggeling on the following problem using variables in an Oozie workflow definition checking if a specific file was created. It is working with absolute path like the following, but I cannot use an absolute…
Gerd
  • 777
  • 1
  • 7
  • 12
7
votes
1 answer

Oozie shell action: exec and file tags

I'm a newbie in Oozie and I've read some Oozie shell action examples but this got me confused about certain things. There are examples I've seen where there is no tag. Some example, like in Cloudera here, repeats the shell script in file…
oikonomiyaki
  • 7,691
  • 15
  • 62
  • 101
7
votes
3 answers

Error: E0505 : E0505: App definition

Getting error when executing below command. oozie job -oozie http://localhost:11000/oozie -config coordinator.properties -run Error: E0505 : E0505: App definition [hdfs://localhost:8020/tmp/oozie-app/coordinator/] does not exist any suggestions.
Mohammed Niaz
  • 386
  • 1
  • 5
  • 17
7
votes
1 answer

where does oozie stores the captured output values of the Java action (or) any action

I am using the capture-output option for my Java Action. The values I am using in the downstream actions. Which is working fine. When I do -rerun of the oozie job also framework is fetching the values, without running the Java action again. I would…
user3700301
  • 73
  • 1
  • 5
7
votes
2 answers

Error on running multiple Workflow in OOZIE-4.1.0

I installed oozie 4.1.0 on a Linux machine by following the steps at http://gauravkohli.com/2014/08/26/apache-oozie-installation-on-hadoop-2-4-1/ hadoop version - 2.6.0 maven - 3.0.4 pig - 0.12.0 Cluster Setup - MASTER NODE runnig - Namenode,…
ǨÅVËĔŊ RĀǞĴĄŅ
  • 530
  • 6
  • 30
7
votes
2 answers

Oozie stuck in PREP state

I have a problem with running oozie coordinator jobs. Below is my job properties and xml file codes. While submitting the job there was no errors, but the job is always in 'PREP' state. I have to manually kill the job after some extent of waiting.…
Kalai
  • 319
  • 1
  • 4
  • 21
7
votes
4 answers

How can I reload oozie job configuration file without restart oozie job

I'd like to know if there is a way to reload the configuration file of the oozie job without restart the oozie job ( coordinator ). Because the coordinator actually runs many our tasks, maybe sometimes we only need change one line of the job…
lixinso
  • 713
  • 3
  • 11
  • 15
7
votes
1 answer

What is significance of the Oozie MR launcher?

I created a simple Oozie work flow with Sqoop, Hive and Pig actions. For each of there actions, Oozie launches a MR launcher and which in turn launches the action (Sqoop/Hive/Pig). So, there are a total of 6 MR jobs for 3 actions in the work…
Praveen Sripati
  • 32,799
  • 16
  • 80
  • 117
7
votes
3 answers

How can I check Oozie logs

My coordinator failed with Error : E0301 invalid resource [filename] when I do hadoop fs -ls [filename] the file is listed. how can I debug what is wrong. how can I check log files??? oozie job -log requires jobId. in my case i dont have job id.…
user2694419
  • 81
  • 1
  • 1
  • 4
1
2
3
…
99 100