2

I have a few System.out.println() statements in my Java code that I am trying to view in the job scheduled using Oozie. I'm unable to find those println() statements despite the Job status showing as SUCCESSFUL.

My workflow.xml looks like:

<workflow-app xmlns='uri:oozie:workflow:0.5' name='CMDW-Security-User-Management-Spark'>

    <start to='spark-node' />

    <action name='spark-node'>
        <spark xmlns="uri:oozie:spark-action:0.1">
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <master>${master}</master>
            <name>HDFS-Test</name>
            <class>Test</class>
            <jar>${nameNode}/user/clsadmin/HDFS-Test/HDFS-Test.jar</jar>
            <configuration> 
                <property> 
                    <name>mapreduce.cluster.acls.enabled</name> 
                    <value>true</value> 
                </property> 
                <property> 
                    <name>mapreduce.job.acl-view-job</name> 
                    <value>*</value> 
                </property> 
            </configuration>
        </spark>
        <ok to="end" />
        <error to="fail" />
    </action>

    <kill name="fail">
        <message>Workflow failed, error message:
            ${wf:errorMessage(wf:lastErrorNode())}
        </message>
    </kill>
    <end name="end" />
</workflow-app>

jobTracker, nameNode and master are all defined in my job.properties.

Can someone help me with getting the println statements to be viewed inside the job log in Oozie web console?

EDIT 1:

Here's the Job Log after you posted that comment. Also updated the above workflow.xml:

2020-01-21 10:22:55,000  INFO JPAService:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[] No results found
2020-01-21 10:22:55,036  INFO ActionStartXCommand:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@:start:] Start action [0000028-200103135925853-oozie-oozi-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2020-01-21 10:22:55,052  INFO StartActionExecutor:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@:start:] Starting action
2020-01-21 10:22:55,075  INFO ActionStartXCommand:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@:start:] [***0000028-200103135925853-oozie-oozi-W@:start:***]Action status=DONE
2020-01-21 10:22:55,076  INFO ActionStartXCommand:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@:start:] [***0000028-200103135925853-oozie-oozi-W@:start:***]Action updated in DB!
2020-01-21 10:22:55,098  INFO StartActionExecutor:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@:start:] Action ended with external status [OK]
2020-01-21 10:22:55,128  INFO JPAService:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@:start:] No results found
2020-01-21 10:22:55,145  INFO WorkflowNotificationXCommand:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@:start:] No Notification URL is defined. Therefore nothing to notify for job 0000028-200103135925853-oozie-oozi-W@:start:
2020-01-21 10:22:55,145  INFO WorkflowNotificationXCommand:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000028-200103135925853-oozie-oozi-W
2020-01-21 10:22:55,182  INFO ActionStartXCommand:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] Start action [0000028-200103135925853-oozie-oozi-W@spark-node] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2020-01-21 10:22:55,188  INFO SparkActionExecutor:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] Starting action. Getting Action File System
2020-01-21 10:22:55,212  INFO SparkActionExecutor:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] Trying to load ShareLib exclude pattern for action.
2020-01-21 10:22:55,214  INFO ShareLibExcluder:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] The following sharelib exclude pattern will be used: oozie/jackson.*
2020-01-21 10:22:55,473  INFO ShareLibExcluder:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] Mark file for excluding from distributed cache: /user/oozie/share/lib/lib_20190813213920/oozie/jackson-core-2.9.5.jar
2020-01-21 10:22:55,474  INFO ShareLibExcluder:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] Mark file for excluding from distributed cache: /user/oozie/share/lib/lib_20190813213920/oozie/jackson-core-asl-1.9.13.jar
2020-01-21 10:22:55,474  INFO ShareLibExcluder:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] Mark file for excluding from distributed cache: /user/oozie/share/lib/lib_20190813213920/oozie/jackson-mapper-asl-1.9.13.jar
2020-01-21 10:22:55,483  WARN DependencyDeduplicator:523 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] aws-java-sdk-bundle-1.11.271.jar[hdfs://chs-gqc-379-mn002.us-south.ae.appdomain.cloud:8020/user/oozie/share/lib/lib_20190813213920/oozie/aws-java-sdk-bundle-1.11.271.jar] is already defined in mapreduce.job.cache.files. Skipping.
2020-01-21 10:22:55,484  WARN DependencyDeduplicator:523 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] azure-data-lake-store-sdk-2.2.7.jar[hdfs://chs-gqc-379-mn002.us-south.ae.appdomain.cloud:8020/user/oozie/share/lib/lib_20190813213920/oozie/azure-data-lake-store-sdk-2.2.7.jar] is already defined in mapreduce.job.cache.files. Skipping.
2020-01-21 10:22:55,484  WARN DependencyDeduplicator:523 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] azure-keyvault-core-1.0.0.jar[hdfs://chs-gqc-379-mn002.us-south.ae.appdomain.cloud:8020/user/oozie/share/lib/lib_20190813213920/oozie/azure-keyvault-core-1.0.0.jar] is already defined in mapreduce.job.cache.files. Skipping.
2020-01-21 10:22:55,484  WARN DependencyDeduplicator:523 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] azure-storage-7.0.0.jar[hdfs://chs-gqc-379-mn002.us-south.ae.appdomain.cloud:8020/user/oozie/share/lib/lib_20190813213920/oozie/azure-storage-7.0.0.jar] is already defined in mapreduce.job.cache.files. Skipping.
2020-01-21 10:22:55,484  WARN DependencyDeduplicator:523 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] hadoop-aws-3.1.1.3.1.0.0-78.jar[hdfs://chs-gqc-379-mn002.us-south.ae.appdomain.cloud:8020/user/oozie/share/lib/lib_20190813213920/oozie/hadoop-aws-3.1.1.3.1.0.0-78.jar] is already defined in mapreduce.job.cache.files. Skipping.
2020-01-21 10:22:55,485  WARN DependencyDeduplicator:523 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] hadoop-azure-3.1.1.3.1.0.0-78.jar[hdfs://chs-gqc-379-mn002.us-south.ae.appdomain.cloud:8020/user/oozie/share/lib/lib_20190813213920/oozie/hadoop-azure-3.1.1.3.1.0.0-78.jar] is already defined in mapreduce.job.cache.files. Skipping.
2020-01-21 10:22:55,485  WARN DependencyDeduplicator:523 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] hadoop-azure-datalake-3.1.1.3.1.0.0-78.jar[hdfs://chs-gqc-379-mn002.us-south.ae.appdomain.cloud:8020/user/oozie/share/lib/lib_20190813213920/oozie/hadoop-azure-datalake-3.1.1.3.1.0.0-78.jar] is already defined in mapreduce.job.cache.files. Skipping.
2020-01-21 10:22:55,485  WARN DependencyDeduplicator:523 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] jetty-util-9.3.24.v20180605.jar[hdfs://chs-gqc-379-mn002.us-south.ae.appdomain.cloud:8020/user/oozie/share/lib/lib_20190813213920/oozie/jetty-util-9.3.24.v20180605.jar] is already defined in mapreduce.job.cache.files. Skipping.
2020-01-21 10:22:55,485  WARN DependencyDeduplicator:523 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] jetty-util-ajax-9.3.24.v20180605.jar[hdfs://chs-gqc-379-mn002.us-south.ae.appdomain.cloud:8020/user/oozie/share/lib/lib_20190813213920/oozie/jetty-util-ajax-9.3.24.v20180605.jar] is already defined in mapreduce.job.cache.files. Skipping.
2020-01-21 10:22:55,485  WARN DependencyDeduplicator:523 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] wildfly-openssl-1.0.4.Final.jar[hdfs://chs-gqc-379-mn002.us-south.ae.appdomain.cloud:8020/user/oozie/share/lib/lib_20190813213920/oozie/wildfly-openssl-1.0.4.Final.jar] is already defined in mapreduce.job.cache.files. Skipping.
2020-01-21 10:22:55,485  INFO DependencyDeduplicator:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] mapreduce.job.cache.files dependencies are unified by their filename or symlink name.
2020-01-21 10:22:55,488  INFO SparkActionExecutor:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] Added into spark action configuration mapred.child.env=SPARK_HOME=.,HDP_VERSION=3.1.0.0-78
2020-01-21 10:22:55,489  INFO SparkActionExecutor:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] Trying to load ShareLib exclude pattern for action.
2020-01-21 10:22:55,490  INFO ShareLibExcluder:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] The following sharelib exclude pattern will be used: oozie/jackson.*
2020-01-21 10:22:55,748  INFO ShareLibExcluder:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] Mark file for excluding from distributed cache: /user/oozie/share/lib/lib_20190813213920/oozie/jackson-core-2.9.5.jar
2020-01-21 10:22:55,748  INFO ShareLibExcluder:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] Mark file for excluding from distributed cache: /user/oozie/share/lib/lib_20190813213920/oozie/jackson-core-asl-1.9.13.jar
2020-01-21 10:22:55,748  INFO ShareLibExcluder:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] Mark file for excluding from distributed cache: /user/oozie/share/lib/lib_20190813213920/oozie/jackson-mapper-asl-1.9.13.jar
2020-01-21 10:22:55,798  WARN DependencyDeduplicator:523 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] aws-java-sdk-bundle-1.11.271.jar[hdfs://chs-gqc-379-mn002.us-south.ae.appdomain.cloud:8020/user/oozie/share/lib/lib_20190813213920/oozie/aws-java-sdk-bundle-1.11.271.jar] is already defined in mapreduce.job.cache.files. Skipping.
2020-01-21 10:22:55,799  WARN DependencyDeduplicator:523 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] azure-data-lake-store-sdk-2.2.7.jar[hdfs://chs-gqc-379-mn002.us-south.ae.appdomain.cloud:8020/user/oozie/share/lib/lib_20190813213920/oozie/azure-data-lake-store-sdk-2.2.7.jar] is already defined in mapreduce.job.cache.files. Skipping.
2020-01-21 10:22:55,799  WARN DependencyDeduplicator:523 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] azure-keyvault-core-1.0.0.jar[hdfs://chs-gqc-379-mn002.us-south.ae.appdomain.cloud:8020/user/oozie/share/lib/lib_20190813213920/oozie/azure-keyvault-core-1.0.0.jar] is already defined in mapreduce.job.cache.files. Skipping.
2020-01-21 10:22:55,799  WARN DependencyDeduplicator:523 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] azure-storage-7.0.0.jar[hdfs://chs-gqc-379-mn002.us-south.ae.appdomain.cloud:8020/user/oozie/share/lib/lib_20190813213920/oozie/azure-storage-7.0.0.jar] is already defined in mapreduce.job.cache.files. Skipping.
2020-01-21 10:22:55,799  WARN DependencyDeduplicator:523 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] hadoop-aws-3.1.1.3.1.0.0-78.jar[hdfs://chs-gqc-379-mn002.us-south.ae.appdomain.cloud:8020/user/oozie/share/lib/lib_20190813213920/oozie/hadoop-aws-3.1.1.3.1.0.0-78.jar] is already defined in mapreduce.job.cache.files. Skipping.
2020-01-21 10:22:55,799  WARN DependencyDeduplicator:523 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] hadoop-azure-3.1.1.3.1.0.0-78.jar[hdfs://chs-gqc-379-mn002.us-south.ae.appdomain.cloud:8020/user/oozie/share/lib/lib_20190813213920/oozie/hadoop-azure-3.1.1.3.1.0.0-78.jar] is already defined in mapreduce.job.cache.files. Skipping.
2020-01-21 10:22:55,799  WARN DependencyDeduplicator:523 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] hadoop-azure-datalake-3.1.1.3.1.0.0-78.jar[hdfs://chs-gqc-379-mn002.us-south.ae.appdomain.cloud:8020/user/oozie/share/lib/lib_20190813213920/oozie/hadoop-azure-datalake-3.1.1.3.1.0.0-78.jar] is already defined in mapreduce.job.cache.files. Skipping.
2020-01-21 10:22:55,799  WARN DependencyDeduplicator:523 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] jetty-util-9.3.24.v20180605.jar[hdfs://chs-gqc-379-mn002.us-south.ae.appdomain.cloud:8020/user/oozie/share/lib/lib_20190813213920/oozie/jetty-util-9.3.24.v20180605.jar] is already defined in mapreduce.job.cache.files. Skipping.
2020-01-21 10:22:55,799  WARN DependencyDeduplicator:523 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] jetty-util-ajax-9.3.24.v20180605.jar[hdfs://chs-gqc-379-mn002.us-south.ae.appdomain.cloud:8020/user/oozie/share/lib/lib_20190813213920/oozie/jetty-util-ajax-9.3.24.v20180605.jar] is already defined in mapreduce.job.cache.files. Skipping.
2020-01-21 10:22:55,799  WARN DependencyDeduplicator:523 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] wildfly-openssl-1.0.4.Final.jar[hdfs://chs-gqc-379-mn002.us-south.ae.appdomain.cloud:8020/user/oozie/share/lib/lib_20190813213920/oozie/wildfly-openssl-1.0.4.Final.jar] is already defined in mapreduce.job.cache.files. Skipping.
2020-01-21 10:22:55,799  INFO DependencyDeduplicator:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] mapreduce.job.cache.files dependencies are unified by their filename or symlink name.
2020-01-21 10:22:57,466  INFO SparkActionExecutor:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] Trying to get job [job_1575974364931_0384], attempt [1]
2020-01-21 10:22:57,488  INFO SparkActionExecutor:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] checking action, hadoop job ID [job_1575974364931_0384] status [RUNNING]
2020-01-21 10:22:57,492  INFO ActionStartXCommand:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] [***0000028-200103135925853-oozie-oozi-W@spark-node***]Action status=RUNNING
2020-01-21 10:22:57,492  INFO ActionStartXCommand:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] [***0000028-200103135925853-oozie-oozi-W@spark-node***]Action updated in DB!
2020-01-21 10:22:57,498  INFO WorkflowNotificationXCommand:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] No Notification URL is defined. Therefore nothing to notify for job 0000028-200103135925853-oozie-oozi-W@spark-node
2020-01-21 10:23:31,755  INFO CallbackServlet:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] callback for action [0000028-200103135925853-oozie-oozi-W@spark-node]
2020-01-21 10:23:31,916  INFO SparkActionExecutor:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] Trying to get job [job_1575974364931_0384], attempt [1]
2020-01-21 10:23:32,165  INFO SparkActionExecutor:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] action completed, external ID [job_1575974364931_0384]
2020-01-21 10:23:32,216  INFO SparkActionExecutor:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] Action ended with external status [SUCCEEDED]
2020-01-21 10:23:32,261  INFO JPAService:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] No results found
2020-01-21 10:23:32,281  INFO ActionStartXCommand:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@end] Start action [0000028-200103135925853-oozie-oozi-W@end] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2020-01-21 10:23:32,289  INFO EndActionExecutor:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@end] Starting action
2020-01-21 10:23:32,299  INFO ActionStartXCommand:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@end] [***0000028-200103135925853-oozie-oozi-W@end***]Action status=DONE
2020-01-21 10:23:32,299  INFO ActionStartXCommand:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@end] [***0000028-200103135925853-oozie-oozi-W@end***]Action updated in DB!
2020-01-21 10:23:32,320  INFO EndActionExecutor:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[clsadmin] GROUP[-] TOKEN[] APP[CMDW-Security-User-Management-Spark] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@end] Action ended with external status [OK]
2020-01-21 10:23:32,377  INFO WorkflowNotificationXCommand:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@end] No Notification URL is defined. Therefore nothing to notify for job 0000028-200103135925853-oozie-oozi-W@end
2020-01-21 10:23:32,378  INFO WorkflowNotificationXCommand:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000028-200103135925853-oozie-oozi-W
2020-01-21 10:23:32,378  INFO WorkflowNotificationXCommand:520 - SERVER[chs-gqc-379-mn002.us-south.ae.appdomain.cloud] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000028-200103135925853-oozie-oozi-W] ACTION[0000028-200103135925853-oozie-oozi-W@spark-node] No Notification URL is defined. Therefore nothing to notify for job 0000028-200103135925853-oozie-oozi-W@spark-node

Job Log

Sparker0i
  • 1,787
  • 4
  • 35
  • 60
  • Is it possible to show oozie log info what is being printed for this action? Also are you running in yarn-cluster mode or yarn-client mode? Just wanted to understand if it is acl permission issues or something else. Try to run in local mode from oozie and you should see the print statements. – yammanuruarun Jan 19 '20 at 20:49
  • I'm running in local mode and no println statements are visible there.. – Sparker0i Jan 20 '20 at 15:12
  • Can you tell what is being displayed in the default section under logs of spark action. In-case if you are facing permission issues while seeing logs have the below comment properties declared in the xml within global tag as shown in the reference. https://oozie.apache.org/docs/4.2.0/WorkflowFunctionalSpec.html#a19_Global_Configurations https://hadoop.apache.org/docs/r2.7.2/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml Note: For now give * in view acl property but you can change accordingly as per need. – yammanuruarun Jan 20 '20 at 18:23
  • ${job-tracker} ${name-node} job1.xml mapreduce.cluster.acls.enabled true mapreduce.job.acl-view-job * If it was java or shell action we could have used – yammanuruarun Jan 21 '20 at 10:05
  • Please check Edit 1 – Sparker0i Jan 21 '20 at 10:42
  • The log provided seems to be oozie launcher job log. The log i referred to is spark action specific log from hue-->workflows-->(select)job-->tasks-->log(of spark action)-->default. In case if you are not using hue try to check the spark action specific log from oozie web cosole. – yammanuruarun Jan 21 '20 at 12:00
  • I dont use Hue (and have no option to do so either). I can only use Oozie web console and/or run Oozie commands from terminal. How to check spark action log from Oozie? – Sparker0i Jan 21 '20 at 12:28
  • 1
    oozie job -oozie ${oozie_url} -info 0000028-200103135925853-oozie-oozi-W@spark-node From above output, go to 'Console URL' link provided using browser. Choose successful maps number-->Mapper attempts will be displayed-->Click logs-->Different "Log Type:" will be displayed in this page. Under "Log Type: stdout" click for full log and you should find the println outputs. – yammanuruarun Jan 23 '20 at 10:39

0 Answers0