4

I have multiple GetSplunk processors running using a Cron driven scheduling strategy. The Cron expression looks like '0 30 13 * * ?'. They all successfully execute the query the first time it's run. But, the next day it errors out with a 401 error from Splunk. The error from nifi-app.log is as below. The Cron scheduler in NiFi is a QuartzScheduler.

WARN [Timer-Driven Process Thread-7] o.a.n.c.t.ContinuallyRunProcessorTask Administratively 
Yielding GetSplunk[id=01581009-026c-114b-5e2e-401ebea6427d] due to uncaught Exception: 
com.splunk.HttpException: HTTP 401 -- call not properly authenticated
2016-12-21 13:30:00,300 WARN [Timer-Driven Process Thread-2] o.a.n.c.t.ContinuallyRunProcessorTask
com.splunk.HttpException: HTTP 401 -- call not properly authenticated
at com.splunk.HttpException.create(HttpException.java:84) ~[na:na]
at com.splunk.HttpService.send(HttpService.java:452) ~[na:na]
at com.splunk.Service.send(Service.java:1293) ~[na:na]
at com.splunk.HttpService.get(HttpService.java:165) ~[na:na]
at com.splunk.Service.export(Service.java:222) ~[na:na]
at com.splunk.Service.export(Service.java:237) ~[na:na]
at org.apache.nifi.processors.splunk.GetSplunk.onTrigger(GetSplunk.java:461) ~[na:na]
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) ~[nifi-api-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1064) ~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at    org.apache.nifi.controller.scheduling.QuartzSchedulingAgent$2.run(QuartzSchedulingAgent.java:165) [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_101]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_101]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_101]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.8.0_101]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_101]
at    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_101]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]`

The GetSplunk processor makes use of Java SDK for Splunk. My earliest time is '-24h' and latest time is 'now'. I am hitting splunkd with port 8089. After doing some research, it seems this might be an issue due to multiple threads. Have anyone of you bumped into this? Help appreciated.

MoDrags
  • 141
  • 2
  • 13

1 Answers1

5

After looking into the code for a few minutes, I think this is an issue in the NiFi processor.

The first time the processor executes, it lazily initializes the Splunk Service object:

https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-splunk-bundle/nifi-splunk-processors/src/main/java/org/apache/nifi/processors/splunk/GetSplunk.java#L372-L377

As part of this initialization, the Splunk service calls a login method like this:

public Service login(String username, String password) {
        this.username = username;
        this.password = password;

        Args args = new Args();
        args.put("username", username);
        args.put("password", password);
        args.put("cookie", "1");
        ResponseMessage response = post("/services/auth/login", args);
        String sessionKey = Xml.parse(response.getContent())
            .getElementsByTagName("sessionKey")
            .item(0)
            .getTextContent();
        this.token = "Splunk " + sessionKey;
        this.version = this.getInfo().getVersion();
        if (versionCompare("4.3") >= 0)
            this.passwordEndPoint = "storage/passwords";

        return this;
    }

Since this only happens the first time the processor executes, it will only happen again if you stop and start the processor. In your case, your processor has been running long enough that session probably expired and the processor is continuing to attempt to execute.

I created this JIRA for the issue: https://issues.apache.org/jira/browse/NIFI-3349

Bryan Bende
  • 18,320
  • 1
  • 28
  • 39