Solr upstart script using start-stop-daemon

Question

I want to start my solr using upstart command.

The command I use to start solar normally is

/usr/bin/java \
-Dsolr.solr.home=/home/justeat/work/projects/solr/apache-solr-4.0.0/example/solr \
-Djetty.port=8983 \
-Djetty.logs=/home/justeat/work/projects/solr/apache-solr-4.0.0/example/solr/logs \
-Djetty.home=/home/justeat/work/projects/solr/apache-solr-4.0.0/example \
-Djava.io.tmpdir=/tmp \
-jar /home/justeat/work/projects/solr/apache-solr-4.0.0/example/start.jar

Now, when I use

DAEMON_OPTS="-Dsolr.solr.home=/home/bitnami/search/solr/solr-4.2.1/example/solr -Djetty.port=8983 -Djetty.logs=/home/bitnami/search/solr/solr-4.2.1/example/logs -Djetty.home=/home/bitnami/search/solr/solr-4.2.1/example -Djava.io.tmpdir=/tmp -jar /home/bitnami/search/solr/solr-4.2.1/example/start.jar"

exec /usr/bin/java ${DAEMON_OPTS} >> /var/log/log.log 2>&1

Upstart works fine. When I'm trying to use start-stop-daemon with upstart script, solr is not getting some of the paramers I'm sending in DAEMON_OPTS.

exec start-stop-daemon \
        --start \
        --make-pidfile \
        --pidfile /var/run/solr_upstart.pid \
        --chuid $USER \
        --exec /usr/bin/java -- $DAEMON_OPTS >> /var/log/log.log 2>&1

Solr logs are below,

2014-10-23 12:32:15.878:INFO:oejs.Server:jetty-8.1.2.v20120308
2014-10-23 12:32:15.932:INFO:oejdp.ScanningAppProvider:Deployment monitor /home/justeat/work/projects/solr/apache-solr-4.0.0/example/contexts at interval 0
2014-10-23 12:32:15.946:INFO:oejd.DeploymentManager:Deployable added: /home/justeat/work/projects/solr/apache-solr-4.0.0/example/contexts/solr.xml
2014-10-23 12:32:16.056:WARN:oejx.XmlConfiguration:Config error at <Set name="tempDirectory"><Property name="jetty.home" default="."/>/solr-webapp</Set> java.lang.reflect.InvocationTargetException
2014-10-23 12:32:16.056:WARN:oejd.DeploymentManager:Unable to reach node goal: started
java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.eclipse.jetty.xml.XmlConfiguration$JettyXmlConfiguration.set(XmlConfiguration.java:565)
    at org.eclipse.jetty.xml.XmlConfiguration$JettyXmlConfiguration.configure(XmlConfiguration.java:378)
    at org.eclipse.jetty.xml.XmlConfiguration$JettyXmlConfiguration.configure(XmlConfiguration.java:338)
    at org.eclipse.jetty.xml.XmlConfiguration.configure(XmlConfiguration.java:291)
    at org.eclipse.jetty.deploy.providers.ContextProvider.createContextHandler(ContextProvider.java:68)
    at org.eclipse.jetty.deploy.App.getContextHandler(App.java:97)
    at org.eclipse.jetty.deploy.bindings.StandardDeployer.processBinding(StandardDeployer.java:33)
    at org.eclipse.jetty.deploy.AppLifeCycle.runBindings(AppLifeCycle.java:183)
    at org.eclipse.jetty.deploy.DeploymentManager.requestAppGoal(DeploymentManager.java:491)
    at org.eclipse.jetty.deploy.DeploymentManager.addApp(DeploymentManager.java:138)
    at org.eclipse.jetty.deploy.providers.ScanningAppProvider.fileAdded(ScanningAppProvider.java:142)
    at org.eclipse.jetty.deploy.providers.ScanningAppProvider$1.fileAdded(ScanningAppProvider.java:53)
    at org.eclipse.jetty.util.Scanner.reportAddition(Scanner.java:604)
    at org.eclipse.jetty.util.Scanner.reportDifferences(Scanner.java:535)
    at org.eclipse.jetty.util.Scanner.scan(Scanner.java:398)
    at org.eclipse.jetty.util.Scanner.doStart(Scanner.java:332)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
    at org.eclipse.jetty.deploy.providers.ScanningAppProvider.doStart(ScanningAppProvider.java:118)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
    at org.eclipse.jetty.deploy.DeploymentManager.startAppProvider(DeploymentManager.java:552)
    at org.eclipse.jetty.deploy.DeploymentManager.doStart(DeploymentManager.java:227)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
    at org.eclipse.jetty.util.component.AggregateLifeCycle.doStart(AggregateLifeCycle.java:63)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:53)
    at org.eclipse.jetty.server.handler.HandlerWrapper.doStart(HandlerWrapper.java:91)
    at org.eclipse.jetty.server.Server.doStart(Server.java:263)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
    at org.eclipse.jetty.xml.XmlConfiguration$1.run(XmlConfiguration.java:1215)
    at java.security.AccessController.doPrivileged(Native Method)
    at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1138)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.eclipse.jetty.start.Main.invokeMain(Main.java:457)
    at org.eclipse.jetty.start.Main.start(Main.java:602)
    at org.eclipse.jetty.start.Main.main(Main.java:82)
Caused by: 
java.lang.IllegalArgumentException: Bad temp directory: /solr-webapp
    at org.eclipse.jetty.webapp.WebAppContext.setTempDirectory(WebAppContext.java:1098)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.eclipse.jetty.xml.XmlConfiguration$JettyXmlConfiguration.set(XmlConfiguration.java:565)
    at org.eclipse.jetty.xml.XmlConfiguration$JettyXmlConfiguration.configure(XmlConfiguration.java:378)
    at org.eclipse.jetty.xml.XmlConfiguration$JettyXmlConfiguration.configure(XmlConfiguration.java:338)
    at org.eclipse.jetty.xml.XmlConfiguration.configure(XmlConfiguration.java:291)
    at org.eclipse.jetty.deploy.providers.ContextProvider.createContextHandler(ContextProvider.java:68)
    at org.eclipse.jetty.deploy.App.getContextHandler(App.java:97)
    at org.eclipse.jetty.deploy.bindings.StandardDeployer.processBinding(StandardDeployer.java:33)
    at org.eclipse.jetty.deploy.AppLifeCycle.runBindings(AppLifeCycle.java:183)
    at org.eclipse.jetty.deploy.DeploymentManager.requestAppGoal(DeploymentManager.java:491)
    at org.eclipse.jetty.deploy.DeploymentManager.addApp(DeploymentManager.java:138)
    at org.eclipse.jetty.deploy.providers.ScanningAppProvider.fileAdded(ScanningAppProvider.java:142)
    at org.eclipse.jetty.deploy.providers.ScanningAppProvider$1.fileAdded(ScanningAppProvider.java:53)
    at org.eclipse.jetty.util.Scanner.reportAddition(Scanner.java:604)
    at org.eclipse.jetty.util.Scanner.reportDifferences(Scanner.java:535)
    at org.eclipse.jetty.util.Scanner.scan(Scanner.java:398)
    at org.eclipse.jetty.util.Scanner.doStart(Scanner.java:332)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
    at org.eclipse.jetty.deploy.providers.ScanningAppProvider.doStart(ScanningAppProvider.java:118)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
    at org.eclipse.jetty.deploy.DeploymentManager.startAppProvider(DeploymentManager.java:552)
    at org.eclipse.jetty.deploy.DeploymentManager.doStart(DeploymentManager.java:227)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
    at org.eclipse.jetty.util.component.AggregateLifeCycle.doStart(AggregateLifeCycle.java:63)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:53)
    at org.eclipse.jetty.server.handler.HandlerWrapper.doStart(HandlerWrapper.java:91)
    at org.eclipse.jetty.server.Server.doStart(Server.java:263)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
    at org.eclipse.jetty.xml.XmlConfiguration$1.run(XmlConfiguration.java:1215)
    at java.security.AccessController.doPrivileged(Native Method)
    at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1138)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.eclipse.jetty.start.Main.invokeMain(Main.java:457)
    at org.eclipse.jetty.start.Main.start(Main.java:602)
    at org.eclipse.jetty.start.Main.main(Main.java:82)
2014-10-23 12:32:16.081:INFO:oejs.AbstractConnector:Started SocketConnector@0.0.0.0:8983

score 1 · Answer 1 · answered May 07 '15 at 13:32

I don’t use exactly the same setup, but I got the same error message. The culprit was that the WAR file under /opt/solr-5.1.0/server/webapps/solr.war was extracted to /opt/solr-5.1.0/server/solr-webapp/ upon each start of Solr after the credentials were changed. So that path had to be writable for the user running Solr (or you could change some configuration instead, I guess).

score 0 · Answer 2 · answered Oct 24 '14 at 00:00

0

A few ideas:

Upstart might be silently shortening the exec stanza
s-s-d (start-stop-daemon) might be doing that
maybe you are hitting the kernel limit for number or size of options?

Best bet is to:

remove the --pidfile and --make-pidfile options from s-s-d's opts, they are useless with Upstart
remove the redirection to /var/log/log.log, just rely on Upstart to catch stdout/err
consider using the setuid stanza of Upstart and completely removing the s-s-d invocation

answered Oct 24 '14 at 00:00

CameronNemo

616
3
10

I tried your suggestions. I'm getting same error when I use setuid stanza. If don't use it, everything works fine. So, I guess error is something related to file permissions. Not sure what permission is missing. – robert Oct 27 '14 at 08:45

Solr upstart script using start-stop-daemon

2 Answers2