How to solve the Jenkins' java.nio.file.AccessDeniedException on a remote node?

Question

I created a sandbox project and want to run its declarative pipeline on a remote slave/node/agent. I see, that the SSH connection between Jenkins and the slave is working. Jenkins can login on the remove machine (as user ec2-user). But the execution ends up in an exception: java.nio.file.AccessDeniedException: /srv/www/htdocs/sandbox@tmp.

The agent's root directory (where Jenkins creates the remoting.jar) is /srv/www/htdocs/jenkins-agents. The project's root directory (passed to the pipeline as parameter) is /srv/www/htdocs/sandbox. But Jenkins seems to try creating a new folder: /srv/www/htdocs/sandbox@tmp. And that cannot work, since the parent folder /srv/www/htdocs has some access recrictions and creating new directories in it is only allowed for root / sudo users.

Why does Jenkins try to create this @tmp folder? How to avoid this behavior and get it working?

Addition info

Executing of CLI commands is failing with this console output:

Started by user admin
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Produktivserver in /srv/www/htdocs/jenkins-agents/workspace/Sandbox
[Pipeline] {
[Pipeline] ws
Running in /srv/www/htdocs/sandbox
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Build)
[Pipeline] sh
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // ws
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to Produktivserver
        at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1788)
        at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
        at hudson.remoting.Channel.call(Channel.java:998)
        at hudson.FilePath.act(FilePath.java:1069)
        at hudson.FilePath.act(FilePath.java:1058)
        at hudson.FilePath.mkdirs(FilePath.java:1243)
        at org.jenkinsci.plugins.durabletask.FileMonitoringTask$FileMonitoringController.<init>(FileMonitoringTask.java:198)
        at org.jenkinsci.plugins.durabletask.BourneShellScript$ShellController.<init>(BourneShellScript.java:337)
        at org.jenkinsci.plugins.durabletask.BourneShellScript$ShellController.<init>(BourneShellScript.java:326)
        at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:178)
        at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:103)
        at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:317)
        at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:286)
        at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:179)
        at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
        at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
        at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:163)
        at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
        at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:157)
        at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:161)
        at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:165)
        at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:135)
        at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
        at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)
        at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
        at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)
        at jdk.internal.reflect.GeneratedMethodAccessor187.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
        at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
        at com.cloudbees.groovy.cps.Next.step(Next.java:83)
        at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
        at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
        at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
        at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
        at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
        at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
        at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
        at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)
        at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:400)
        at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
        at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:312)
        at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:276)
        at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
        at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
        at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
java.nio.file.AccessDeniedException: /srv/www/htdocs/sandbox@tmp
    at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
    at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
    at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
    at java.base/sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:389)
    at java.base/java.nio.file.Files.createDirectory(Files.java:689)
    at java.base/java.nio.file.Files.createAndCheckIsDirectory(Files.java:796)
    at java.base/java.nio.file.Files.createDirectories(Files.java:782)
    at hudson.FilePath.mkdirs(FilePath.java:3256)
    at hudson.FilePath.access$1300(FilePath.java:211)
    at hudson.FilePath$Mkdirs.invoke(FilePath.java:1251)
    at hudson.FilePath$Mkdirs.invoke(FilePath.java:1247)
    at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3069)
    at hudson.remoting.UserRequest.perform(UserRequest.java:211)
    at hudson.remoting.UserRequest.perform(UserRequest.java:54)
    at hudson.remoting.Request$2.run(Request.java:369)
    at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)
Finished: FAILURE

score 4 · Answer 1 · answered Apr 08 '21 at 11:54

See Jenkins Issue 52750 for more details.

Short summary: the @tmp directories are created by the sh step to record internal logs etc.

Jenkins assumes that the agent has full control of the directory it is running in.

The recommendation in that issue is to use a script that runs in a Jenkins workspace and deploys from there rather than trying to manipulate the target directory directly.

This will also allow you to handle file system errors/permissions.

Opening up permissions to world readable and world writable is a bad idea. Especially on a directory that serves as a DocumentRoot for a web server.

score -4 · Answer 2 · answered Jun 09 '20 at 18:46

-4

The solution is to make the parent of the workspace folder writable for Jenkins, or speaking more precisely, for the user, the Jenkins logs on the remote machine with. Here is a very simple (but insecure) way to do this (for the workspace folder /path/parent/project and the temp folder /path/parent/project@tmp):

$ chmod 777 /path/parent/

answered Jun 09 '20 at 18:46

automatix

14,018
26
105
230

3

Please don't recommend people to open directories read-write for all users on a system. – Bram Apr 08 '21 at 11:48
@Bram In general you are right. But I mentioned, that it's insecure. And everyone is free to decide, whether to use an insecure solution or not. – automatix Apr 09 '21 at 07:08

How to solve the Jenkins' java.nio.file.AccessDeniedException on a remote node?

2 Answers2