The dir step is generally used for workspace-relative paths where the agent is assumed to have full control over the directory, and Jenkins will use sibling directories for various things such as temp directories or shared libraries. If you want to install things in the absolute directory /opt/tmp on the agent, I would do it in a script with the sh step so that you have full control over what will happen and so that the Pipeline does not run with /opt/tmp as its workspace directory at any point.

You might be able to use the Workspace Cleanup Plugin to clear out the temp directory, but to take a step back, I think it would make more sense to just use an agent that already has those tools installed in the first place, i.e. some kind of container or pre-built VM image, so that you aren't changing your agent configuration inside of Jenkins at all.

In the scenario where you have a pipeline where you have 2 stages, each of them executed in a docker container where the USER of the container is "root". You would have the following problem.

Imagine that Stage A executes something like: sh ("mkdir -p foo/bar"). This will generate the directories foo and bar with user and group permissions of "root".

Stage B executes using dir("foo/bar") the following command : sh ("touch somefile.txt"). 

Due to the fact that dir() tries to create a @tmp directory within "foo/bar" the pipeline will fail prompting something like this:

Exception: java.nio.file.AccessDeniedException: </path/to/the/workspace...>/foo/bar/@tmp

I omit the workspace path as it is not relevant. Why is this @tmp creation actually needed?

Why is this @tmp creation actually needed?

The @tmp directory is created by the sh step to store things like the log output and return code of the script (see all uses of controlDir in FileMonitoringTask.java), and is created as a sibling to the workspace directory, since the agent is assumed to have write access to the workspace and its parent directory. The problem is that the dir step actually changes the workspace for everything inside of its body, so the shell step ends up creating a temp directory adjacent to the directory specified by dir, and if your permissions are such that creating directories next to the one specified by dir fails, the sh step will fail as well.

I don't know if there is a way for FileMonitoringController to use the original workspace rather than the directory specified by dir when creating the temp directory, but if so that might be a way to fix this (I think only the current workspace is available, although maybe we could change the dir step to provide the original value in some way. I wonder if looking at what the parallel step does for the workspace would be useful here.). As a workaround you should be able to use cd in your sh script instead of the dir step.

So, if I correctly understand the answer above, the point is that Jenkins needs some space to record its own stuff when a job is running. IMHA (but perhaps there are some cases I don't imagine where it is not possible ?) the better place to do that is the build workspace, whatever the current working directory is. At least, jenkins is sure to be able (i.e. have rwx rights) to create its temp directory.

I guess that it should not be too difficult for a jenkins code to retrieve the aboslute path of the build workspace.

Besides, as a jenkins admin, I wouldn't still see this temporary directory once the build is achieved. Please, let this place as clean as you found it and sweep your own dust ! . Especially given that this temp dir looks to be empty in most cases when the build is over.

Best Regards

The behavior is both undocumented (as far as I can tell), and surprising, since normally "change current directory" would not be a mutating operation.
 
Also, using the name of the directory requested by the user compounds the confusion, because it's easy to miss the @tmp suffix, or think it's an artifact of the exception rendering. I didn't really notice the @tmp suffix until after I had been fighting with this problem for a while, believing Jenkins was trying to create a directory that already existed.

Do not use dir for this purpose. Simply start your sh script with cd /opt/tools.

A have a pipeline which builds and creates installer/.zip archive. I'm using "dir" to change directory to a folder and execute python script there. The problem I encountered is that now the installer/zip contains additional empty folder@tmp.

Edit: I've read the explanation that Devin wrote again, and I see that it's not "dir" that created the folder@tmp, but rather the "sh" step (or in my case - the "bat" step probably).

Is that the only step that does it?

It's a bit problematic. Now I'm affraid to use "dir", because who knows how will it affect other steps? And it's a very useful step...

If "sh"/"bat" is the only such step, maybe it could create the temp folder in some location for temporary files, if jenkins provides such thing. If not, maybe jenkins could be extended to provide it.

jglick ‘Simply’ adding a sh “cd /opt/tools” does not work in a declarative pipeline. Subsequent sh steps will have their current directory reset back to the workspace directory. This is frustrating and not documented.

Hi all,

The point is not only related to sh something, it is a more generic issue. Have I done the same with python, ruby or groovy step, I would have the same.
The point is actually that when a dir() occurs, a @tmp folder is created behind the scene, and this is potentially a problem in some cases.

Any update here. I consider this a bug because it makes pipelines fail with strange errors and I have to add code that cleans up the mess. Also `dir()` is simply not relly usable then.

I've got the same problem with dir() in my Pipeline script. Is there any solution available yet?

This is indeed a bug and I don't even manage to make the "cd " workaround work:

The following pipeline:

pipeline {
    agent {label 'ansible'}
    stages {
        stage('playbook') {
            steps {
                sh '''
                    cd /git/it/ansible/infrastructure
                    pwd
                    ls -al
                    ./vault_pass
                    ansible-playbook --check docker.yml
                '''
            }
        }
    }
}

yields that result:

+ cd /git/it/ansible/infrastructure
+ pwd
/git/it/ansible/infrastructure
+ ls -al
total 8
drwxrwxrwx    1 root     root          4096 Nov 18 19:26 .
drwxrwxrwx    1 root     root          4096 Nov 18 19:25 ..
drwxrwxrwx    1 root     root          4096 Dec 14  2020 .git
-rwxrwxrwx    1 root     root           234 Nov 20  2019 .gitignore
-rwxrwxrwx    1 root     root           218 Nov 20  2019 .project
-rwxrwxrwx    1 root     root          3354 Nov 20  2019 Vagrantfile
-rwxrwxrwx    1 root     root           131 Nov 20  2019 all.yml
-rwxrwxrwx    1 root     root           380 Nov 20  2019 ansible.cfg
-rwxrwxrwx    1 root     root           720 Nov 20  2019 ansible.yml
-rwxrwxrwx    1 root     root           126 Nov 20  2019 aptgetclean.yml
-rwxrwxrwx    1 root     root            61 Nov 20  2019 common.yml
-rwxrwxrwx    1 root     root           513 Nov 20  2019 docker.yml
drwxrwxrwx    1 root     root          4096 Nov 20  2019 files
drwxrwxrwx    1 root     root          4096 Nov 20  2019 group_vars
drwxrwxrwx    1 root     root          4096 Nov 20  2019 host_vars
drwxrwxrwx    1 root     root          4096 Nov 20  2019 inventories
-rwxrwxrwx    1 root     root            94 Nov 20  2019 monitoring.yml
drwxrwxrwx    1 root     root          4096 Nov 20  2019 public_keys
-rwxrwxrwx    1 root     root           227 Nov 20  2019 requirements.yml
drwxrwxrwx    1 root     root          4096 Nov 20  2019 roles
drwxrwxrwx    1 root     root          4096 Nov 20  2019 scripts
drwxrwxrwx    1 root     root          4096 Nov 20  2019 templates
-rwxrwxrwx    1 root     root          2172 Nov 20  2019 updater-rsync.yml
-rwxrwxrwx    1 root     root            44 Nov 18 19:56 vault_pass
-rwxrwxrwx    1 root     root            64 Nov 20  2019 weblate.yml
+ ./vault_pass
/home/workspace/ansible@tmp/durable-ac15a4f4/script.sh: line 1: ./vault_pass: not found

EDIT: this formatting language is difficult to handle....
 

I'm surprised to see this issue is still not fixed yet in 2023...

I don't think the AccessDeniedException is unexpected.
dir seems to expect a relative path as argument.

Ref: https://www.jenkins.io/doc/pipeline/steps/workflow-durable-task-step/#dir-change-current-directory

I fail to see how AccessDeniedException is to be expected.

It is at least not mentioned in the documentation, let alone be intuitive that whenever we change directory a @tmp folder will be generated as sibling (and therefore the parent must be writable). 

What I expected, is that when I use the dir directive to change folder, I would actually be in that folder.

We ran into this very same issue some days ago. Actually I am puzzled we haven't had this trouble earlier already.

According to my experiments there is a (subtile) difference on wether the directory does exist on checkout(seems working) or wether it was created during the build in another step (causing troubles).

Any suggestions for workarounds available?

Could anyone suggest a possible workaround for this issue?