Summary

Per the Kubernetes plugin documentation, one can restrict which can execute on clouds at the folder level.

Jenkins however, seems to use label selectors to find a build agent for a job (as reported by JENKINS-56546). These label selectors do not take into account whether the build agent selected is running in a restricted cloud or not.

If you have a job in a folder that does not have access to a restricted cloud but specifies a label selector that matches an agent running in a restricted cloud, that job will run on the restricted cloud's agent regardless of the fact that cloud is restricted.

Use Case

Our organization has two tiers of developers: privileged and unprivileged. Unprivileged developers are not allowed to deploy to production environments.

We use Jenkinsfiles to define pipelines. Our pipelines deploy to dev, test, and production environments. We want to build contributions from unprivileged developers, but ensure unmerged contributions don't go beyond dev environments. Since Jenkinsfiles can be modified on a branch, and Jenkins will build whatever is in the modified Jenkinsfile, there's nothing inherently preventing a unprivileged developer from modifying a Jenkinsfile to deploy say to production.

To mitigate we have:

• two different jobs
  • a privileged job which triggers off master and tags
  • an unprivileged job which triggers off pull requests.
• those jobs are in two different folders
  • a privileged folder
  • an unprivileged folder.
• two clouds
  • privileged: restricted for use only by the privileged folder
  • unprivileged: open to any job. Cloud is restricted to only have permissions and network access to our dev environments.

Both the privileged and unprivileged jobs reference the same Jenkinsfile, but the cloud restrictions prevent modifications on branches to the Jenkinsfile from reaching beyond dev.

The problem is, since these jobs reference the same Jenkinsfile, they used the same label, and we realized that we had unprivileged jobs running on privileged agents. We can parameterize the label so this doesn't happen by default, but since the labels are configured in the Jenkinsfile, you can't ensure the Jenkinsfile isn't modified on a branch to use a privileged agent. Not good.

Is there a way to prevent jobs from running on agents in clouds they shouldn't have access to in a manner that is controlled outside of the Jenkinsfile?

Currently the only solution we can think of is to have two separate Jenkins masters, one for privileged jobs and one for unprivileged, but this makes the folder restrictions for clouds seem less useful.