Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-31866

Attempts to restrict Pipeline jobs from running on master result in job hanging

      Trying the job-restrictions-plugin with Workflow jobs - I added a restriction to master that only jobs matching a regex could run on there. At first glance, that looked perfect - a Workflow job that didn't match that regex stayed in queue while another Workflow job that did match ran fine.

      But then I realized that the non-matching job was still sitting in queue, even after another executor not on the master was available. The job is just stuck - it's not even getting to the node { }, etc. If it can't run on master (when master has executors), it just flops around. Ow.

          [JENKINS-31866] Attempts to restrict Pipeline jobs from running on master result in job hanging

          Great idea!  Thanks.

          michael jackson added a comment - Great idea!  Thanks.

          Hi,

          i think this is not a minor issue. Because of reasons I don't want to run pipelines on master node, I want to run it on slave. Piepeline is always waiting for a node master, no matter if I specify.

          pipeline

          {     agent <label> ... }

           or 

           

          pipeline

          {     agent any ... }

           

          I found no simple solution for this. Is there any?

           

          Martin Cimbalek added a comment - Hi, i think this is not a minor issue. Because of reasons I don't want to run pipelines on master node, I want to run it on slave. Piepeline is always waiting for a node master, no matter if I specify. pipeline {     agent <label> ... }  or    pipeline {     agent any ... }   I found no simple solution for this. Is there any?  

          Artur Szostak added a comment -

          A possible workaround: Use the "Job class" restriction with "org.jenkinsci.plugins.workflow.job.WorkflowJob" for the class name in a "Or" combination with other restriction expressions.

          In my particular case I want to reserve the executors on the master to jobs owned by an account called jenkins and for jobs from an Admin folder. So I have the configuration as indicated in the following image:

          Note that apparently one can ignore the warning indicated in the GUI. Its yet another Jenkins bug. Things seem to work in my case anyway.

          Artur Szostak added a comment - A possible workaround: Use the "Job class" restriction with "org.jenkinsci.plugins.workflow.job.WorkflowJob" for the class name in a "Or" combination with other restriction expressions. In my particular case I want to reserve the executors on the master to jobs owned by an account called jenkins and for jobs from an Admin folder. So I have the configuration as indicated in the following image: Note that apparently one can ignore the warning indicated in the GUI. Its yet another Jenkins bug. Things seem to work in my case anyway.

          Alexandre Martel added a comment - - edited

          Pipeline dispatch to slave node seems to always execute in the master first. The workaround seems to create a local slave node for jobs that should run on the master machine and never use the master node for running any jobs. I set the master node usage to "Only builds jobs with label expressions matching this node".

          But i think that dispatching pipeline job to nodes shouldn't be restricted by the master number of executor.

          Alexandre Martel added a comment - - edited Pipeline dispatch to slave node seems to always execute in the master first. The workaround seems to create a local slave node for jobs that should run on the master machine and never use the master node for running any jobs. I set the master node usage to "Only builds jobs with label expressions matching this node". But i think that dispatching pipeline job to nodes shouldn't be restricted by the master number of executor.

          Sam Gleske added a comment - - edited

          Pipelines orchestrate from the master runtime. There's nothing you can change that. You can set the master executors to 0 and it will still be running the pipeline orchestration within the master.

          In general, a major concern for me was users abusing the master by adding to their Jenkinsfiles.

          node('master') {
              sh 'im-a-bad-person.sh'
          }
          

          However, with suggestions from this thread one can easily restrict the master by only allowing the Jenkins pipeline orchestration classes (and not the WorkflowRun class which is the actual pipeline build that runs on agents). sdalex check out some of my earlier comments in this issue. I share some of my scripts and methods for handling pipelines and cleanup as well as automatically configuring.

          Sam Gleske added a comment - - edited Pipelines orchestrate from the master runtime. There's nothing you can change that. You can set the master executors to 0 and it will still be running the pipeline orchestration within the master. In general, a major concern for me was users abusing the master by adding to their Jenkinsfiles. node('master') { sh 'im-a-bad-person.sh' } However, with suggestions from this thread one can easily restrict the master by only allowing the Jenkins pipeline orchestration classes (and not the WorkflowRun class which is the actual pipeline build that runs on agents). sdalex check out some of my earlier comments in this issue. I share some of my scripts and methods for handling pipelines and cleanup as well as automatically configuring.

          Sam Gleske added a comment -

          aszostak I get that warning too. The primary reason for the error is the class loader for plugins is only populated with related plugins in which it depends. Since the job-restrictions plugin does not directly depend on pipeline plugins, the class loader for it does not include the workflow set of classes. You're right that you can ignore the error and it will "just work". Can probably be considered a bug. It should probably be using the uber classloader which exists in the plugin manager.

          Sam Gleske added a comment - aszostak I get that warning too. The primary reason for the error is the class loader for plugins is only populated with related plugins in which it depends. Since the job-restrictions plugin does not directly depend on pipeline plugins, the class loader for it does not include the workflow set of classes. You're right that you can ignore the error and it will "just work". Can probably be considered a bug. It should probably be using the uber classloader which exists in the plugin manager.

          Sam Gleske added a comment -

          aszostak I opened a PR addressing the warning you experienced https://github.com/jenkinsci/job-restrictions-plugin/pull/23

          Sam Gleske added a comment - aszostak I opened a PR addressing the warning you experienced https://github.com/jenkinsci/job-restrictions-plugin/pull/23

          Artur Szostak added a comment -

          Thank you for the warning fix. Looking forward to seeing the warning disappear in some future upgrade of our Jenkins instance.

          Artur Szostak added a comment - Thank you for the warning fix. Looking forward to seeing the warning disappear in some future upgrade of our Jenkins instance.

          Adrian Vlad added a comment -

          I have a similar problem. I have a pipeline called "test_pipeline" and inside it does:

          node('some_node') {
          ...
          }

          On "some_node" I use Restrict jobs execution at node -> Regular Expression (Job Name) with value test_pipeline.

          This configuration doesn't work (the build hangs). But if I change the value to test_pipeline.* then it works.

          What is the problem? Is job-restrictions using different pipeline names with some suffix? (test_pipeline/.* also works)

          Adrian Vlad added a comment - I have a similar problem. I have a pipeline called "test_pipeline" and inside it does: node( 'some_node' ) { ... } On "some_node" I use  Restrict jobs execution at node ->  Regular Expression (Job Name) with value  test_pipeline . This configuration doesn't work (the build hangs). But if I change the value to  test_pipeline.* then it works. What is the problem? Is  job-restrictions using different pipeline names with some suffix? ( test_pipeline/.* also works)

          Oleg Nenashev added a comment -

          As stated in https://groups.google.com/d/msg/jenkinsci-dev/uc6NsMoCFQI/AIO4WG1UCwAJ , I have no bandwidth to work on this issue anytime soon. I have marked the plugin for adoption. But I will be happy to review/release anything related to this issue while there is no other maintainer.

           

          Oleg Nenashev added a comment - As stated in https://groups.google.com/d/msg/jenkinsci-dev/uc6NsMoCFQI/AIO4WG1UCwAJ , I have no bandwidth to work on this issue anytime soon. I have marked the plugin for adoption. But I will be happy to review/release anything related to this issue while there is no other maintainer.  

            Unassigned Unassigned
            abayer Andrew Bayer
            Votes:
            11 Vote for this issue
            Watchers:
            17 Start watching this issue

              Created:
              Updated: