Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-34377

job-restrictions Job Property does not support Pipeline Plugin

    XMLWordPrintable

Details

    Description

      job-restrictions-plugin does not support Pipeline Plugin (aka Workflow plugin) and it's behavior is not as expected.

      Steps to reproduce it:

      Create a job using Pipeline Plugin (https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Plugin)
      Create a slave in your master called for example 'demo-slave'.

      In your Groovy script using pipeline do:

      node('master') {

      }

      node('demo-slave') {

      }

      After in 'demo-slave' configuration, restrict it:

      No matter if you start your job with 'daniel_hernandez' user or other user, the job will not get a executor from the demo-slave node getting the message:

      Waiting for next available executor on demo-slave

      After, You can release it setting the value to 'Take any job'

      Attachments

        1. Captura.PNG
          Captura.PNG
          18 kB
        2. job-restrictions.hpi
          62 kB
        3. screenshot-1.png
          screenshot-1.png
          5 kB

        Issue Links

          Activity

            oleg_nenashev Oleg Nenashev added a comment -

            Could you please describe your use-case?
            Or just attach the data from the private e-mail you've sent

            oleg_nenashev Oleg Nenashev added a comment - Could you please describe your use-case? Or just attach the data from the private e-mail you've sent

            I have described the case

            daniel_alejandro_hc Daniel Hernandez added a comment - I have described the case

            Currently protect the node can be do it by the 'input' step of Pipeline plugin but the security it is developed by the script, provide node protection by job-restriction will provide too much value to your plugin because since few months ago Cloudbees release the Stage View and Pipeline plugin most of the developer will migrate their jobs using these very useful plugins

            daniel_alejandro_hc Daniel Hernandez added a comment - Currently protect the node can be do it by the 'input' step of Pipeline plugin but the security it is developed by the script, provide node protection by job-restriction will provide too much value to your plugin because since few months ago Cloudbees release the Stage View and Pipeline plugin most of the developer will migrate their jobs using these very useful plugins
            domruf Dominik Ruf added a comment -

            I looked at at code (AbstractUserCauseRestriction.java canTake(Queue.BuildableItem item))
            and the problem is this :
            In case the task of the BuildableItem object is of type org.jenkinsci.plugins.workflow.support.steps.ExecutorStepExecution getActions() returns an empty list.

            AFAIS this problem can not be solved without changes in the workflow/pipeline plugin.

            domruf Dominik Ruf added a comment - I looked at at code (AbstractUserCauseRestriction.java canTake(Queue.BuildableItem item)) and the problem is this : In case the task of the BuildableItem object is of type org.jenkinsci.plugins.workflow.support.steps.ExecutorStepExecution getActions() returns an empty list. AFAIS this problem can not be solved without changes in the workflow/pipeline plugin.
            oleg_nenashev Oleg Nenashev added a comment -

            domruf Actually this change can be solved by more or less hackish code in the plugin. Maybe upstream user retrieval works out of the box, but I'm not 100% sure.

            oleg_nenashev Oleg Nenashev added a comment - domruf Actually this change can be solved by more or less hackish code in the plugin. Maybe upstream user retrieval works out of the box, but I'm not 100% sure.
            domruf Dominik Ruf added a comment - - edited

            oleg_nenashev Okay then lead the way.
            Because I have no idea how this hackish code would work

            domruf Dominik Ruf added a comment - - edited oleg_nenashev Okay then lead the way. Because I have no idea how this hackish code would work
            daniel_alejandro_hc Daniel Hernandez added a comment - - edited

            Handle the users security from Jenkins configuration and forget to verify the security in the "input" step would be a great step

            daniel_alejandro_hc Daniel Hernandez added a comment - - edited Handle the users security from Jenkins configuration and forget to verify the security in the "input" step would be a great step
            mahler Gustav Mahler added a comment -

            I am planning to use to Jenkins EC2 Container Service Plugin in combination with my project that uses the Pipeline Plugin. This will not work, as the job needs to be restricted to the labels used in the ECS Slave Template configuration.

            mahler Gustav Mahler added a comment - I am planning to use to Jenkins EC2 Container Service Plugin in combination with my project that uses the Pipeline Plugin. This will not work, as the job needs to be restricted to the labels used in the ECS Slave Template configuration.
            oleg_nenashev Oleg Nenashev added a comment -

            So the issue is still open. According to my past experience with ExecutorStepExecution, it is possible to add an ugly hack to this plugin without patching Pipeline. Still needs to be done by somebody.

            mahler how is it related to this issue? For me it seems rather like an RFE to EC2CS Plugin for implementing a custom JobRestriction (this is an extension point in this plugin)

            oleg_nenashev Oleg Nenashev added a comment - So the issue is still open. According to my past experience with ExecutorStepExecution, it is possible to add an ugly hack to this plugin without patching Pipeline. Still needs to be done by somebody. mahler how is it related to this issue? For me it seems rather like an RFE to EC2CS Plugin for implementing a custom JobRestriction (this is an extension point in this plugin)
            mansoor Mansoor Saqib added a comment -

            oleg_nenashev Firstly, thank you for your great work in creating this excellent plugin. We are currently working on a project and found this great plugin, however since we're also using pipeline jobs we have encountered this same issue that daniel_alejandro_hc has described.

            We would really like to use this plugin with pipeline jobs and as such, I'd like to offer any help I can in implementing this feature. oleg_nenashev I know you described in earlier comments that there can be two solutions: with and without patching pipeline. Would you be able to offer any advice on how to go about implementing this fix directly in the Job-Restrictions-Plugin without patching pipeline?

            Thank you

            mansoor Mansoor Saqib added a comment - oleg_nenashev Firstly, thank you for your great work in creating this excellent plugin. We are currently working on a project and found this great plugin, however since we're also using pipeline jobs we have encountered this same issue that daniel_alejandro_hc has described. We would really like to use this plugin with pipeline jobs and as such, I'd like to offer any help I can in implementing this feature. oleg_nenashev I know you described in earlier comments that there can be two solutions: with and without patching pipeline. Would you be able to offer any advice on how to go about implementing this fix directly in the Job-Restrictions-Plugin without patching pipeline? Thank you
            oleg_nenashev Oleg Nenashev added a comment -

            mansoor Hi, sorry for the response delay. I am traveling this week, but I will do my best to respond within a couple of days.

            oleg_nenashev Oleg Nenashev added a comment - mansoor Hi, sorry for the response delay. I am traveling this week, but I will do my best to respond within a couple of days.
            mansoor Mansoor Saqib added a comment -

            oleg_nenashev No problem, thank you for the update, looking forward to it. Safe travels

            mansoor Mansoor Saqib added a comment - oleg_nenashev No problem, thank you for the update, looking forward to it. Safe travels
            mansoor Mansoor Saqib added a comment -

            Hello oleg_nenashev , hope all is well. I have implemented a fix and created a pull request here https://github.com/jenkinsci/job-restrictions-plugin/pull/16

            I would appreciate it if anyone can review it daniel_alejandro_hc mahler domruf.

            Thank you.

            mansoor Mansoor Saqib added a comment - Hello oleg_nenashev , hope all is well. I have implemented a fix and created a pull request here https://github.com/jenkinsci/job-restrictions-plugin/pull/16 I would appreciate it if anyone can review it daniel_alejandro_hc mahler domruf . Thank you.

            Hi mansoor would you send it to me the .hpi file? I would test it. My mail is danielahcardona@gmail.com.

            Thanks !

            danielahcardona Daniel Alejandro Hernández added a comment - Hi mansoor would you send it to me the .hpi file? I would test it. My mail is danielahcardona@gmail.com. Thanks !
            mansoor Mansoor Saqib added a comment -

            job-restrictions.hpi

            Hello daniel_alejandro_hc , I've attached the hpi file. Let me know if you can access it. Thanks for the help.

            mansoor Mansoor Saqib added a comment - job-restrictions.hpi Hello daniel_alejandro_hc , I've attached the hpi file. Let me know if you can access it. Thanks for the help.

            Thanks mansoor ! I will try to have some time this weekend to give it a try ! This must be very helpful I would still need it.

            daniel_alejandro_hc Daniel Hernandez added a comment - Thanks mansoor ! I will try to have some time this weekend to give it a try ! This must be very helpful I would still need it.

            Hi mansoor I have give a try to the plugin and I have a couple of comments.

            Not sure if Jenkins 2.60.3 would be mandatory of some lowers versions, I had to upgrade my Jenkins from 2.60.1, but his is somehow out of current topic.

            Regarding the scenario and to me it seems it is working fine the "to be" scenario on the other hand if you run the Job with a user that has no permits on the Agent/Node and you run a Job there it doesn't start you just get:

            "pending - waiting for next available executor"

            I would say ideally you should get an exception, for instance Job must start and just throw an error saying that the user has no grant to run Jobs over the Node.

            danielahcardona Daniel Alejandro Hernández added a comment - Hi mansoor I have give a try to the plugin and I have a couple of comments. Not sure if Jenkins 2.60.3 would be mandatory of some lowers versions, I had to upgrade my Jenkins from 2.60.1, but his is somehow out of current topic. Regarding the scenario and to me it seems it is working fine the "to be" scenario on the other hand if you run the Job with a user that has no permits on the Agent/Node and you run a Job there it doesn't start you just get: "pending - waiting for next available executor" I would say ideally you should get an exception, for instance Job must start and just throw an error saying that the user has no grant to run Jobs over the Node.
            mansoor Mansoor Saqib added a comment -

            job-restrictions.hpi

            Hi daniel_alejandro_hc thank you for the feedback, I appreciate it. As per oleg_nenashev 's pull request review I've lowered the required Jenkins version to 1.651.3 and have fixed some NPE issues. I've uploaded the new version of the hpi file for you.

            I'm glad that the scenario that led to this bug report is working, please let me know if you encounter any other problems. As for the "to be" scenario you've described, I can definitely look into creating a checkbox option on the restriction configuration screen, something like "cancel jobs not meeting this restriction". Let me know what you think of this idea, it could be an added feature.

            For now, I like that anyone else besides the restricted user can run jobs and they are queued instead of being cancelled.

            Thanks

            mansoor Mansoor Saqib added a comment - job-restrictions.hpi Hi daniel_alejandro_hc thank you for the feedback, I appreciate it. As per oleg_nenashev 's pull request review I've lowered the required Jenkins version to 1.651.3 and have fixed some NPE issues. I've uploaded the new version of the hpi file for you. I'm glad that the scenario that led to this bug report is working, please let me know if you encounter any other problems. As for the "to be" scenario you've described, I can definitely look into creating a checkbox option on the restriction configuration screen, something like "cancel jobs not meeting this restriction". Let me know what you think of this idea, it could be an added feature. For now, I like that anyone else besides the restricted user can run jobs and they are queued instead of being cancelled. Thanks

            Hi !, sorry I couldn't replied quick. Regarding this "cancel jobs not meeting this restriction" sounds great, I just wonder oleg_nenashev mansoor why this behavior cant be as default? What would be the reason to hang the process? If the user can't run jobs in that Node and then it hangs , how this job will be released?

            I hope to test it again by this weekend.

            Thanks !

            danielahcardona Daniel Alejandro Hernández added a comment - Hi !, sorry I couldn't replied quick. Regarding this "cancel jobs not meeting this restriction" sounds great, I just wonder oleg_nenashev mansoor why this behavior cant be as default? What would be the reason to hang the process? If the user can't run jobs in that Node and then it hangs , how this job will be released? I hope to test it again by this weekend. Thanks !

            Hi oleg_nenashev mansoor I have give it a try and it still working fine the expected scenario.

            Just a few comments with a common scenario:

            When you run a job with a user that has no grant over the Node to run jobs there so the job will be queued or stuck with the label "pending - waiting for next available executor".

            Then if a different user try to run the same job but has the correct grants further job will not run until manually stop or cancel previous job and then trigger the new ones. I did check that the job can run concurrent builds and the Node has enough executors and regardless that the new builds cannot run until manually release the one that is "pending - waiting for next available executor" status.

            So with this miss behaviors can arise, for instance if there a job with the status "pending - waiting for next available executor" and the job it is triggered with some webhook after pushing changes to GIT Developers may not notice that theirs expected jobs will not run until they manually check the job and release or cancel the one give them issues.

            danielahcardona Daniel Alejandro Hernández added a comment - Hi oleg_nenashev mansoor I have give it a try and it still working fine the expected scenario. Just a few comments with a common scenario: When you run a job with a user that has no grant over the Node to run jobs there so the job will be queued or stuck with the label "pending - waiting for next available executor". Then if a different user try to run the same job but has the correct grants further job will not run until manually stop or cancel previous job and then trigger the new ones. I did check that the job can run concurrent builds and the Node has enough executors and regardless that the new builds cannot run until manually release the one that is "pending - waiting for next available executor" status. So with this miss behaviors can arise, for instance if there a job with the status "pending - waiting for next available executor" and the job it is triggered with some webhook after pushing changes to GIT Developers may not notice that theirs expected jobs will not run until they manually check the job and release or cancel the one give them issues.
            mansoor Mansoor Saqib added a comment -

            Hello daniel_alejandro_hc ,

            Thank you for the reply. I don't know the reasons for the default behavior and oleg_nenashev could provide it, but for our purposes this works great when using it for pipeline jobs.

            I have tried the scenario that you've outlined above with freestyle type jobs and you're correct. When using freestyle jobs for example it does indeed hang if a non restricted user runs a build, it doesn't let a restricted user's builds through without first removing this hanging job from the previous user. I will try to come up with a solution but it may be a problem that is not within the scope of this current issue I believe. It may need another issue created for it that oleg_nenashev could possibly look at. However it is an interesting scenario and I'll be sure to try to provide some workaround or solution if I can.

            I appreciate the help and feedback daniel_alejandro_hc , thanks.

            mansoor Mansoor Saqib added a comment - Hello daniel_alejandro_hc , Thank you for the reply. I don't know the reasons for the default behavior and oleg_nenashev could provide it, but for our purposes this works great when using it for pipeline jobs. I have tried the scenario that you've outlined above with freestyle type jobs and you're correct. When using freestyle jobs for example it does indeed hang if a non restricted user runs a build, it doesn't let a restricted user's builds through without first removing this hanging job from the previous user. I will try to come up with a solution but it may be a problem that is not within the scope of this current issue I believe. It may need another issue created for it that oleg_nenashev could possibly look at. However it is an interesting scenario and I'll be sure to try to provide some workaround or solution if I can. I appreciate the help and feedback daniel_alejandro_hc , thanks.
            oleg_nenashev Oleg Nenashev added a comment -

            When I was working on the plugin, there was no valid way to cancel the job execution from Job Properties when there are no suitable executors. It becomes even more complicated with Cloud API plugins, because Jenkins NEVER knows which labels can be provided by Cloud plugins, and there are some plugins which really have dynamic lists of supported labels (private-source example: cloud for provisioning shared agents/clouds in CloudBees Jenkins Operations Center). I implemented the current behavior for starters, but there are real use-cases for the current behavior.

            Adding support of build termination is doable, but it likely has to be done by another plugin or the core itself. Job Restrictions are just a single case of a wider issue (e.g. Label Restrictions, Throttle Concurrent Builds, etc., etc.).

            oleg_nenashev Oleg Nenashev added a comment - When I was working on the plugin, there was no valid way to cancel the job execution from Job Properties when there are no suitable executors. It becomes even more complicated with Cloud API plugins, because Jenkins NEVER knows which labels can be provided by Cloud plugins, and there are some plugins which really have dynamic lists of supported labels (private-source example: cloud for provisioning shared agents/clouds in CloudBees Jenkins Operations Center). I implemented the current behavior for starters, but there are real use-cases for the current behavior. Adding support of build termination is doable, but it likely has to be done by another plugin or the core itself. Job Restrictions are just a single case of a wider issue (e.g. Label Restrictions, Throttle Concurrent Builds, etc., etc.).

            The last update was a year ago, any update? This is one key thing that is holding us back from pinning nodes to certain users so we don't have to use a power user to connect to the nodes and then restrict who can use the node.

            mtdeguzis Michael DeGuzis added a comment - The last update was a year ago, any update? This is one key thing that is holding us back from pinning nodes to certain users so we don't have to use a power user to connect to the nodes and then restrict who can use the node.
            oleg_nenashev Oleg Nenashev added a comment -

            No update. The described use-case can be rather done by Authorize Project and user roles  in matrix-auth/role-strategy

            oleg_nenashev Oleg Nenashev added a comment - No update. The described use-case can be rather done by Authorize Project and user roles  in matrix-auth/role-strategy

            Hmm ok, I will check into that. I was hoping to restrict this at the node level. I'm not entirely sure how to achieve this with that plugin, but maybe I missed something.

            mtdeguzis Michael DeGuzis added a comment - Hmm ok, I will check into that. I was hoping to restrict this at the node level. I'm not entirely sure how to achieve this with that plugin, but maybe I missed something.

            I don't really see a way to restrict nodes with Authorize Project. Any notes on that or a post that describes that?

            mtdeguzis Michael DeGuzis added a comment - I don't really see a way to restrict nodes with Authorize Project. Any notes on that or a post that describes that?

            I do not see the way how to do it in role-strategy as well. I need to restrict execution on certain Agents for certain people or groups and it is not possible to achieve it in standard authorisation method. In role-strategy I am able to not to allow build for certain group or people, but I want them to execute builds, but not on all agents

            lacot5 Ladislav Toldy added a comment - I do not see the way how to do it in role-strategy as well. I need to restrict execution on certain Agents for certain people or groups and it is not possible to achieve it in standard authorisation method. In role-strategy I am able to not to allow build for certain group or people, but I want them to execute builds, but not on all agents

            I have tried the version in attachment, job-restrictions.hpiand it works for my use case thanks mansoor, solving my problems (restrict execution only for certain groups). Build termination was not requested in this ticket, is there a vision of releasing this fix officially?

            lacot5 Ladislav Toldy added a comment - I have tried the version in attachment, job-restrictions.hpi and it works for my use case thanks mansoor , solving my problems (restrict execution only for certain groups). Build termination was not requested in this ticket, is there a vision of releasing this fix officially?

            People

              mansoor Mansoor Saqib
              daniel_alejandro_hc Daniel Hernandez
              Votes:
              21 Vote for this issue
              Watchers:
              25 Start watching this issue

              Dates

                Created:
                Updated: