Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-45472

Job parameters appear in multiple jobs

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Blocker
    • Resolution: Unresolved
    • Component/s: git-plugin
    • Labels:
      None
    • Similar Issues:

      Description

      We discoverd that parameters of one Pipeline job affect the parameters of other jobs. This means that in a job you have multiple parameters with the same name. A part of the parameters is from the current job the rest is defined in another (independent) job. (Both jobs should run simultaneously (but independent from each other) on the same Repository/PullRequest)

      Is this normal? Can I configure that parameters are only valid for one specific Job.

      Example: Two jobs have the Parameter "BranchSpecifier". The Parameter in Job A is set to "develop" and in Job B to "master". What is happening is that both Jobs now have two times the Parameter "BranchSpecifier" – once with value 'develop' and once 'master'.
      --> This is valid for all Parameters of both jobs (in both directions)

       

      Note: Job B in this example was created by copying Job A.
      Also the mentioned parameter in the example is used in the Job config as specifier for the branch the job will use to check out the Pipeline scripts from our Pipeline repo. This way we can trigger the job using pipeline scripts  from another branch (for tests).

      The effect of this bug is that both jobs will check out  the same branch and nt the one specified in the parameter.

        Attachments

          Issue Links

            Activity

            ditschi Christian Ditscher created issue -
            ditschi Christian Ditscher made changes -
            Field Original Value New Value
            Description We discoverd that parameters of one Pipeline job affect the parameters of other jobs. This means that in a job you have multiple parameters with the same name. A part of the parameters is from the current job the rest is defined in annother (independent) job.


             Is this normal? Can I configure that parameters are only valid for one specific Job.

            Example: Two jobs have the Parameter "BranchSpecifier". The Parameter in Job A is set to "develop" and in Job B to "master". What is happening is that both Jobs now have two times the Parameter "BranchSpecifier" – once with value 'develop' and once 'master'.
            --> This is valid for all Parameters of both jobs (in both directions)

             

            Note: Job B in this example was created by copying Job A.
            Also the mentioned parameter in the example is used in the Job config as specifier for the branch the job will use to check out the Pipeline scripts from our Pipeline repo. This way we can trigger the job using pipeline scripts  from another branch (for tests).

            The effect of this bug is that both jobs will check out  the same branch and nt the one specified in the parameter.
            We discoverd that parameters of one Pipeline job affect the parameters of other jobs. This means that in a job you have multiple parameters with the same name. A part of the parameters is from the current job the rest is defined in another (independent) job. (Both jobs should run simultaneously (but independent from each other) on the same Repository/PullRequest)

            Is this normal? Can I configure that parameters are only valid for one specific Job.

            Example: Two jobs have the Parameter "BranchSpecifier". The Parameter in Job A is set to "develop" and in Job B to "master". What is happening is that both Jobs now have two times the Parameter "BranchSpecifier" – once with value 'develop' and once 'master'.
             --> This is valid for all Parameters of both jobs (in both directions)

             

            Note: Job B in this example was created by copying Job A.
             Also the mentioned parameter in the example is used in the Job config as specifier for the branch the job will use to check out the Pipeline scripts from our Pipeline repo. This way we can trigger the job using pipeline scripts  from another branch (for tests).

            The effect of this bug is that both jobs will check out  the same branch and nt the one specified in the parameter.
            ditschi Christian Ditscher made changes -
            Priority Major [ 3 ] Blocker [ 1 ]
            Hide
            udaybandr uday reddy added a comment -

            you can use parameterized trigger plugin will work for type of scenarios

            Show
            udaybandr uday reddy added a comment - you can use parameterized trigger plugin will work for type of scenarios
            Hide
            michaeldeckert Michael Deckert added a comment -

            Hi uday reddy,

            may I get some more info of how to use the parametrized trigger plugin?

            What we want is to trigger several pipelines from one pull request. All of them shall return the build result to BitBucket. We think we would manage to trigger several builds but how do they return the build results to BitBucket?

            Show
            michaeldeckert Michael Deckert added a comment - Hi uday reddy, may I get some more info of how to use the parametrized trigger plugin? What we want is to trigger several pipelines from one pull request. All of them shall return the build result to BitBucket. We think we would manage to trigger several builds but how do they return the build results to BitBucket?
            Hide
            danielbeck Daniel Beck added a comment -

            Is this a declarative pipeline?

            Andrew Bayer PTAL – perhaps fallout from your "properties defined in the job" feature?

            Show
            danielbeck Daniel Beck added a comment - Is this a declarative pipeline? Andrew Bayer PTAL – perhaps fallout from your "properties defined in the job" feature?
            Hide
            abayer Andrew Bayer added a comment -

            Can you include the Jenkinsfiles and ideally the jobs' config.xmls, please?

            Show
            abayer Andrew Bayer added a comment - Can you include the Jenkinsfiles and ideally the jobs' config.xmls, please?
            ditschi Christian Ditscher made changes -
            Attachment configOutput.xml [ 39171 ]
            Attachment configOutput2.xml [ 39172 ]
            Hide
            ditschi Christian Ditscher added a comment -

            Daniel Beck No, we do not use the declarative syntax (yet)

             

            Here are the (slightly adapted) Config files.

            configOutput.xml

            configOutput2.xml

            Show
            ditschi Christian Ditscher added a comment - Daniel Beck No, we do not use the declarative syntax (yet)   Here are the (slightly adapted) Config files. configOutput.xml configOutput2.xml
            ditschi Christian Ditscher made changes -
            Attachment configOutput2.xml [ 39172 ]
            ditschi Christian Ditscher made changes -
            Attachment configOutput2.xml [ 39173 ]
            ditschi Christian Ditscher made changes -
            Attachment 2017-08-29_15h51_02.png [ 39532 ]
            Hide
            danielbeck Daniel Beck added a comment -

            This needs a minimal reproduction case with complete, unambiguous and detailed steps to reproduce. The description hints at a nontrivial setup (I've tried to get close, but it's a PITA).

            Show
            danielbeck Daniel Beck added a comment - This needs a minimal reproduction case with complete, unambiguous and detailed steps to reproduce. The description hints at a nontrivial setup (I've tried to get close, but it's a PITA).
            Hide
            mtarnawa Mateusz Tarnawa added a comment - - edited

            Hello Daniel Beck, I can provide the quick steps to reproduce this behavior although the context is bit different.

            Prerequisites:

            • Git repository with a branch and a commit (to reference sha)
            • Jenkins 2.46.3
            1. Have two jobs (e.g. BAR1-job and BAR2-job) that poll the same Git repository and branch with Poll SCM step enabled (to leverage /notifyCommit GET request)
            2. Set "This build is parametrized" in BAR1-job and provide e.g. a string parameter FOO with default value BAR1
            3. For BAR2-job do FOO parameter with default value BAR2

            When doing

            curl "https://localhost:8000/git/notifyCommit?url=ssh://something.com/repo.git&branches=dev&sha1=3d0e8b6cc855104c7329d6bb96c653d49c60ae71
            

            You will end up with both jobs being triggered, each showing two parameters called FOO in PARAMETERS. The problem occurs because when looking at env variables they do not always correspond to their jobs - so you end up with BAR1-job having FOO=BAR2 or BAR2-job having FOO=BAR1.

            This does not occur when hook has no sha1 parameter. So the following:

            curl "https://localhost:8000/git/notifyCommit?url=ssh://something.com/repo.git&branches=dev

            will trigger two jobs with correct parameters. BAR1-job with FOO=BAR1 and BAR2-job with FOO=BAR2.

            This happens for Jenkins 2.46.3. It will not happen if parameters have different name of course e.g. FOO1, FOO2.

            Show
            mtarnawa Mateusz Tarnawa added a comment - - edited Hello Daniel Beck , I can provide the quick steps to reproduce this behavior although the context is bit different. Prerequisites: Git repository with a branch and a commit (to reference sha) Jenkins 2.46.3 Have two jobs (e.g. BAR1-job and BAR2-job) that poll the same Git repository and branch with Poll SCM step enabled (to leverage /notifyCommit GET request) Set "This build is parametrized" in BAR1-job and provide e.g. a string parameter FOO with default value BAR1 For BAR2-job do FOO parameter with default value BAR2 When doing curl "https: //localhost:8000/git/notifyCommit?url=ssh://something.com/repo.git&branches=dev&sha1=3d0e8b6cc855104c7329d6bb96c653d49c60ae71 You will end up with both jobs being triggered, each showing two parameters called FOO in PARAMETERS. The problem occurs because when looking at env variables they do not always correspond to their jobs - so you end up with BAR1-job having FOO=BAR2 or BAR2-job having FOO=BAR1. This does not occur when hook has no sha1 parameter. So the following: curl "https: //localhost:8000/git/notifyCommit?url=ssh://something.com/repo.git&branches=dev will trigger two jobs with correct parameters. BAR1-job with FOO=BAR1 and BAR2-job with FOO=BAR2. This happens for Jenkins 2.46.3. It will not happen if parameters have different name of course e.g. FOO1, FOO2.
            danielbeck Daniel Beck made changes -
            Component/s git-plugin [ 15543 ]
            Hide
            danielbeck Daniel Beck added a comment -

            I think I found the culprit for this. Untested, just read the code.

            Git Plugin passes a List of job parameters to the ParametersAction of the queue item:

            https://github.com/jenkinsci/git-plugin/blob/f58648e9005293ab07b2389212603ff9a460b80a/src/main/java/hudson/plugins/git/GitStatus.java#L436

            The problem is that this list is mutated for every job this is iterating over, so the second job will likely also get the first job's parameters (ultimately probably prevented by the fix for SECURITY-170 in core 1.651.2+, but still – also, Git Plugin still depends on 1.625.x):

            https://github.com/jenkinsci/git-plugin/blob/f58648e9005293ab07b2389212603ff9a460b80a/src/main/java/hudson/plugins/git/GitStatus.java#L424

            On top of that, core stupidly just uses whatever list is passed instead of copying it, which I assume causes the problems observed here. Queue the first job, override its args with the next iteration…

            https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/ParametersAction.java#L103

            Pinging Mark Waite who is probably better equipped to investigate Git Plugin issues than me.

            Show
            danielbeck Daniel Beck added a comment - I think I found the culprit for this. Untested, just read the code. Git Plugin passes a List of job parameters to the ParametersAction of the queue item: https://github.com/jenkinsci/git-plugin/blob/f58648e9005293ab07b2389212603ff9a460b80a/src/main/java/hudson/plugins/git/GitStatus.java#L436 The problem is that this list is mutated for every job this is iterating over, so the second job will likely also get the first job's parameters (ultimately probably prevented by the fix for SECURITY-170 in core 1.651.2+, but still – also, Git Plugin still depends on 1.625.x): https://github.com/jenkinsci/git-plugin/blob/f58648e9005293ab07b2389212603ff9a460b80a/src/main/java/hudson/plugins/git/GitStatus.java#L424 On top of that, core stupidly just uses whatever list is passed instead of copying it, which I assume causes the problems observed here. Queue the first job, override its args with the next iteration… https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/ParametersAction.java#L103 Pinging Mark Waite who is probably better equipped to investigate Git Plugin issues than me.
            Hide
            danielbeck Daniel Beck added a comment -

            Core PR up (without tests) at https://github.com/jenkinsci/jenkins/pull/3028

            Would probably be a good idea to get the above steps tried out with the PR build if someone has the time.

            Show
            danielbeck Daniel Beck added a comment - Core PR up (without tests) at https://github.com/jenkinsci/jenkins/pull/3028 Would probably be a good idea to get the above steps tried out with the PR build if someone has the time.
            Hide
            markewaite Mark Waite added a comment -

            I believe the key statement is in the comment from Mateusz Tarnawa which says:

            This does not occur when hook has no sha1 parameter.

            When a sha1 parameter is passed, I believe the git plugin (incorrectly) treats that as definitive, and will attempt to build that specific SHA1 on every job which is monitoring that URL.  Don't pass the sha1 parameter to the notifyCommit.  Let the plugin determine (for the branches it is monitoring) if any change has happened.

            I haven't investigated the details of the environment variables leakage which Daniel Beck has detected.

            Show
            markewaite Mark Waite added a comment - I believe the key statement is in the comment from Mateusz Tarnawa which says: This does not occur when hook has no sha1 parameter. When a sha1 parameter is passed, I believe the git plugin (incorrectly) treats that as definitive, and will attempt to build that specific SHA1 on every job which is monitoring that URL.  Don't pass the sha1 parameter to the notifyCommit.  Let the plugin determine (for the branches it is monitoring) if any change has happened. I haven't investigated the details of the environment variables leakage which Daniel Beck has detected.
            Hide
            danielbeck Daniel Beck added a comment -

            Mateusz Tarnawa Could you test the core PR build from my previous comment to see whether that resolves the issue?

            Show
            danielbeck Daniel Beck added a comment - Mateusz Tarnawa Could you test the core PR build from my previous comment to see whether that resolves the issue?
            Hide
            mtarnawa Mateusz Tarnawa added a comment - - edited

            Hi Daniel Beck - I've commented in the PR. Unfortunately this doesn't fully work.

             

            Mark Waite - unfortunately skipping sha1 doesn't cut it . We use Bitbucket Webhook which requires sha1 to be present if the build is being triggered from the PR level. We skipped sha1 passing for build on commits, but that prevents us from rebuilding PRs.

            Show
            mtarnawa Mateusz Tarnawa added a comment - - edited Hi Daniel Beck - I've commented in the PR. Unfortunately this doesn't fully work.   Mark Waite - unfortunately skipping sha1 doesn't cut it . We use Bitbucket Webhook which requires sha1 to be present if the build is being triggered from the PR level. We skipped sha1 passing for build on commits, but that prevents us from rebuilding PRs.
            Hide
            danielbeck Daniel Beck added a comment -

            Mateusz Tarnawa Well, it resolves half the issue, sort of, and you confirmed that

            the second job will likely also get the first job's parameters

            since Git Plugin accumulates parameter values in the same list.

            Show
            danielbeck Daniel Beck added a comment - Mateusz Tarnawa Well, it resolves half the issue, sort of, and you confirmed that the second job will likely also get the first job's parameters since Git Plugin accumulates parameter values in the same list.
            Hide
            mtarnawa Mateusz Tarnawa added a comment - - edited

            Thanks, got it. Should I raise a separate bug for the first problem you mentioned in the PR?

            It accumulates all builds' parameters as the list doesn't get cleared between build triggers.

            Show
            mtarnawa Mateusz Tarnawa added a comment - - edited Thanks, got it. Should I raise a separate bug for the first problem you mentioned in the PR? It accumulates all builds' parameters as the list doesn't get cleared between build triggers.
            Hide
            danielbeck Daniel Beck added a comment -

            I think this issue is appropriate for that behavior.

            My change to core just sanitizes some questionable behavior, but a fix in Git Plugin is needed anyway.

            If anything, my core PR could use a separate issue, but as it's just a form of robustness hardening, that's not strictly needed. I can just point here for motivation.

            Show
            danielbeck Daniel Beck added a comment - I think this issue is appropriate for that behavior. My change to core just sanitizes some questionable behavior, but a fix in Git Plugin is needed anyway. If anything, my core PR could use a separate issue, but as it's just a form of robustness hardening, that's not strictly needed. I can just point here for motivation.
            danielbeck Daniel Beck made changes -
            Component/s core [ 15593 ]
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Daniel Beck
            Path:
            core/src/main/java/hudson/model/ParametersAction.java
            http://jenkins-ci.org/commit/jenkins/e044675e15c2303a7a3c9a391f0c08b2ac97dc78
            Log:
            JENKINS-45472 Copy the list in case it's reused elsewhere

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Daniel Beck Path: core/src/main/java/hudson/model/ParametersAction.java http://jenkins-ci.org/commit/jenkins/e044675e15c2303a7a3c9a391f0c08b2ac97dc78 Log: JENKINS-45472 Copy the list in case it's reused elsewhere
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Daniel Beck
            Path:
            test/src/test/java/hudson/model/ParametersAction2Test.java
            http://jenkins-ci.org/commit/jenkins/9c939c4eb421f2ab08f27bfef10c1754d5295ab7
            Log:
            JENKINS-45472 Add test

            Without the fix, test fails in line 308 with:

            org.junit.ComparisonFailure:
            Expected :for p2
            Actual :for p1

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Daniel Beck Path: test/src/test/java/hudson/model/ParametersAction2Test.java http://jenkins-ci.org/commit/jenkins/9c939c4eb421f2ab08f27bfef10c1754d5295ab7 Log: JENKINS-45472 Add test Without the fix, test fails in line 308 with: org.junit.ComparisonFailure: Expected :for p2 Actual :for p1
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Oleg Nenashev
            Path:
            core/src/main/java/hudson/model/ParametersAction.java
            test/src/test/java/hudson/model/ParametersAction2Test.java
            http://jenkins-ci.org/commit/jenkins/fbaa059cf0a842a107858cc39f8df5e8795858a7
            Log:
            Merge pull request #3028 from daniel-beck/JENKINS-45472

            JENKINS-45472 Copy the list in case it's reused elsewhere

            Compare: https://github.com/jenkinsci/jenkins/compare/f2ac41e975cb...fbaa059cf0a8

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: core/src/main/java/hudson/model/ParametersAction.java test/src/test/java/hudson/model/ParametersAction2Test.java http://jenkins-ci.org/commit/jenkins/fbaa059cf0a842a107858cc39f8df5e8795858a7 Log: Merge pull request #3028 from daniel-beck/ JENKINS-45472 JENKINS-45472 Copy the list in case it's reused elsewhere Compare: https://github.com/jenkinsci/jenkins/compare/f2ac41e975cb...fbaa059cf0a8
            Hide
            danielbeck Daniel Beck added a comment -

            Not fixed by the core change referenced above, needs a fix in Git Plugin.

            Show
            danielbeck Daniel Beck added a comment - Not fixed by the core change referenced above, needs a fix in Git Plugin.
            Hide
            mtarnawa Mateusz Tarnawa added a comment -

            Daniel Beck Mark Waite

            Sorry to follow-up, but is this issue somewhere in the pipeline for any upcoming Jenkins/GitPlugin release? Thanks.

            Show
            mtarnawa Mateusz Tarnawa added a comment - Daniel Beck Mark Waite Sorry to follow-up, but is this issue somewhere in the pipeline for any upcoming Jenkins/GitPlugin release? Thanks.
            Hide
            markewaite Mark Waite added a comment -

            This is not planned for any upcoming git plugin release.  My weekends and evenings have been focused on JENKINS-45447 and JENKINS-47169.  You're welcome to submit a pull request to confirm the problem with tests and to propose a fix for the problem.

            Show
            markewaite Mark Waite added a comment - This is not planned for any upcoming git plugin release.  My weekends and evenings have been focused on JENKINS-45447 and JENKINS-47169 .  You're welcome to submit a pull request to confirm the problem with tests and to propose a fix for the problem.
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-39495 [ JENKINS-39495 ]

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              ditschi Christian Ditscher
              Votes:
              4 Vote for this issue
              Watchers:
              10 Start watching this issue

                Dates

                Created:
                Updated: