• Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Blocker Blocker
    • git-plugin
    • None

      We discoverd that parameters of one Pipeline job affect the parameters of other jobs. This means that in a job you have multiple parameters with the same name. A part of the parameters is from the current job the rest is defined in another (independent) job. (Both jobs should run simultaneously (but independent from each other) on the same Repository/PullRequest)

      Is this normal? Can I configure that parameters are only valid for one specific Job.

      Example: Two jobs have the Parameter "BranchSpecifier". The Parameter in Job A is set to "develop" and in Job B to "master". What is happening is that both Jobs now have two times the Parameter "BranchSpecifier" – once with value 'develop' and once 'master'.
      --> This is valid for all Parameters of both jobs (in both directions)

       

      Note: Job B in this example was created by copying Job A.
      Also the mentioned parameter in the example is used in the Job config as specifier for the branch the job will use to check out the Pipeline scripts from our Pipeline repo. This way we can trigger the job using pipeline scripts  from another branch (for tests).

      The effect of this bug is that both jobs will check out  the same branch and nt the one specified in the parameter.

          [JENKINS-45472] Job parameters appear in multiple jobs

          uday reddy added a comment -

          you can use parameterized trigger plugin will work for type of scenarios

          uday reddy added a comment - you can use parameterized trigger plugin will work for type of scenarios

          Hi uday reddy,

          may I get some more info of how to use the parametrized trigger plugin?

          What we want is to trigger several pipelines from one pull request. All of them shall return the build result to BitBucket. We think we would manage to trigger several builds but how do they return the build results to BitBucket?

          Michael Deckert added a comment - Hi uday reddy, may I get some more info of how to use the parametrized trigger plugin? What we want is to trigger several pipelines from one pull request. All of them shall return the build result to BitBucket. We think we would manage to trigger several builds but how do they return the build results to BitBucket?

          Daniel Beck added a comment -

          Is this a declarative pipeline?

          abayer PTAL – perhaps fallout from your "properties defined in the job" feature?

          Daniel Beck added a comment - Is this a declarative pipeline? abayer PTAL – perhaps fallout from your "properties defined in the job" feature?

          Andrew Bayer added a comment -

          Can you include the Jenkinsfiles and ideally the jobs' config.xmls, please?

          Andrew Bayer added a comment - Can you include the Jenkinsfiles and ideally the jobs' config.xmls, please?

          danielbeck No, we do not use the declarative syntax (yet)

           

          Here are the (slightly adapted) Config files.

          configOutput.xml

          configOutput2.xml

          Christian Ditscher added a comment - danielbeck No, we do not use the declarative syntax (yet)   Here are the (slightly adapted) Config files. configOutput.xml configOutput2.xml

          Daniel Beck added a comment -

          This needs a minimal reproduction case with complete, unambiguous and detailed steps to reproduce. The description hints at a nontrivial setup (I've tried to get close, but it's a PITA).

          Daniel Beck added a comment - This needs a minimal reproduction case with complete, unambiguous and detailed steps to reproduce. The description hints at a nontrivial setup (I've tried to get close, but it's a PITA).

          Mateusz Tarnawa added a comment - - edited

          Hello danielbeck, I can provide the quick steps to reproduce this behavior although the context is bit different.

          Prerequisites:

          • Git repository with a branch and a commit (to reference sha)
          • Jenkins 2.46.3
          1. Have two jobs (e.g. BAR1-job and BAR2-job) that poll the same Git repository and branch with Poll SCM step enabled (to leverage /notifyCommit GET request)
          2. Set "This build is parametrized" in BAR1-job and provide e.g. a string parameter FOO with default value BAR1
          3. For BAR2-job do FOO parameter with default value BAR2

          When doing

          curl "https://localhost:8000/git/notifyCommit?url=ssh://something.com/repo.git&branches=dev&sha1=3d0e8b6cc855104c7329d6bb96c653d49c60ae71
          

          You will end up with both jobs being triggered, each showing two parameters called FOO in PARAMETERS. The problem occurs because when looking at env variables they do not always correspond to their jobs - so you end up with BAR1-job having FOO=BAR2 or BAR2-job having FOO=BAR1.

          This does not occur when hook has no sha1 parameter. So the following:

          curl "https://localhost:8000/git/notifyCommit?url=ssh://something.com/repo.git&branches=dev

          will trigger two jobs with correct parameters. BAR1-job with FOO=BAR1 and BAR2-job with FOO=BAR2.

          This happens for Jenkins 2.46.3. It will not happen if parameters have different name of course e.g. FOO1, FOO2.

          Mateusz Tarnawa added a comment - - edited Hello danielbeck , I can provide the quick steps to reproduce this behavior although the context is bit different. Prerequisites: Git repository with a branch and a commit (to reference sha) Jenkins 2.46.3 Have two jobs (e.g. BAR1-job and BAR2-job) that poll the same Git repository and branch with Poll SCM step enabled (to leverage /notifyCommit GET request) Set "This build is parametrized" in BAR1-job and provide e.g. a string parameter FOO with default value BAR1 For BAR2-job do FOO parameter with default value BAR2 When doing curl "https: //localhost:8000/git/notifyCommit?url=ssh://something.com/repo.git&branches=dev&sha1=3d0e8b6cc855104c7329d6bb96c653d49c60ae71 You will end up with both jobs being triggered, each showing two parameters called FOO in PARAMETERS. The problem occurs because when looking at env variables they do not always correspond to their jobs - so you end up with BAR1-job having FOO=BAR2 or BAR2-job having FOO=BAR1. This does not occur when hook has no sha1 parameter. So the following: curl "https: //localhost:8000/git/notifyCommit?url=ssh://something.com/repo.git&branches=dev will trigger two jobs with correct parameters. BAR1-job with FOO=BAR1 and BAR2-job with FOO=BAR2. This happens for Jenkins 2.46.3. It will not happen if parameters have different name of course e.g. FOO1, FOO2.

          Daniel Beck added a comment -

          I think I found the culprit for this. Untested, just read the code.

          Git Plugin passes a List of job parameters to the ParametersAction of the queue item:

          https://github.com/jenkinsci/git-plugin/blob/f58648e9005293ab07b2389212603ff9a460b80a/src/main/java/hudson/plugins/git/GitStatus.java#L436

          The problem is that this list is mutated for every job this is iterating over, so the second job will likely also get the first job's parameters (ultimately probably prevented by the fix for SECURITY-170 in core 1.651.2+, but still – also, Git Plugin still depends on 1.625.x):

          https://github.com/jenkinsci/git-plugin/blob/f58648e9005293ab07b2389212603ff9a460b80a/src/main/java/hudson/plugins/git/GitStatus.java#L424

          On top of that, core stupidly just uses whatever list is passed instead of copying it, which I assume causes the problems observed here. Queue the first job, override its args with the next iteration…

          https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/ParametersAction.java#L103

          Pinging markewaite who is probably better equipped to investigate Git Plugin issues than me.

          Daniel Beck added a comment - I think I found the culprit for this. Untested, just read the code. Git Plugin passes a List of job parameters to the ParametersAction of the queue item: https://github.com/jenkinsci/git-plugin/blob/f58648e9005293ab07b2389212603ff9a460b80a/src/main/java/hudson/plugins/git/GitStatus.java#L436 The problem is that this list is mutated for every job this is iterating over, so the second job will likely also get the first job's parameters (ultimately probably prevented by the fix for SECURITY-170 in core 1.651.2+, but still – also, Git Plugin still depends on 1.625.x): https://github.com/jenkinsci/git-plugin/blob/f58648e9005293ab07b2389212603ff9a460b80a/src/main/java/hudson/plugins/git/GitStatus.java#L424 On top of that, core stupidly just uses whatever list is passed instead of copying it, which I assume causes the problems observed here. Queue the first job, override its args with the next iteration… https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/ParametersAction.java#L103 Pinging markewaite who is probably better equipped to investigate Git Plugin issues than me.

          Daniel Beck added a comment -

          Core PR up (without tests) at https://github.com/jenkinsci/jenkins/pull/3028

          Would probably be a good idea to get the above steps tried out with the PR build if someone has the time.

          Daniel Beck added a comment - Core PR up (without tests) at https://github.com/jenkinsci/jenkins/pull/3028 Would probably be a good idea to get the above steps tried out with the PR build if someone has the time.

          Mark Waite added a comment -

          I believe the key statement is in the comment from mtarnawa which says:

          This does not occur when hook has no sha1 parameter.

          When a sha1 parameter is passed, I believe the git plugin (incorrectly) treats that as definitive, and will attempt to build that specific SHA1 on every job which is monitoring that URL.  Don't pass the sha1 parameter to the notifyCommit.  Let the plugin determine (for the branches it is monitoring) if any change has happened.

          I haven't investigated the details of the environment variables leakage which danielbeck has detected.

          Mark Waite added a comment - I believe the key statement is in the comment from mtarnawa which says: This does not occur when hook has no sha1 parameter. When a sha1 parameter is passed, I believe the git plugin (incorrectly) treats that as definitive, and will attempt to build that specific SHA1 on every job which is monitoring that URL.  Don't pass the sha1 parameter to the notifyCommit.  Let the plugin determine (for the branches it is monitoring) if any change has happened. I haven't investigated the details of the environment variables leakage which danielbeck has detected.

          Daniel Beck added a comment -

          mtarnawa Could you test the core PR build from my previous comment to see whether that resolves the issue?

          Daniel Beck added a comment - mtarnawa Could you test the core PR build from my previous comment to see whether that resolves the issue?

          Mateusz Tarnawa added a comment - - edited

          Hi danielbeck - I've commented in the PR. Unfortunately this doesn't fully work.

           

          markewaite - unfortunately skipping sha1 doesn't cut it . We use Bitbucket Webhook which requires sha1 to be present if the build is being triggered from the PR level. We skipped sha1 passing for build on commits, but that prevents us from rebuilding PRs.

          Mateusz Tarnawa added a comment - - edited Hi danielbeck - I've commented in the PR. Unfortunately this doesn't fully work.   markewaite - unfortunately skipping sha1 doesn't cut it . We use Bitbucket Webhook which requires sha1 to be present if the build is being triggered from the PR level. We skipped sha1 passing for build on commits, but that prevents us from rebuilding PRs.

          Daniel Beck added a comment -

          mtarnawa Well, it resolves half the issue, sort of, and you confirmed that

          the second job will likely also get the first job's parameters

          since Git Plugin accumulates parameter values in the same list.

          Daniel Beck added a comment - mtarnawa Well, it resolves half the issue, sort of, and you confirmed that the second job will likely also get the first job's parameters since Git Plugin accumulates parameter values in the same list.

          Mateusz Tarnawa added a comment - - edited

          Thanks, got it. Should I raise a separate bug for the first problem you mentioned in the PR?

          It accumulates all builds' parameters as the list doesn't get cleared between build triggers.

          Mateusz Tarnawa added a comment - - edited Thanks, got it. Should I raise a separate bug for the first problem you mentioned in the PR? It accumulates all builds' parameters as the list doesn't get cleared between build triggers.

          Daniel Beck added a comment -

          I think this issue is appropriate for that behavior.

          My change to core just sanitizes some questionable behavior, but a fix in Git Plugin is needed anyway.

          If anything, my core PR could use a separate issue, but as it's just a form of robustness hardening, that's not strictly needed. I can just point here for motivation.

          Daniel Beck added a comment - I think this issue is appropriate for that behavior. My change to core just sanitizes some questionable behavior, but a fix in Git Plugin is needed anyway. If anything, my core PR could use a separate issue, but as it's just a form of robustness hardening, that's not strictly needed. I can just point here for motivation.

          Code changed in jenkins
          User: Daniel Beck
          Path:
          core/src/main/java/hudson/model/ParametersAction.java
          http://jenkins-ci.org/commit/jenkins/e044675e15c2303a7a3c9a391f0c08b2ac97dc78
          Log:
          JENKINS-45472 Copy the list in case it's reused elsewhere

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Daniel Beck Path: core/src/main/java/hudson/model/ParametersAction.java http://jenkins-ci.org/commit/jenkins/e044675e15c2303a7a3c9a391f0c08b2ac97dc78 Log: JENKINS-45472 Copy the list in case it's reused elsewhere

          Code changed in jenkins
          User: Daniel Beck
          Path:
          test/src/test/java/hudson/model/ParametersAction2Test.java
          http://jenkins-ci.org/commit/jenkins/9c939c4eb421f2ab08f27bfef10c1754d5295ab7
          Log:
          JENKINS-45472 Add test

          Without the fix, test fails in line 308 with:

          org.junit.ComparisonFailure:
          Expected :for p2
          Actual :for p1

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Daniel Beck Path: test/src/test/java/hudson/model/ParametersAction2Test.java http://jenkins-ci.org/commit/jenkins/9c939c4eb421f2ab08f27bfef10c1754d5295ab7 Log: JENKINS-45472 Add test Without the fix, test fails in line 308 with: org.junit.ComparisonFailure: Expected :for p2 Actual :for p1

          Code changed in jenkins
          User: Oleg Nenashev
          Path:
          core/src/main/java/hudson/model/ParametersAction.java
          test/src/test/java/hudson/model/ParametersAction2Test.java
          http://jenkins-ci.org/commit/jenkins/fbaa059cf0a842a107858cc39f8df5e8795858a7
          Log:
          Merge pull request #3028 from daniel-beck/JENKINS-45472

          JENKINS-45472 Copy the list in case it's reused elsewhere

          Compare: https://github.com/jenkinsci/jenkins/compare/f2ac41e975cb...fbaa059cf0a8

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: core/src/main/java/hudson/model/ParametersAction.java test/src/test/java/hudson/model/ParametersAction2Test.java http://jenkins-ci.org/commit/jenkins/fbaa059cf0a842a107858cc39f8df5e8795858a7 Log: Merge pull request #3028 from daniel-beck/ JENKINS-45472 JENKINS-45472 Copy the list in case it's reused elsewhere Compare: https://github.com/jenkinsci/jenkins/compare/f2ac41e975cb...fbaa059cf0a8

          Daniel Beck added a comment -

          Not fixed by the core change referenced above, needs a fix in Git Plugin.

          Daniel Beck added a comment - Not fixed by the core change referenced above, needs a fix in Git Plugin.

          danielbeck markewaite

          Sorry to follow-up, but is this issue somewhere in the pipeline for any upcoming Jenkins/GitPlugin release? Thanks.

          Mateusz Tarnawa added a comment - danielbeck markewaite Sorry to follow-up, but is this issue somewhere in the pipeline for any upcoming Jenkins/GitPlugin release? Thanks.

          Mark Waite added a comment -

          This is not planned for any upcoming git plugin release.  My weekends and evenings have been focused on JENKINS-45447 and JENKINS-47169.  You're welcome to submit a pull request to confirm the problem with tests and to propose a fix for the problem.

          Mark Waite added a comment - This is not planned for any upcoming git plugin release.  My weekends and evenings have been focused on JENKINS-45447 and JENKINS-47169 .  You're welcome to submit a pull request to confirm the problem with tests and to propose a fix for the problem.

            Unassigned Unassigned
            ditschi Christian Ditscher
            Votes:
            4 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated: