Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-29765

Build Retry (both Naginator and Rebuild) pick the latest commit, not retrying one

    XMLWordPrintable

    Details

    • Type: New Feature
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Labels:
      None
    • Environment:
    • Similar Issues:

      Description

      This is easily reproduced on a standalone Cent 6 or Cent 7 machine.

      Create a job:

      • Uses Git
      • A trivial build step that fails, "false" or "exit 1" in the shell build-step for example
      • Uses one of the above retry mechanisms (Naginator or Rebuilder)
      • In the case of Naginator, stub it with a regex to keep it from automatically retrying
      • In the case of Rebuilder, enable the rebuild without parameters
      • Make a commit and run a build. It should fail. Note Git SHA (This is build #1)
      • Make a second commit, then build it. It should also fail. Note the Git SHA (This is build #2)
      • Now go back to build #1 and retrigger it. (build #3)

      Expectation:

      The retriggered build (build #3) should use the same Git SHA as build #1.

      Observed problem:

      The Git SHA is the HEAD of the branch at the time of the retrigger.

        Attachments

          Activity

          eegah Eric Griswold created issue -
          eegah Eric Griswold made changes -
          Field Original Value New Value
          Description This is easily reproduced on a standalone Cent 6 or Cent 7 machine.

          Create a job:
          * Uses Git
          * A trivial build step that fails, "false" or "exit 1" in the shell build-step for example
          * Uses one of the above retry mechanisms (Naginator or Rebuilder)
          * In the case of Naginator, stub it with a regex to keep it from automatically retrying
          * In the case of Rebuilder, enable the rebuild without parameters
          * Commit and run a build. It should fail. Note Git SHA (This is build #1)
          * Make a second build and run it. It should also fail. Note the Git SHA (This is build #2)
          * Now go back to build #1 and retrigger it.

          My expectation is that the retriggered build should have the same Git SHA as build #1. When I try it, the Git SHA is the HEAD of the branch at the time of the retrigger.
          This is easily reproduced on a standalone Cent 6 or Cent 7 machine.

          Create a job:
          * Uses Git
          * A trivial build step that fails, "false" or "exit 1" in the shell build-step for example
          * Uses one of the above retry mechanisms (Naginator or Rebuilder)
          * In the case of Naginator, stub it with a regex to keep it from automatically retrying
          * In the case of Rebuilder, enable the rebuild without parameters
          * Commit and run a build. It should fail. Note Git SHA (This is build #1)
          * Make a second build and run it. It should also fail. Note the Git SHA (This is build #2)
          * Now go back to build #1 and retrigger it. (build #3)

          Expectation:

          The retriggered build (build #3) should have the same Git SHA as build #1.

          Observed problem:

          The Git SHA is the HEAD of the branch at the time of the retrigger.
          eegah Eric Griswold made changes -
          Description This is easily reproduced on a standalone Cent 6 or Cent 7 machine.

          Create a job:
          * Uses Git
          * A trivial build step that fails, "false" or "exit 1" in the shell build-step for example
          * Uses one of the above retry mechanisms (Naginator or Rebuilder)
          * In the case of Naginator, stub it with a regex to keep it from automatically retrying
          * In the case of Rebuilder, enable the rebuild without parameters
          * Commit and run a build. It should fail. Note Git SHA (This is build #1)
          * Make a second build and run it. It should also fail. Note the Git SHA (This is build #2)
          * Now go back to build #1 and retrigger it. (build #3)

          Expectation:

          The retriggered build (build #3) should have the same Git SHA as build #1.

          Observed problem:

          The Git SHA is the HEAD of the branch at the time of the retrigger.
          This is easily reproduced on a standalone Cent 6 or Cent 7 machine.

          Create a job:
          * Uses Git
          * A trivial build step that fails, "false" or "exit 1" in the shell build-step for example
          * Uses one of the above retry mechanisms (Naginator or Rebuilder)
          * In the case of Naginator, stub it with a regex to keep it from automatically retrying
          * In the case of Rebuilder, enable the rebuild without parameters
          * Make a commit and run a build. It should fail. Note Git SHA (This is build #1)
          * Make a second commit, then build it. It should also fail. Note the Git SHA (This is build #2)
          * Now go back to build #1 and retrigger it. (build #3)

          Expectation:

          The retriggered build (build #3) should have the same Git SHA as build #1.

          Observed problem:

          The Git SHA is the HEAD of the branch at the time of the retrigger.
          eegah Eric Griswold made changes -
          Description This is easily reproduced on a standalone Cent 6 or Cent 7 machine.

          Create a job:
          * Uses Git
          * A trivial build step that fails, "false" or "exit 1" in the shell build-step for example
          * Uses one of the above retry mechanisms (Naginator or Rebuilder)
          * In the case of Naginator, stub it with a regex to keep it from automatically retrying
          * In the case of Rebuilder, enable the rebuild without parameters
          * Make a commit and run a build. It should fail. Note Git SHA (This is build #1)
          * Make a second commit, then build it. It should also fail. Note the Git SHA (This is build #2)
          * Now go back to build #1 and retrigger it. (build #3)

          Expectation:

          The retriggered build (build #3) should have the same Git SHA as build #1.

          Observed problem:

          The Git SHA is the HEAD of the branch at the time of the retrigger.
          This is easily reproduced on a standalone Cent 6 or Cent 7 machine.

          Create a job:
          * Uses Git
          * A trivial build step that fails, "false" or "exit 1" in the shell build-step for example
          * Uses one of the above retry mechanisms (Naginator or Rebuilder)
          * In the case of Naginator, stub it with a regex to keep it from automatically retrying
          * In the case of Rebuilder, enable the rebuild without parameters
          * Make a commit and run a build. It should fail. Note Git SHA (This is build #1)
          * Make a second commit, then build it. It should also fail. Note the Git SHA (This is build #2)
          * Now go back to build #1 and retrigger it. (build #3)

          Expectation:

          The retriggered build (build #3) should use the same Git SHA as build #1.

          Observed problem:

          The Git SHA is the HEAD of the branch at the time of the retrigger.
          Hide
          fnaum Federico Naum added a comment -

          I'm experiencing the same issue:

          Environmnet: CentOS-6.6
          Jenkins 1.628
          git-client 1.19.0
          github-api plugin 1.69
          github-plugin 1.13.3
          rebuilder 1.25

          The rebuilder plugin does not take into account the revision that was built.

          Looking at the Status of the build, this is what I see: (Note: Git Build Data shows the same information on the commits)

          ----------------------------------------------------------
          Build #1
          Changes
          test1 (commit: bb78bfa2a67d1feb0a6ff887ec4f09aafbba5fa6) (detail)

          Started by an SCM change

          Revision: bb78bfa2a67d1feb0a6ff887ec4f09aafbba5fa6
          origin/master
          ----------------------------------------------------------------------

          make another change

          ----------------------------------------------------------------------
          Build #2
          Changes
          test2 (commit: ec56d24cb0fab5fc41963b22d1ae9af0871a14b9) (detail)

          Started by an SCM change

          Revision: ec56d24cb0fab5fc41963b22d1ae9af0871a14b9
          origin/master
          ----------------------------------------------------------------------

          Now when I go to build #1 and press rebuild, It ends up rebuilding against the last sha (ec56d24cb0fab5fc41963b22d1ae9af0871a14b9 from #2) instead of #1

          ----------------------------------------------------------------------
          Build #3

          No changes.

          Started by an SCM change

          Started by GitHub push by USERNAME

          Started by user USER NAME

          Rebuilds build #1

          Revision: ec56d24cb0fab5fc41963b22d1ae9af0871a14b9
          origin/master
          ----------------------------------------------------------------------

          Show
          fnaum Federico Naum added a comment - I'm experiencing the same issue: Environmnet: CentOS-6.6 Jenkins 1.628 git-client 1.19.0 github-api plugin 1.69 github-plugin 1.13.3 rebuilder 1.25 The rebuilder plugin does not take into account the revision that was built. Looking at the Status of the build, this is what I see: (Note: Git Build Data shows the same information on the commits) ---------------------------------------------------------- Build #1 Changes test1 (commit: bb78bfa2a67d1feb0a6ff887ec4f09aafbba5fa6) (detail) Started by an SCM change Revision: bb78bfa2a67d1feb0a6ff887ec4f09aafbba5fa6 origin/master ---------------------------------------------------------------------- make another change ---------------------------------------------------------------------- Build #2 Changes test2 (commit: ec56d24cb0fab5fc41963b22d1ae9af0871a14b9) (detail) Started by an SCM change Revision: ec56d24cb0fab5fc41963b22d1ae9af0871a14b9 origin/master ---------------------------------------------------------------------- Now when I go to build #1 and press rebuild, It ends up rebuilding against the last sha (ec56d24cb0fab5fc41963b22d1ae9af0871a14b9 from #2) instead of #1 ---------------------------------------------------------------------- Build #3 No changes. Started by an SCM change Started by GitHub push by USERNAME Started by user USER NAME Rebuilds build #1 Revision: ec56d24cb0fab5fc41963b22d1ae9af0871a14b9 origin/master ----------------------------------------------------------------------
          Hide
          xjom Mattias Vannergård added a comment - - edited

          I have the same problem with the Retrigger function in the Gerrit Trigger plugin, so I think it is a fault in the Git plugin, because all the Parameters and Environment Variables are correct, but the Git Build Data is wrong.

          Show
          xjom Mattias Vannergård added a comment - - edited I have the same problem with the Retrigger function in the Gerrit Trigger plugin, so I think it is a fault in the Git plugin, because all the Parameters and Environment Variables are correct, but the Git Build Data is wrong.
          xjom Mattias Vannergård made changes -
          Component/s git-plugin [ 15543 ]
          xjom Mattias Vannergård made changes -
          Environment Standalone CentOS 7 machine. Jenkins 1.622; Git plugin 2.4.0; Naginator 1.15; rebuilder 1.25

          Replicated on a CentOS 6 machine. Jenkins 1.580.3; Git plugin 2.3.5; Naginator 1.14; Rebuilder 1.22
          Standalone CentOS 7 machine. Jenkins 1.622; Git plugin 2.4.0; Naginator 1.15; rebuilder 1.25

          Replicated on a CentOS 6 machine. Jenkins 1.580.3; Git plugin 2.3.5; Naginator 1.14; Rebuilder 1.22

          Replicated on a Win 7 machine. Jenkins 1.596.2; Git plugin 2.4.0; Gerrit plugin 2.16.0; Rebuilder 1.25 (I do have Naginator 1.15 installed, but it is not involved in this job)
          Hide
          danielbeck Daniel Beck added a comment -

          The Git SHA is the HEAD of the branch at the time of the retrigger.

          Not a maintainer of any of these plugins, but I think this is how they're supposed to work. In the case of Rebuild, it just saves you from having to enter the same parameters again when you want the "same" build, but it doesn't care about SCM.

          Show
          danielbeck Daniel Beck added a comment - The Git SHA is the HEAD of the branch at the time of the retrigger. Not a maintainer of any of these plugins, but I think this is how they're supposed to work. In the case of Rebuild, it just saves you from having to enter the same parameters again when you want the "same" build, but it doesn't care about SCM.
          Hide
          markewaite Mark Waite added a comment -

          I think Daniel Beck is correct about the expected behavior of Naginator and Rebuild plugins. They make no mention of retrying a specific SCM revision, only about retrying a build (or a parameterized build). I don't see a "Gerrit Retrigger" plugin in the list of Jenkins plugins, so I can't read its documentation to see if it mentions any expectation of rebuilding the same SCM revision as the previous build.

          Show
          markewaite Mark Waite added a comment - I think Daniel Beck is correct about the expected behavior of Naginator and Rebuild plugins. They make no mention of retrying a specific SCM revision, only about retrying a build (or a parameterized build). I don't see a "Gerrit Retrigger" plugin in the list of Jenkins plugins, so I can't read its documentation to see if it mentions any expectation of rebuilding the same SCM revision as the previous build.
          markewaite Mark Waite made changes -
          Component/s git-plugin [ 15543 ]
          Component/s core [ 15593 ]
          Hide
          xjom Mattias Vannergård added a comment -

          Edited the comment above. Still you may be correct about rebuilding. It could be the Gerrit Trigger/Naginator/Rebuild plugins promising a function, which they cannot achieve using the Git Plugin.
          I think it will probably be easier to change the Git Plugin, then change all of these plugins made for rebuilding earlier (not only the latest) build.

          Show
          xjom Mattias Vannergård added a comment - Edited the comment above. Still you may be correct about rebuilding. It could be the Gerrit Trigger/Naginator/Rebuild plugins promising a function, which they cannot achieve using the Git Plugin. I think it will probably be easier to change the Git Plugin, then change all of these plugins made for rebuilding earlier (not only the latest) build.
          xjom Mattias Vannergård made changes -
          Component/s gerrit-trigger-plugin [ 15731 ]
          Component/s git-plugin [ 15543 ]
          Hide
          markewaite Mark Waite added a comment -

          Mattias Vannergård please explain further why you think they cannot achieve the use model with the git plugin, or why it would be easier to change the git plugin than to change all these plugins. The git plugin does not control the rebuild, that is what is done by the rebuild plugin and the naginator plugin. Jenkins core and the git plugin can answer questions like "what revision was used for build 32?" When rebuild or naginator start a new build (the "rebuild"), they will need to specify the SCM revision to use for that rebuild, if that is the desired behavior. This is not functionality which the git plugin can (or should) implement.

          Show
          markewaite Mark Waite added a comment - Mattias Vannergård please explain further why you think they cannot achieve the use model with the git plugin, or why it would be easier to change the git plugin than to change all these plugins. The git plugin does not control the rebuild, that is what is done by the rebuild plugin and the naginator plugin. Jenkins core and the git plugin can answer questions like "what revision was used for build 32?" When rebuild or naginator start a new build (the "rebuild"), they will need to specify the SCM revision to use for that rebuild, if that is the desired behavior. This is not functionality which the git plugin can (or should) implement.
          markewaite Mark Waite made changes -
          Component/s git-plugin [ 15543 ]
          markewaite Mark Waite made changes -
          Assignee Nicolas De Loof [ ndeloof ]
          xjom Mattias Vannergård made changes -
          Component/s gerrit-trigger-plugin [ 15731 ]
          Hide
          xjom Mattias Vannergård added a comment -

          OK. It was just a guess from my side. Now I have learned more, and that the Gerrit Trigger actually can do it. I had done the configurarion of the Gerrit Trigger wrong, so it was not actually retriggerable. Removing my tag.

          Show
          xjom Mattias Vannergård added a comment - OK. It was just a guess from my side. Now I have learned more, and that the Gerrit Trigger actually can do it. I had done the configurarion of the Gerrit Trigger wrong, so it was not actually retriggerable. Removing my tag.
          Hide
          fnaum Federico Naum added a comment -

          Thanks all, I understand now that the main functionality of the rebuild-plugin is to save you from having to enter the same explicit parameters again. But I still think that the desired behaviour should be to gather somehow the SCM revision from the SCM plugins and rebuild that instead of HEAD/the last revision built. It feels to me that the SCM revision is sort of an implicit parameter of a given build? Could it be done as an option in the rebuild page asking you whether or not to include the SCM data?

          Without knowing the architecture/code, I imagine something like Mark Waite mentioned. When rebuild is pressed on build X, then the rebuild-plugin asks Jenkins core or the SCM plugins which was the revision used to build X.

          I'm happy for this to be considered a nice to have instead of a Major issue, so please feel free to lower the priority to whatever you think is best for this type of issue

          Show
          fnaum Federico Naum added a comment - Thanks all, I understand now that the main functionality of the rebuild-plugin is to save you from having to enter the same explicit parameters again. But I still think that the desired behaviour should be to gather somehow the SCM revision from the SCM plugins and rebuild that instead of HEAD/the last revision built. It feels to me that the SCM revision is sort of an implicit parameter of a given build? Could it be done as an option in the rebuild page asking you whether or not to include the SCM data? Without knowing the architecture/code, I imagine something like Mark Waite mentioned. When rebuild is pressed on build X, then the rebuild-plugin asks Jenkins core or the SCM plugins which was the revision used to build X. I'm happy for this to be considered a nice to have instead of a Major issue, so please feel free to lower the priority to whatever you think is best for this type of issue
          ikedam ikedam made changes -
          Component/s git-plugin [ 15543 ]
          Issue Type Bug [ 1 ] New Feature [ 2 ]
          Summary Build Retry (both Naginator and Rebuild) pick wrong Git SHA under certain circumstances Build Retry (both Naginator and Rebuild) pick the latest commit, not retrying one
          Hide
          ikedam ikedam added a comment -

          Rebuild and naginator just replay "retrieve HEAD" of git-plugin as pointed by Daniel Beck.
          Rebuilding plugins doesn't know about git-plugin and can do nothing for this by themselves.

          Possible solutions:

          • Git-plugin provides an implementation for rebuilding plugins to set up the new build to retrieve the last commit. This is just like GitRevisionBuildParameters provided by git-plugin for parameterized-trigger plugin.
            • This requires rebuilding plugins provide an extension point for this. Git-plugin have to provide implementations for each plugins as they provide their own extension points.
          • Git-plugin detects the rebuilded origin build and extract the used commit.
            • Rebuilding plugins provides information to track the origin build in their own ways.
          Show
          ikedam ikedam added a comment - Rebuild and naginator just replay "retrieve HEAD" of git-plugin as pointed by Daniel Beck . Rebuilding plugins doesn't know about git-plugin and can do nothing for this by themselves. Possible solutions: Git-plugin provides an implementation for rebuilding plugins to set up the new build to retrieve the last commit. This is just like GitRevisionBuildParameters provided by git-plugin for parameterized-trigger plugin. This requires rebuilding plugins provide an extension point for this. Git-plugin have to provide implementations for each plugins as they provide their own extension points. Git-plugin detects the rebuilded origin build and extract the used commit. Rebuilding plugins provides information to track the origin build in their own ways.
          rtyler R. Tyler Croy made changes -
          Workflow JNJira [ 164841 ] JNJira + In-Review [ 181706 ]
          Hide
          pascalbrouwers Pascal Brouwers added a comment -

          Would love to see this feature. Or is there an alternative?

          Show
          pascalbrouwers Pascal Brouwers added a comment - Would love to see this feature. Or is there an alternative?
          Hide
          markewaite Mark Waite added a comment -

          The Jenkins pipeline includes a capability called "Replay" which will run the same build again. I believe it uses the same SHA1 (or subversion commit) on a replay build as it did on the original build.

          Show
          markewaite Mark Waite added a comment - The Jenkins pipeline includes a capability called "Replay" which will run the same build again. I believe it uses the same SHA1 (or subversion commit) on a replay build as it did on the original build.
          Hide
          gst Grégory Starck added a comment -

          Hi,

          I'm facing the same problem in a jenkins project building lots of git branches where we have quite often activity (pushes).. really often there are failed jobs that I want to retry but effectively it's always the last build that was run which is retried, and not the one on which I actually retry ..

          any high level idea when this feature/bugfix could be foreseen/released :-?

          Many thx.

          Show
          gst Grégory Starck added a comment - Hi, I'm facing the same problem in a jenkins project building lots of git branches where we have quite often activity (pushes).. really often there are failed jobs that I want to retry but effectively it's always the last build that was run which is retried, and not the one on which I actually retry .. any high level idea when this feature/bugfix could be foreseen/released :-? Many thx.
          Hide
          markewaite Mark Waite added a comment -

          I have no plans to implement any changes to support this in the git plugin or the git client plugin. I believe the necessary API calls are already available, as evidenced by the pipeline "Replay" function. The pipeline "Replay" uses the same commit in the new build as was used in the original build.

          Show
          markewaite Mark Waite added a comment - I have no plans to implement any changes to support this in the git plugin or the git client plugin. I believe the necessary API calls are already available, as evidenced by the pipeline "Replay" function. The pipeline "Replay" uses the same commit in the new build as was used in the original build.
          Hide
          danielbeck Daniel Beck added a comment -

          as evidenced by the pipeline "Replay" function

          It stores the revision in the job config, which results in side effects such as polling being stupidly broken: JENKINS-40722

          Show
          danielbeck Daniel Beck added a comment - as evidenced by the pipeline "Replay" function It stores the revision in the job config, which results in side effects such as polling being stupidly broken: JENKINS-40722
          Hide
          mingfan ming fan added a comment -

          Have the same issue withJenkins ver. 2.77

          Show
          mingfan ming fan added a comment - Have the same issue withJenkins ver. 2.77
          mingfan ming fan made changes -
          Environment Standalone CentOS 7 machine. Jenkins 1.622; Git plugin 2.4.0; Naginator 1.15; rebuilder 1.25

          Replicated on a CentOS 6 machine. Jenkins 1.580.3; Git plugin 2.3.5; Naginator 1.14; Rebuilder 1.22

          Replicated on a Win 7 machine. Jenkins 1.596.2; Git plugin 2.4.0; Gerrit plugin 2.16.0; Rebuilder 1.25 (I do have Naginator 1.15 installed, but it is not involved in this job)
          Standalone CentOS 7 machine. Jenkins 1.622; Git plugin 2.4.0; Naginator 1.15; rebuilder 1.25

          Replicated on a CentOS 6 machine. Jenkins 1.580.3; Git plugin 2.3.5; Naginator 1.14; Rebuilder 1.22

          Replicated on a Win 7 machine. Jenkins 1.596.2; Git plugin 2.4.0; Gerrit plugin 2.16.0; Rebuilder 1.25 (I do have Naginator 1.15 installed, but it is not involved in this job)

          Replicated on amazon-linux-ami 2017.03.
          mingfan ming fan made changes -
          Environment Standalone CentOS 7 machine. Jenkins 1.622; Git plugin 2.4.0; Naginator 1.15; rebuilder 1.25

          Replicated on a CentOS 6 machine. Jenkins 1.580.3; Git plugin 2.3.5; Naginator 1.14; Rebuilder 1.22

          Replicated on a Win 7 machine. Jenkins 1.596.2; Git plugin 2.4.0; Gerrit plugin 2.16.0; Rebuilder 1.25 (I do have Naginator 1.15 installed, but it is not involved in this job)

          Replicated on amazon-linux-ami 2017.03.
          Standalone CentOS 7 machine. Jenkins 1.622; Git plugin 2.4.0; Naginator 1.15; rebuilder 1.25

          Replicated on a CentOS 6 machine. Jenkins 1.580.3; Git plugin 2.3.5; Naginator 1.14; Rebuilder 1.22

          Replicated on a Win 7 machine. Jenkins 1.596.2; Git plugin 2.4.0; Gerrit plugin 2.16.0; Rebuilder 1.25 (I do have Naginator 1.15 installed, but it is not involved in this job)

          Replicated on amazon-linux-ami 2017.03. Jenkins ver. 2.77

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            eegah Eric Griswold
            Votes:
            13 Vote for this issue
            Watchers:
            14 Start watching this issue

              Dates

              Created:
              Updated: