Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-37880

Starting, stopping and replaying a run in quick succession does not work

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Trivial Trivial
    • blueocean-plugin
    • None

      Steps to reproduce

      1. Create a new multi-branch pipeline using https://github.com/i386/hellonode
      2. Go to the branches screen
      3. Trigger a new build for master
      4. Open the run and click stop
      5. Then click replay again

      The goal here is to try starting, stopping then replaying really quickly.

      You will get a 500 error back from the replay action

      PUT http://localhost:8080/jenkins/blue/rest/organizations/jenkins/pipelines/hellonode/branches/master/runs/23/replay/ 500 (Server Error)

      {
      "message" : "Run was not added to queue.",
      "code" : 500,
      "errors" : [ ]
      }

      I attached a debugger when reproducing and got org.jenkinsci.plugins.workflow.cps.replay.ReplayAction#run2 returning null. Digging deeper it appears we can't get the flow owner

      private @CheckForNull CpsFlowExecution getExecution() {
              FlowExecutionOwner owner = ((FlowExecutionOwner.Executable) run).asFlowExecutionOwner();
              if (owner == null) {
                  return null;
              }
              FlowExecution exec = owner.getOrNull();
              return exec instanceof CpsFlowExecution ? (CpsFlowExecution) exec : null;
          }
      

      I suspect this is a bug in workflow?

          [JENKINS-37880] Starting, stopping and replaying a run in quick succession does not work

          Vivek Pandey added a comment - - edited

          jamesdumay It looks legit failure. In debugger I see its logging with FINE level "java.util.concurrent.ExecutionException: org.jenkinsci.plugins.github_branch_source.RateLimitExceededException: GitHub API rate limit exceeded" and eats away the exception. Perhaps it should be logged as error and maybe thrown as exception so that callers know what went wrong instead of getting null execution instance.

          @Override public FlowExecution getOrNull() {
                      try {
                          ListenableFuture<FlowExecution> promise = run().getExecutionPromise();
                          if (promise.isDone()) {
                              return promise.get();
                          }
                      } catch (Exception x) {
                          LOGGER.log(/* not important */Level.FINE, null, x); //Vivek: This is where its print what went wrong
                      }
          

          Vivek Pandey added a comment - - edited jamesdumay It looks legit failure. In debugger I see its logging with FINE level "java.util.concurrent.ExecutionException: org.jenkinsci.plugins.github_branch_source.RateLimitExceededException: GitHub API rate limit exceeded" and eats away the exception. Perhaps it should be logged as error and maybe thrown as exception so that callers know what went wrong instead of getting null execution instance. @Override public FlowExecution getOrNull() { try { ListenableFuture<FlowExecution> promise = run().getExecutionPromise(); if (promise.isDone()) { return promise.get(); } } catch (Exception x) { LOGGER.log( /* not important */ Level.FINE, null , x); //Vivek: This is where its print what went wrong }

          James Dumay added a comment -

          vivek I think bubbling more info up to the client here would be the right way to go. Then we can follow up with a UI story to display the error to the user. WDYT?

          James Dumay added a comment - vivek I think bubbling more info up to the client here would be the right way to go. Then we can follow up with a UI story to display the error to the user. WDYT?

          Vivek Pandey added a comment -

          Right, this needs to be fixed in underlying pipeline plugin, once it's thrown as exception then BO API can send that error back to UI.

          Vivek Pandey added a comment - Right, this needs to be fixed in underlying pipeline plugin, once it's thrown as exception then BO API can send that error back to UI.

          James Dumay added a comment -

          vivek cool - this can wait until another sprint to pick up.

          James Dumay added a comment - vivek cool - this can wait until another sprint to pick up.

          James Dumay added a comment -

          vivek michaelneale I found another case where replay doesn't work (500 is received).

          I believe this run had a problem connecting to Github and is now un-replayable.

          Started by user James Dumay
          Connecting to https://api.github.com using i386/******
           > git rev-parse --is-inside-work-tree # timeout=10
          Fetching changes from 2 remote Git repositories
           > git config remote.origin.url https://github.com/i386/app-store-demo.git # timeout=10
          Fetching upstream changes from https://github.com/i386/app-store-demo.git
           > git --version # timeout=10
          using .gitcredentials to set credentials
           > git config --local credential.username i386 # timeout=10
           > git config --local credential.helper store --file=/var/folders/11/5vmtkd9j6kd3w6sbrsrw0sh80000gn/T/git561487727535648502.credentials # timeout=10
           > git -c core.askpass=true fetch --tags --progress https://github.com/i386/app-store-demo.git +refs/heads/*:refs/remotes/origin/*
           > git config --local --remove-section credential # timeout=10
          ERROR: Error fetching remote repo 'origin'
          hudson.plugins.git.GitException: Failed to fetch from https://github.com/i386/app-store-demo.git
          	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:799)
          	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1055)
          	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1086)
          	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:109)
          	at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:108)
          	at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:85)
          	at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:207)
          	at hudson.model.ResourceController.execute(ResourceController.java:98)
          	at hudson.model.Executor.run(Executor.java:404)
          Caused by: hudson.plugins.git.GitException: Command "git -c core.askpass=true fetch --tags --progress https://github.com/i386/app-store-demo.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
          stdout: 
          stderr: fatal: unable to access 'https://github.com/i386/app-store-demo.git/': Couldn't connect to server
          
          	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1740)
          	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1476)
          	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:63)
          	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:314)
          	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:797)
          	... 8 more
          
          GitHub has been notified of this commit’s build result
          
          ERROR: null
          Finished: FAILURE
          
          

          James Dumay added a comment - vivek michaelneale I found another case where replay doesn't work (500 is received). I believe this run had a problem connecting to Github and is now un-replayable. Started by user James Dumay Connecting to https: //api.github.com using i386/****** > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from 2 remote Git repositories > git config remote.origin.url https: //github.com/i386/app-store-demo.git # timeout=10 Fetching upstream changes from https: //github.com/i386/app-store-demo.git > git --version # timeout=10 using .gitcredentials to set credentials > git config --local credential.username i386 # timeout=10 > git config --local credential.helper store --file=/ var /folders/11/5vmtkd9j6kd3w6sbrsrw0sh80000gn/T/git561487727535648502.credentials # timeout=10 > git -c core.askpass= true fetch --tags --progress https: //github.com/i386/app-store-demo.git +refs/heads/*:refs/remotes/origin/* > git config --local --remove-section credential # timeout=10 ERROR: Error fetching remote repo 'origin' hudson.plugins.git.GitException: Failed to fetch from https: //github.com/i386/app-store-demo.git at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:799) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1055) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1086) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:109) at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:108) at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:85) at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:207) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:404) Caused by: hudson.plugins.git.GitException: Command "git -c core.askpass= true fetch --tags --progress https: //github.com/i386/app-store-demo.git +refs/heads/*:refs/remotes/origin/*" returned status code 128: stdout: stderr: fatal: unable to access 'https: //github.com/i386/app-store-demo.git/' : Couldn't connect to server at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1740) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1476) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:63) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:314) at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:797) ... 8 more GitHub has been notified of this commit’s build result ERROR: null Finished: FAILURE

          Michael Neale added a comment -

          So perhaps not trivial?

          Michael Neale added a comment - So perhaps not trivial?

          James Dumay added a comment -

          michaelneale replay seems to only work if the flow graph was persisted correctly to disk. If it is not there (in the case of there being no flow nodes in the last example) then it fails.

          James Dumay added a comment - michaelneale replay seems to only work if the flow graph was persisted correctly to disk. If it is not there (in the case of there being no flow nodes in the last example) then it fails.

          Vivek Pandey added a comment - - edited

          jamesdumay Might be worth checking with imeredith, think he wrote replay functionality in pipeline plugin.

          Vivek Pandey added a comment - - edited jamesdumay Might be worth checking with imeredith , think he wrote replay functionality in pipeline plugin.

          James Dumay added a comment -

          vivek yes, all of these cases received a 500.

          imeredith / vivek it would be nice we could sidestep this stuff for replay. Take the commit Ids from the run that is being replayed, ask Jenkins to build those, etc (allowing it to read the Jenkinsfile again instead of using the previous pipeline state).

          Probably easier said than done?

          James Dumay added a comment - vivek yes, all of these cases received a 500. imeredith / vivek it would be nice we could sidestep this stuff for replay. Take the commit Ids from the run that is being replayed, ask Jenkins to build those, etc (allowing it to read the Jenkinsfile again instead of using the previous pipeline state). Probably easier said than done?

          James Dumay added a comment -

          Found the cause of this and there is a fix in Blue Ocean 1.1 coming for it.

          James Dumay added a comment - Found the cause of this and there is a fix in Blue Ocean 1.1 coming for it.

            vivek Vivek Pandey
            jamesdumay James Dumay
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: