Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-40771

Race condition in FlowExecutionList

    XMLWordPrintable

Details

    Description

      Hi,

      We found a potential bug that can only be replicated in pipeline jobs. Essentially when a job a running and a Jenkins restart occurs, the job is left hanging infinitely:

      Resuming build at Tue Jan 03 10:37:18 UTC 2017 after Jenkins restart
      Waiting to resume part of TestRun2 #2: Waiting for next available executor
      Waiting to resume part of TestRun2 #2: Waiting for next available executor
      Waiting to resume part of TestRun2 #2: Waiting for next available executor
      Waiting to resume part of TestRun2 #2: Waiting for next available executor
      Waiting to resume part of TestRun2 #2: Waiting for next available executor
      Waiting to resume part of TestRun2 #2: Waiting for next available executor
      Waiting to resume part of TestRun2 #2: Waiting for next available executor
      ...
      

      I noticed that this behaviour does not exhibit on any other job types. i.e. freestyle.

      Here is a simple test pipeline script:

      node('XXXXX') {
      
        stage 'Stage 1'
          println 'Deploying to Stage 1...'
      
        stage 'Stage 2'
          println 'Running Tests in Stage 2'
          sleep 120
          println 'Tests passed!'
      
        stage 'Stage 3'
          println 'Deploying to Stage 3...'
      
      }
      

      ...Restart Jenkins as soon as it enters Stage 2, to replicate such behaviour.

      Currently I am using version 2.3, but I believe this issue was replicated in previous versions.

      Please can you help me explain why this behaviour only exists in pipeline jobs?

      Kind Regards,
      Tuan

      Attachments

        Issue Links

          Activity

            derng Tuan Nguyen created issue -
            derng Tuan Nguyen made changes -
            Field Original Value New Value
            Summary Pipeline causes job to hang infinitely on restart Pipeline causes job to hang infinitely on Jenkins restart
            derng Tuan Nguyen made changes -
            Description Hi,

            We found a potential bug that can only be replicated in pipeline jobs. Essentially when a job a running and a Jenkins restart occurs, the job is left hanging infinitely:
            {noformat}
            Resuming build at Tue Jan 03 10:37:18 UTC 2017 after Jenkins restart
            Waiting to resume part of TestRun2 #2: Waiting for next available executor
            Waiting to resume part of TestRun2 #2: Waiting for next available executor
            Waiting to resume part of TestRun2 #2: Waiting for next available executor
            Waiting to resume part of TestRun2 #2: Waiting for next available executor
            Waiting to resume part of TestRun2 #2: Waiting for next available executor
            Waiting to resume part of TestRun2 #2: Waiting for next available executor
            Waiting to resume part of TestRun2 #2: Waiting for next available executor
            ...
            {noformat}

            I noticed that this does not occur on any other job types. i.e. freestyle.


            Here is a simple test pipeline script:
            {noformat}
            node('XXXXX') {

              stage 'Stage 1'
                println 'Deploying to Stage 1...'

              stage 'Stage 2'
                println 'Running Tests in Stage 2'
                sleep 120
                println 'Tests passed!'

              stage 'Stage 3'
                println 'Deploying to Stage 3...'

            }
            {noformat}
            ...Restart Jenkins as soon as it enters Stage 2, to replicate such behaviour.

            Currently I am using version 2.3, but I believe this issue was replicated in previous versions.

            Please can you help me explain why this behaviour only exists in pipeline jobs?

            Kind Regards,
            Tuan
            Hi,

            We found a potential bug that can only be replicated in pipeline jobs. Essentially when a job a running and a Jenkins restart occurs, the job is left hanging infinitely:
            {noformat}
            Resuming build at Tue Jan 03 10:37:18 UTC 2017 after Jenkins restart
            Waiting to resume part of TestRun2 #2: Waiting for next available executor
            Waiting to resume part of TestRun2 #2: Waiting for next available executor
            Waiting to resume part of TestRun2 #2: Waiting for next available executor
            Waiting to resume part of TestRun2 #2: Waiting for next available executor
            Waiting to resume part of TestRun2 #2: Waiting for next available executor
            Waiting to resume part of TestRun2 #2: Waiting for next available executor
            Waiting to resume part of TestRun2 #2: Waiting for next available executor
            ...
            {noformat}

            I noticed that this behaviour does not exhibit on any other job types. i.e. freestyle.


            Here is a simple test pipeline script:
            {noformat}
            node('XXXXX') {

              stage 'Stage 1'
                println 'Deploying to Stage 1...'

              stage 'Stage 2'
                println 'Running Tests in Stage 2'
                sleep 120
                println 'Tests passed!'

              stage 'Stage 3'
                println 'Deploying to Stage 3...'

            }
            {noformat}
            ...Restart Jenkins as soon as it enters Stage 2, to replicate such behaviour.

            Currently I am using version 2.3, but I believe this issue was replicated in previous versions.

            Please can you help me explain why this behaviour only exists in pipeline jobs?

            Kind Regards,
            Tuan
            jglick Jesse Glick made changes -
            Component/s workflow-basic-steps-plugin [ 21712 ]
            Component/s workflow-aggregator-plugin [ 21710 ]
            jglick Jesse Glick made changes -
            Resolution Incomplete [ 4 ]
            Status Open [ 1 ] Resolved [ 5 ]
            derng Tuan Nguyen made changes -
            Resolution Incomplete [ 4 ]
            Status Resolved [ 5 ] Reopened [ 4 ]
            jglick Jesse Glick made changes -
            Component/s workflow-api-plugin [ 21711 ]
            Component/s workflow-basic-steps-plugin [ 21712 ]
            Labels threads
            Summary Pipeline causes job to hang infinitely on Jenkins restart Race condition in FlowExecutionList
            jglick Jesse Glick made changes -
            Assignee Jesse Glick [ jglick ]
            jglick Jesse Glick made changes -
            Status Reopened [ 4 ] In Progress [ 3 ]
            jglick Jesse Glick made changes -
            Remote Link This issue links to "PR 31 (Web Link)" [ 15446 ]
            jglick Jesse Glick made changes -
            Status In Progress [ 3 ] In Review [ 10005 ]
            jglick Jesse Glick made changes -
            Priority Minor [ 4 ] Critical [ 2 ]
            jglick Jesse Glick made changes -
            Resolution Fixed [ 1 ]
            Status In Review [ 10005 ] Resolved [ 5 ]
            cloudbees CloudBees Inc. made changes -
            Remote Link This issue links to "CloudBees Internal OSS-1990 (Web Link)" [ 18477 ]

            People

              jglick Jesse Glick
              derng Tuan Nguyen
              Votes:
              3 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: