Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-30475

bad design for Polling vs Checkout algorithms

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Blocker Blocker
    • git-plugin
    • None
    • 2.4.0/1.18.0
      1.609.3

      After long time of debugging i found that current polling algorithm works in the next way:

      Polling:

      1. Get from remote repo branchnames + sha1
      2. compare with existed BuilData from latest build (ONE logic)
      3. If sha1/branch not found -> trigger build (NO RPA attached)

      Checkout:

      1. Build is running and it calls checkout() step in GITScm
      2. checkout() calls getBuildRevision that calls DefaultBuildChooser that calls getAdvancedCandidateRevisions() that calls revs = GitUtils.filterTipBranches(revs); (SECOND logic)
      3. filter tips removes all intermediate branches and they never appears in BuildData

      That ends in situations that Polling needs branches, but checkout didn't build them.
      I fixed this issue by commenting tips filtering https://github.com/KostyaSha/git-plugin/commit/8336202ee5ec8d5a12caa875aeba27b89ac3af58 this allowed build all branches and Polling now satisfied.

      Suggestion:

      1. Put RPA for triggered build as polling result -> allows ensure that BuildData will get what polling wants
      2. Use the same logic for checkout and polling -> more or less allows hope that checkout will pick what polling mean

          [JENKINS-30475] bad design for Polling vs Checkout algorithms

          Kanstantsin Shautsou created issue -
          Kanstantsin Shautsou made changes -
          Description Original: After long time of debugging i found that current polling algorithm works in the next way:

          # Get from remote repo branchnames + sha1
          # compare with existed BuilData in latest build (ONE logic)
          # If sha1/branch not found -> trigger build (NO RPA attached)

          # Build is running and it calls checkout() step in GITScm
          # checkout() calls getBuildRevision that calls DefaultBuildChooser that calls getAdvancedCandidateRevisions() that calls revs = GitUtils.filterTipBranches(revs); (SECOND logic)
          # filter tips removes all intermediate branches and they never appears in BuildData

          That ends in situations that Polling needs branches, but checkout didn't build them.
          I fixed this issue by commenting tips filtering https://github.com/KostyaSha/git-plugin/commit/8336202ee5ec8d5a12caa875aeba27b89ac3af58 this allowed build all branches and Polling now satisfied.

          Suggestion:
          # Put RPA as polling result for triggered build -> allows ensure that BuildData will get what polling wants
          # Use the same logic for checkout and polling -> more or less allows hope that checkout will pick what polling mean
          New: After long time of debugging i found that current polling algorithm works in the next way:

          # Get from remote repo branchnames + sha1
          # compare with existed BuilData from latest build (ONE logic)
          # If sha1/branch not found -> trigger build (NO RPA attached)

          # Build is running and it calls checkout() step in GITScm
          # checkout() calls getBuildRevision that calls DefaultBuildChooser that calls getAdvancedCandidateRevisions() that calls revs = GitUtils.filterTipBranches(revs); (SECOND logic)
          # filter tips removes all intermediate branches and they never appears in BuildData

          That ends in situations that Polling needs branches, but checkout didn't build them.
          I fixed this issue by commenting tips filtering https://github.com/KostyaSha/git-plugin/commit/8336202ee5ec8d5a12caa875aeba27b89ac3af58 this allowed build all branches and Polling now satisfied.

          Suggestion:
          # Put RPA as polling result for triggered build -> allows ensure that BuildData will get what polling wants
          # Use the same logic for checkout and polling -> more or less allows hope that checkout will pick what polling mean
          Kanstantsin Shautsou made changes -
          Description Original: After long time of debugging i found that current polling algorithm works in the next way:

          # Get from remote repo branchnames + sha1
          # compare with existed BuilData from latest build (ONE logic)
          # If sha1/branch not found -> trigger build (NO RPA attached)

          # Build is running and it calls checkout() step in GITScm
          # checkout() calls getBuildRevision that calls DefaultBuildChooser that calls getAdvancedCandidateRevisions() that calls revs = GitUtils.filterTipBranches(revs); (SECOND logic)
          # filter tips removes all intermediate branches and they never appears in BuildData

          That ends in situations that Polling needs branches, but checkout didn't build them.
          I fixed this issue by commenting tips filtering https://github.com/KostyaSha/git-plugin/commit/8336202ee5ec8d5a12caa875aeba27b89ac3af58 this allowed build all branches and Polling now satisfied.

          Suggestion:
          # Put RPA as polling result for triggered build -> allows ensure that BuildData will get what polling wants
          # Use the same logic for checkout and polling -> more or less allows hope that checkout will pick what polling mean
          New: After long time of debugging i found that current polling algorithm works in the next way:

          Polling:
          # Get from remote repo branchnames + sha1
          # compare with existed BuilData from latest build (ONE logic)
          # If sha1/branch not found -> trigger build (NO RPA attached)

          Checkout:
          # Build is running and it calls checkout() step in GITScm
          # checkout() calls getBuildRevision that calls DefaultBuildChooser that calls getAdvancedCandidateRevisions() that calls revs = GitUtils.filterTipBranches(revs); (SECOND logic)
          # filter tips removes all intermediate branches and they never appears in BuildData

          That ends in situations that Polling needs branches, but checkout didn't build them.
          I fixed this issue by commenting tips filtering https://github.com/KostyaSha/git-plugin/commit/8336202ee5ec8d5a12caa875aeba27b89ac3af58 this allowed build all branches and Polling now satisfied.

          Suggestion:
          # Put RPA as polling result for triggered build -> allows ensure that BuildData will get what polling wants
          # Use the same logic for checkout and polling -> more or less allows hope that checkout will pick what polling mean
          Kanstantsin Shautsou made changes -
          Description Original: After long time of debugging i found that current polling algorithm works in the next way:

          Polling:
          # Get from remote repo branchnames + sha1
          # compare with existed BuilData from latest build (ONE logic)
          # If sha1/branch not found -> trigger build (NO RPA attached)

          Checkout:
          # Build is running and it calls checkout() step in GITScm
          # checkout() calls getBuildRevision that calls DefaultBuildChooser that calls getAdvancedCandidateRevisions() that calls revs = GitUtils.filterTipBranches(revs); (SECOND logic)
          # filter tips removes all intermediate branches and they never appears in BuildData

          That ends in situations that Polling needs branches, but checkout didn't build them.
          I fixed this issue by commenting tips filtering https://github.com/KostyaSha/git-plugin/commit/8336202ee5ec8d5a12caa875aeba27b89ac3af58 this allowed build all branches and Polling now satisfied.

          Suggestion:
          # Put RPA as polling result for triggered build -> allows ensure that BuildData will get what polling wants
          # Use the same logic for checkout and polling -> more or less allows hope that checkout will pick what polling mean
          New: After long time of debugging i found that current polling algorithm works in the next way:

          Polling:
          # Get from remote repo branchnames + sha1
          # compare with existed BuilData from latest build (ONE logic)
          # If sha1/branch not found -> trigger build (NO RPA attached)

          Checkout:
          # Build is running and it calls checkout() step in GITScm
          # checkout() calls getBuildRevision that calls DefaultBuildChooser that calls getAdvancedCandidateRevisions() that calls revs = GitUtils.filterTipBranches(revs); (SECOND logic)
          # filter tips removes all intermediate branches and they never appears in BuildData

          That ends in situations that Polling needs branches, but checkout didn't build them.
          I fixed this issue by commenting tips filtering https://github.com/KostyaSha/git-plugin/commit/8336202ee5ec8d5a12caa875aeba27b89ac3af58 this allowed build all branches and Polling now satisfied.

          Suggestion:
          # Put RPA for triggered build as polling result -> allows ensure that BuildData will get what polling wants
          # Use the same logic for checkout and polling -> more or less allows hope that checkout will pick what polling mean
          Mark Waite made changes -
          Resolution New: Duplicate [ 3 ]
          Status Original: Open [ 1 ] New: Closed [ 6 ]
          Kanstantsin Shautsou made changes -
          Resolution Original: Duplicate [ 3 ]
          Status Original: Closed [ 6 ] New: Reopened [ 4 ]
          Mark Waite made changes -
          Assignee Original: Mark Waite [ markewaite ]
          Mark Waite made changes -
          Comment [ Duplicate of JENKINS-30474 ]
          Kanstantsin Shautsou made changes -
          Comment [ Wrong, this issue is latest, Jira didn't response with created 30474 so 30475 was right created and modified.
          UPD: nullified JENKINS-30474 to exclude from search results ]

          I'm bitten by this issue too (Jenkins 1.643, Git Client Plugin 1.19.4, Git Plugin 2.4.2). I worked around it by using the "Alternative build chooser" instead.

          The situation is thus:
          Just updated master, then merged into a side branch.
          Polling detects new change 074727053356fd1ee42fce0f4ee4ea7580c12db3 and kicks off a build
          Build eliminates the tip of both branches as the same commit exists on both side
          Build reverts to building an older commit

          In the next poll period, the same thing happens, resulting in the older commit being built ad-infinitum.

          Heres a log snippet from the build with verbosity enabled (formatted for readability):
          5:22:08 Starting with all the branches: [
          Revision 156807c06f43f162018875ac7e8748e098894f05 (origin/master-next),
          Revision e8edab87cb3031b14fd4e18be35649935ac80ef6 (origin/bleeding-edge),
          Revision 074727053356fd1ee42fce0f4ee4ea7580c12db3 (origin/master-ltc, origin/master),
          Revision ae914e5683c1d72f786f82b08b1c4a3e095d1ad3 (origin/master-next-ltc)]

          15:22:08 After branch filtering: [
          Revision 156807c06f43f162018875ac7e8748e098894f05 (origin/master-next),
          Revision e8edab87cb3031b14fd4e18be35649935ac80ef6 (origin/bleeding-edge),
          Revision 074727053356fd1ee42fce0f4ee4ea7580c12db3 (origin/master-ltc, origin/master),
          Revision ae914e5683c1d72f786f82b08b1c4a3e095d1ad3 (origin/master-next-ltc)]

          15:22:08 After non-tip filtering: [
          Revision ae914e5683c1d72f786f82b08b1c4a3e095d1ad3 (origin/master-next-ltc),
          Revision e8edab87cb3031b14fd4e18be35649935ac80ef6 (origin/bleeding-edge)]

          15:22:08 Removing what's already been built:

          { origin/bleeding-edge=Build #7 of Revision e8edab87cb3031b14fd4e18be35649935ac80ef6 (origin/bleeding-edge), origin/master-next-ltc=Build #1 of Revision ae914e5683c1d72f786f82b08b1c4a3e095d1ad3 (origin/master-next-ltc)}

          15:22:08 After filtering out what's already been built: []
          15:22:08 Nothing seems worth building, so falling back to the previously built revision: Revision e8edab87cb3031b14fd4e18be35649935ac80ef6 (origin/bleeding-edge)

          The relevent commit history:
          master
          commit 074727053356fd1ee42fce0f4ee4ea7580c12db3
          Merge: 396ca00 27cea07
          commit 27cea072e46bdcd9dd8bc320a84883b28609eb99
          commit 396ca006e7f9661ebd69852cc6b12f0fa1fbafbc
          Merge: 3f9d9b5 71faa0f
          commit 71faa0f76f48f415125b9f2d663c617b4131c17c
          Merge: b65cbe9 e4aa6fb

          master-ltc
          commit 074727053356fd1ee42fce0f4ee4ea7580c12db3
          Merge: 396ca00 27cea07
          commit 27cea072e46bdcd9dd8bc320a84883b28609eb99
          commit 396ca006e7f9661ebd69852cc6b12f0fa1fbafbc
          Merge: 3f9d9b5 71faa0f
          commit 71faa0f76f48f415125b9f2d663c617b4131c17c
          Merge: b65cbe9 e4aa6fb

          Alastair D'Silva added a comment - I'm bitten by this issue too (Jenkins 1.643, Git Client Plugin 1.19.4, Git Plugin 2.4.2). I worked around it by using the "Alternative build chooser" instead. The situation is thus: Just updated master, then merged into a side branch. Polling detects new change 074727053356fd1ee42fce0f4ee4ea7580c12db3 and kicks off a build Build eliminates the tip of both branches as the same commit exists on both side Build reverts to building an older commit In the next poll period, the same thing happens, resulting in the older commit being built ad-infinitum. Heres a log snippet from the build with verbosity enabled (formatted for readability): 5:22:08 Starting with all the branches: [ Revision 156807c06f43f162018875ac7e8748e098894f05 (origin/master-next), Revision e8edab87cb3031b14fd4e18be35649935ac80ef6 (origin/bleeding-edge), Revision 074727053356fd1ee42fce0f4ee4ea7580c12db3 (origin/master-ltc, origin/master), Revision ae914e5683c1d72f786f82b08b1c4a3e095d1ad3 (origin/master-next-ltc)] 15:22:08 After branch filtering: [ Revision 156807c06f43f162018875ac7e8748e098894f05 (origin/master-next), Revision e8edab87cb3031b14fd4e18be35649935ac80ef6 (origin/bleeding-edge), Revision 074727053356fd1ee42fce0f4ee4ea7580c12db3 (origin/master-ltc, origin/master), Revision ae914e5683c1d72f786f82b08b1c4a3e095d1ad3 (origin/master-next-ltc)] 15:22:08 After non-tip filtering: [ Revision ae914e5683c1d72f786f82b08b1c4a3e095d1ad3 (origin/master-next-ltc), Revision e8edab87cb3031b14fd4e18be35649935ac80ef6 (origin/bleeding-edge)] 15:22:08 Removing what's already been built: { origin/bleeding-edge=Build #7 of Revision e8edab87cb3031b14fd4e18be35649935ac80ef6 (origin/bleeding-edge), origin/master-next-ltc=Build #1 of Revision ae914e5683c1d72f786f82b08b1c4a3e095d1ad3 (origin/master-next-ltc)} 15:22:08 After filtering out what's already been built: [] 15:22:08 Nothing seems worth building, so falling back to the previously built revision: Revision e8edab87cb3031b14fd4e18be35649935ac80ef6 (origin/bleeding-edge) The relevent commit history: master commit 074727053356fd1ee42fce0f4ee4ea7580c12db3 Merge: 396ca00 27cea07 commit 27cea072e46bdcd9dd8bc320a84883b28609eb99 commit 396ca006e7f9661ebd69852cc6b12f0fa1fbafbc Merge: 3f9d9b5 71faa0f commit 71faa0f76f48f415125b9f2d663c617b4131c17c Merge: b65cbe9 e4aa6fb master-ltc commit 074727053356fd1ee42fce0f4ee4ea7580c12db3 Merge: 396ca00 27cea07 commit 27cea072e46bdcd9dd8bc320a84883b28609eb99 commit 396ca006e7f9661ebd69852cc6b12f0fa1fbafbc Merge: 3f9d9b5 71faa0f commit 71faa0f76f48f415125b9f2d663c617b4131c17c Merge: b65cbe9 e4aa6fb

            Unassigned Unassigned
            integer Kanstantsin Shautsou
            Votes:
            2 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated: