Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-21652

Building at each git-client polling interval even when no changes present

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • git-plugin
    • Linux Ubuntu 13.04
      Apache Tomcat/7.0.35
      Jenkins 1.549
      git-client plugin 1.6.2
      GitLab 6.1.0

      Gitlab is running on the same linux server under a different sub-domain & different linux user name.

      When I upgraded the git-client plugin from 1.6.1 to 1.6.2, a project now builds every time it polls the SCM (a GitLab repo) even if the version hash is identical.

      Some jobs indicate that there is a problem and "could not remove the credential section from the git configuration" (see below polling log sample). But even these jobs build on every SCM poll even when the git version has has not changed. Below, I have 8 builds in a row that all reference the same revision, and all say "changes found."

      ======================
      Started on Feb 4, 2014 5:53:33 PM
      Using strategy: Default
      [poll] Last Built Revision: Revision 1c8eda4fa98b092973c7dfe3b3c1d6c859971922 (origin/MASTER)
      using .gitcredentials to set credentials
      Could not remove the credential section from the git configuration
      Done. Took 0.52 sec
      Changes found

          [JENKINS-21652] Building at each git-client polling interval even when no changes present

          Output looks good. no surprises:

          ===================================
          tomcat@XXXXXXXXXX:~/testing$ git ls-remote -h http://lab.XXXXXXXXXX.com/jason/ioix.git
          Username for 'http://lab.XXXXXXXXXX.com': jason@XXXXXXXXXX.com
          Password for 'http://jason@XXXXXXXXXX.com@lab.XXXXXXXXXX.com':
          5bc93624698780e7faf7dff1931882d824b009b5 refs/heads/XXXXXXXXXX
          6a86d86b943dcd7a6044862d17bdebabdc1e993a refs/heads/XXXXXXXXXX
          1ac0baa92afb0695675e40374d06c3b9353bb80f refs/heads/XXXXXXXXXX
          36999ebc0cd3ed2b4059f06a38be97a11a56a452 refs/heads/XXXXXXXXXX
          tomcat@XXXXXXXXXX:~/testing$
          ===================================

          jason robinson added a comment - Output looks good. no surprises: =================================== tomcat@XXXXXXXXXX:~/testing$ git ls-remote -h http://lab.XXXXXXXXXX.com/jason/ioix.git Username for 'http://lab.XXXXXXXXXX.com': jason@XXXXXXXXXX.com Password for 'http://jason@XXXXXXXXXX.com@lab.XXXXXXXXXX.com': 5bc93624698780e7faf7dff1931882d824b009b5 refs/heads/XXXXXXXXXX 6a86d86b943dcd7a6044862d17bdebabdc1e993a refs/heads/XXXXXXXXXX 1ac0baa92afb0695675e40374d06c3b9353bb80f refs/heads/XXXXXXXXXX 36999ebc0cd3ed2b4059f06a38be97a11a56a452 refs/heads/XXXXXXXXXX tomcat@XXXXXXXXXX:~/testing$ ===================================

          Mark Waite added a comment -

          When you pasted that output, were the "Username for" line and the "Password for" line both in the output?

          I'm not sure how the git ls-remote parser will handle that output. I'll need to do more investigation in the source code this weekend. I think it expects a 40 character SHA1 followed by white space followed by the branch name.

          Mark Waite added a comment - When you pasted that output, were the "Username for" line and the "Password for" line both in the output? I'm not sure how the git ls-remote parser will handle that output. I'll need to do more investigation in the source code this weekend. I think it expects a 40 character SHA1 followed by white space followed by the branch name.

          That command was run from the linux server with out the aid of a credential store reference, so I had to authenticate to the server. That shouldn't be present if using a git command that makes use of the jenkins credential store.

          jason robinson added a comment - That command was run from the linux server with out the aid of a credential store reference, so I had to authenticate to the server. That shouldn't be present if using a git command that makes use of the jenkins credential store.

          Will Saxon added a comment -

          We're seeing this exact same problem with 1.609.1 and git plugin 2.3.5. We have polling configured on a job to look every 3 hours, and it builds every time even if there are no changes.

          We are not using the fast remote polling, because we are trying to use the include/exclude feature to limit triggers to a subset of files in the project.

          One thing that may be contributing to this is that the last build revision is listed as refs/remotes/origin/branch instead of refs/heads/branch. It's not clear what command is being used when performing the poll, but git ls-remote -h <url> never returns anything for refs/remotes/origin, only refs/heads.

          Will Saxon added a comment - We're seeing this exact same problem with 1.609.1 and git plugin 2.3.5. We have polling configured on a job to look every 3 hours, and it builds every time even if there are no changes. We are not using the fast remote polling, because we are trying to use the include/exclude feature to limit triggers to a subset of files in the project. One thing that may be contributing to this is that the last build revision is listed as refs/remotes/origin/branch instead of refs/heads/branch. It's not clear what command is being used when performing the poll, but git ls-remote -h <url> never returns anything for refs/remotes/origin, only refs/heads.

          Mark Waite added a comment -

          wsaxon I've been seeking a way to repeatably show a "builds on every poll" bug that I had seen while testing the pre-release of git plugin 2.3.6. While seeking repeatable steps for the "builds on every poll" bug that I had seen, I found that for the case I was trying, 2.3.4 was better behaved than 2.3.5.

          Would you be willing to try the same sequence of steps with the 2.3.4 git plugin to see if that isolates the problem to specifically the changes between 2.3.4 and 2.3.5?

          Mark Waite added a comment - wsaxon I've been seeking a way to repeatably show a "builds on every poll" bug that I had seen while testing the pre-release of git plugin 2.3.6. While seeking repeatable steps for the "builds on every poll" bug that I had seen, I found that for the case I was trying, 2.3.4 was better behaved than 2.3.5. Would you be willing to try the same sequence of steps with the 2.3.4 git plugin to see if that isolates the problem to specifically the changes between 2.3.4 and 2.3.5?

          Will Saxon added a comment - - edited

          No problem, I just downgraded. Same behavior, although I triggered the poll using the Poll SCM Plugin's 'Poll Now' button.

          I've included the relevant SCM and polling config below; most of our jobs use parameters and env-inject to set some generic stuff.

          Our goal is to have pre-merge builds using the Gerrit trigger, and then QA builds on a timer only if code changes or the build is triggered by upstream. We could just use the Gerrit trigger for post-merge, but that ends up producing too many builds.

           <scm class="hudson.plugins.git.GitSCM" plugin="git@2.3.4">
              <configVersion>2</configVersion>
              <userRemoteConfigs>
                <hudson.plugins.git.UserRemoteConfig>
                  <url>ssh://svc_jenkins_cm@${GERRIT}/${MASTER_REPO}</url>
                  <credentialsId>9f9b349f-f711-46dc-98a6-dfc8e645abb8</credentialsId>
                </hudson.plugins.git.UserRemoteConfig>
              </userRemoteConfigs>
              <branches>
                <hudson.plugins.git.BranchSpec>
                  <name>origin/${BRANCH}</name>
                </hudson.plugins.git.BranchSpec>
              </branches>
              <doGenerateSubmoduleConfigurations>false</doGenerateSubmoduleConfigurations>
              <browser class="hudson.plugins.git.browser.CGit">
                <url>http://${GERRIT}/git/${MASTER_REPO}</url>
              </browser>
              <submoduleCfg class="list"/>
              <extensions>
                <hudson.plugins.git.extensions.impl.CloneOption>
                  <shallow>false</shallow>
                  <reference>${REFERENCE_PATH}\${MASTER_REPO}.git</reference>
                </hudson.plugins.git.extensions.impl.CloneOption>
                <hudson.plugins.git.extensions.impl.RelativeTargetDirectory>
                  <relativeTargetDir>${MASTER_REPO}</relativeTargetDir>
                </hudson.plugins.git.extensions.impl.RelativeTargetDirectory>
                <hudson.plugins.git.extensions.impl.SparseCheckoutPaths>
                  <sparseCheckoutPaths>
                    <hudson.plugins.git.extensions.impl.SparseCheckoutPath>
                      <path>/CM/Common</path>
                    </hudson.plugins.git.extensions.impl.SparseCheckoutPath>
                    <hudson.plugins.git.extensions.impl.SparseCheckoutPath>
                      <path>/CM/build_name</path>
                    </hudson.plugins.git.extensions.impl.SparseCheckoutPath>
                    <hudson.plugins.git.extensions.impl.SparseCheckoutPath>
                      <path>/DotNet/Common</path>
                    </hudson.plugins.git.extensions.impl.SparseCheckoutPath>
                    <hudson.plugins.git.extensions.impl.SparseCheckoutPath>
                      <path>/DotNet/One</path>
                    </hudson.plugins.git.extensions.impl.SparseCheckoutPath>
                    <hudson.plugins.git.extensions.impl.SparseCheckoutPath>
                      <path>/DotNet/Two</path>
                    </hudson.plugins.git.extensions.impl.SparseCheckoutPath>
                    <hudson.plugins.git.extensions.impl.SparseCheckoutPath>
                      <path>/DotNet/StrongNameKey.snk</path>
                    </hudson.plugins.git.extensions.impl.SparseCheckoutPath>
                  </sparseCheckoutPaths>
                </hudson.plugins.git.extensions.impl.SparseCheckoutPaths>
                <hudson.plugins.git.extensions.impl.CleanBeforeCheckout/>
                <hudson.plugins.git.extensions.impl.PathRestriction>
                  <includedRegions>^CM/build_name/.*
          ^DotNet/Common/.*
          ^DotNet/One/.*
          ^DotNet/Two/.*
          ^DotNet/StrongNameKey.snk</includedRegions>
                  <excludedRegions></excludedRegions>
                </hudson.plugins.git.extensions.impl.PathRestriction>
                <hudson.plugins.git.extensions.impl.DisableRemotePoll/>
              </extensions>
            </scm>
            <triggers>
              <hudson.triggers.SCMTrigger>
                <spec>H 7-21/1 * * *</spec>
                <ignorePostCommitHooks>false</ignorePostCommitHooks>
              </hudson.triggers.SCMTrigger>
            </triggers>
          

          Will Saxon added a comment - - edited No problem, I just downgraded. Same behavior, although I triggered the poll using the Poll SCM Plugin's 'Poll Now' button. I've included the relevant SCM and polling config below; most of our jobs use parameters and env-inject to set some generic stuff. Our goal is to have pre-merge builds using the Gerrit trigger, and then QA builds on a timer only if code changes or the build is triggered by upstream. We could just use the Gerrit trigger for post-merge, but that ends up producing too many builds. <scm class= "hudson.plugins.git.GitSCM" plugin= "git@2.3.4" > <configVersion> 2 </configVersion> <userRemoteConfigs> <hudson.plugins.git.UserRemoteConfig> <url> ssh://svc_jenkins_cm@${GERRIT}/${MASTER_REPO} </url> <credentialsId> 9f9b349f-f711-46dc-98a6-dfc8e645abb8 </credentialsId> </hudson.plugins.git.UserRemoteConfig> </userRemoteConfigs> <branches> <hudson.plugins.git.BranchSpec> <name> origin/${BRANCH} </name> </hudson.plugins.git.BranchSpec> </branches> <doGenerateSubmoduleConfigurations> false </doGenerateSubmoduleConfigurations> <browser class= "hudson.plugins.git.browser.CGit" > <url> http://${GERRIT}/git/${MASTER_REPO} </url> </browser> <submoduleCfg class= "list" /> <extensions> <hudson.plugins.git.extensions.impl.CloneOption> <shallow> false </shallow> <reference> ${REFERENCE_PATH}\${MASTER_REPO}.git </reference> </hudson.plugins.git.extensions.impl.CloneOption> <hudson.plugins.git.extensions.impl.RelativeTargetDirectory> <relativeTargetDir> ${MASTER_REPO} </relativeTargetDir> </hudson.plugins.git.extensions.impl.RelativeTargetDirectory> <hudson.plugins.git.extensions.impl.SparseCheckoutPaths> <sparseCheckoutPaths> <hudson.plugins.git.extensions.impl.SparseCheckoutPath> <path> /CM/Common </path> </hudson.plugins.git.extensions.impl.SparseCheckoutPath> <hudson.plugins.git.extensions.impl.SparseCheckoutPath> <path> /CM/build_name </path> </hudson.plugins.git.extensions.impl.SparseCheckoutPath> <hudson.plugins.git.extensions.impl.SparseCheckoutPath> <path> /DotNet/Common </path> </hudson.plugins.git.extensions.impl.SparseCheckoutPath> <hudson.plugins.git.extensions.impl.SparseCheckoutPath> <path> /DotNet/One </path> </hudson.plugins.git.extensions.impl.SparseCheckoutPath> <hudson.plugins.git.extensions.impl.SparseCheckoutPath> <path> /DotNet/Two </path> </hudson.plugins.git.extensions.impl.SparseCheckoutPath> <hudson.plugins.git.extensions.impl.SparseCheckoutPath> <path> /DotNet/StrongNameKey.snk </path> </hudson.plugins.git.extensions.impl.SparseCheckoutPath> </sparseCheckoutPaths> </hudson.plugins.git.extensions.impl.SparseCheckoutPaths> <hudson.plugins.git.extensions.impl.CleanBeforeCheckout/> <hudson.plugins.git.extensions.impl.PathRestriction> <includedRegions> ^CM/build_name/.* ^DotNet/Common/.* ^DotNet/One/.* ^DotNet/Two/.* ^DotNet/StrongNameKey.snk </includedRegions> <excludedRegions> </excludedRegions> </hudson.plugins.git.extensions.impl.PathRestriction> <hudson.plugins.git.extensions.impl.DisableRemotePoll/> </extensions> </scm> <triggers> <hudson.triggers.SCMTrigger> <spec> H 7-21/1 * * * </spec> <ignorePostCommitHooks> false </ignorePostCommitHooks> </hudson.triggers.SCMTrigger> </triggers>

          Mark Waite added a comment -

          Since it is the same behavior whether you downgrade or not, then it is probably not an instance of the bug I've been trying to isolate. The bug I'm trying to isolate first appeared in the transition from 2.3.4 to 2.3.5. You may need to debug the plugin to identify why it is building continually.

          If you can't do that, it would be a great help if you can provide a repeatable set of steps so that I can construct a job which has the same behavior, without requiring that I install and configure a Gerrit server. I can't promise to investigate it (plugin maintenance is done on my personal time), but I'm much more motivated to help when a bug is submitted with detailed steps which can reconstruct the failure case.

          Mark Waite added a comment - Since it is the same behavior whether you downgrade or not, then it is probably not an instance of the bug I've been trying to isolate. The bug I'm trying to isolate first appeared in the transition from 2.3.4 to 2.3.5. You may need to debug the plugin to identify why it is building continually. If you can't do that, it would be a great help if you can provide a repeatable set of steps so that I can construct a job which has the same behavior, without requiring that I install and configure a Gerrit server. I can't promise to investigate it (plugin maintenance is done on my personal time), but I'm much more motivated to help when a bug is submitted with detailed steps which can reconstruct the failure case.

          Will Saxon added a comment -

          I can try to debug it myself. If I run into problems w/ that I will try to replicate with a test repo on github and post steps here.

          Will Saxon added a comment - I can try to debug it myself. If I run into problems w/ that I will try to replicate with a test repo on github and post steps here.

          Mark Waite added a comment -

          The bug that appeared in the transition from 2.3.4 to 2.3.5 is now resolved in git plugin 2.4.0 (which requires git client plugin 1.18.0). You may want to try 2.4.0 (with 1.18.0) to see if your problem is resolved as a side effect of the fixes in 2.4.0.

          Mark Waite added a comment - The bug that appeared in the transition from 2.3.4 to 2.3.5 is now resolved in git plugin 2.4.0 (which requires git client plugin 1.18.0). You may want to try 2.4.0 (with 1.18.0) to see if your problem is resolved as a side effect of the fixes in 2.4.0.

          Mark Waite added a comment -

          Resolved due to no feedback on the "is it fixed in 2.4.0" question in 2 months

          Mark Waite added a comment - Resolved due to no feedback on the "is it fixed in 2.4.0" question in 2 months

            ndeloof Nicolas De Loof
            jasonrobinson jason robinson
            Votes:
            2 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: