Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-61515

Git SCM polling fails with skipDefaultCheckout() in pipeline

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Cannot Reproduce
    • Labels:
      None
    • Environment:
      latest production version of Jenkins & all plugins. Running on Linux (SLES) server.
    • Similar Issues:

      Description

      In job's pipeline, I have the following:

       

      options { skipDefaultCheckout() }

       

      Git SCM polling never finds any changes for the repository. Same job, but without skipDefaultCheckout() in the pipeline causes Git polling to work as expected.

        Attachments

        1. config.xml
          2 kB
        2. Jenkinsfile
          0.2 kB

          Activity

          Hide
          markewaite Mark Waite added a comment -

          Which type of Pipeline job is it?

          • Pipeline job that defines the Pipeline in the job definition
          • Pipeline job that gets the job definition from SCM
          • Pipeline job defined by multibranch pipeline
          • Pipeline job defined by an organization folder
          Show
          markewaite Mark Waite added a comment - Which type of Pipeline job is it? Pipeline job that defines the Pipeline in the job definition Pipeline job that gets the job definition from SCM Pipeline job defined by multibranch pipeline Pipeline job defined by an organization folder
          Hide
          intelmatt Matt G added a comment -
          • Pipeline job that gets the job definition from SCM
          Show
          intelmatt Matt G added a comment - Pipeline job that gets the job definition from SCM
          Hide
          markewaite Mark Waite added a comment -

          I can't duplicate the issue. Steps I took to try to duplicate the problem:

          1. Define a pipeline job JENKINS-61515-using-JENKINS-26100-repo as a Pipeline job that reads its definition from SCM with 'Poll SCM' configured with no schedule so that it will respond to notifyCommit but not perform polls itself
          2. Run the pipeline job once
          3. Commit a change to the branch monitored by that job
          4. Invoke the notifyCommit to cause the job to poll, confirm the job detects the change and runs
          5. Commit a change to the branch monitored by that job
          6. Press the 'Poll Now' button on the job definition (provided by the Poll Now plugin), confirm the job detects the change and runs

          I'll need more details to duplicate the issue. You could upload the job definition XML file or could compare your job definition with my job definition and describe the differences in detail.

          Show
          markewaite Mark Waite added a comment - I can't duplicate the issue. Steps I took to try to duplicate the problem: Define a pipeline job JENKINS-61515-using-JENKINS-26100-repo as a Pipeline job that reads its definition from SCM with 'Poll SCM' configured with no schedule so that it will respond to notifyCommit but not perform polls itself Run the pipeline job once Commit a change to the branch monitored by that job Invoke the notifyCommit to cause the job to poll, confirm the job detects the change and runs Commit a change to the branch monitored by that job Press the 'Poll Now' button on the job definition (provided by the Poll Now plugin), confirm the job detects the change and runs I'll need more details to duplicate the issue. You could upload the job definition XML file or could compare your job definition with my job definition and describe the differences in detail.
          Hide
          intelmatt Matt G added a comment - - edited

          In #1, I set it up with a regular (1 minute) polling interval. Then I built the project manually by clicking 'Build now'. I did not use notifyCommit or 'Poll Now'. New changes pushed to the repository never get detected. The polling log timestamps do update, but it never executes any Git commands; it just says no changes detected. I've attached the relevant config.xml for the job and the Jenkinsfile from the repository in question so you can see exactly how it is set up to reproduce the issue.

          Show
          intelmatt Matt G added a comment - - edited In #1, I set it up with a regular (1 minute) polling interval. Then I built the project manually by clicking 'Build now'. I did not use notifyCommit or 'Poll Now'. New changes pushed to the repository never get detected. The polling log timestamps do update, but it never executes any Git commands; it just says no changes detected. I've attached the relevant config.xml for the job and the Jenkinsfile from the repository in question so you can see exactly how it is set up to reproduce the issue.
          Hide
          intelmatt Matt G added a comment -

          This is what the polling log looks like, even when I know new changes were pushed to the repository:

          Started on Mar 30, 2020, 12:57:00 PM
          Done. Took 1 ms
          No changes
          
          Show
          intelmatt Matt G added a comment - This is what the polling log looks like, even when I know new changes were pushed to the repository: Started on Mar 30, 2020, 12:57:00 PM Done. Took 1 ms No changes
          Hide
          markewaite Mark Waite added a comment - - edited

          I'm unable to duplicate the problem, even with your config.xml file.

          I compared the config.xml you uploaded with the config.xml of my test job. The differences included:

          • Your config.xml polls every minute. Mine is configured with polling but with no schedule so that the job will respond to notifyCommit without wasting the effort polling
          • Your config.xml file is XML 1.0, mine is 1.1. That difference should be irrelevant
          • Your config.xml references /path/to/my/repository.git. My config.xml references my bug check repository. I'm using a network based git server and I assume you are also using a network based git server. If instead you are using file system access to a repository on your Jenkins server, you might try using a network protocol to access the git repository rather than file system access. I rarely test direct file system access to git repositories because it would limit jobs to only run on the master

          Based on the differences between my config.xml and yours, I configured my job to poll every minute. I committed a change to the repository, waited one minute, and confirmed that the change was detected and the job ran. I committed two changes to the repository, waited for the next polling, and confirmed that the changes were detected and the job ran. I committed three changes to the repository, waited for the next poll, and confirmed the changes were detected.

          I also confirmed that if I use a notifyCommit hook in the central git repository to alert the Jenkins server that it should poll, the changes are detected as soon as they are applied, without the overhead of polling once a minute. My central git repository is a small computer hosting ssh based git repositories. In the hooks directory of that repository, I have an executable script named post-receive like this:

          #!/bin/bash
          
          curl -s http://mark-pc2.markwaite.net/git/notifyCommit?url=https://github.com/MarkEWaite/jenkins-bugs
          

          That script is automatically executed by git each time a commit arrives at that repository. That script is an example of the concept that Kohsuke Kawaguchi described in his blog post, "polling must die". It may be worth experimenting with notifyCommit to avoid the overhead of polling and the delay of polling.

          Alternately, if you're using a git hosting system like GitHub, Bitbucket, GitLab, or Gitea, they offer web hooks which can do the same thing. The web hooks are often managed by Jenkins plugins focused on the specific provider.

          There is another possible difference between the job you're running and the job that I'm running. My Jenkinsfile is probably different from yours. I would hope that whatever differences there are between those two files, they would be irrelevant to this case. However, you might experiment with a variant of my Jenkinsfile in your environment, just to see if the contents of my Jenkinsfile make polling work, while yours does not.

          Show
          markewaite Mark Waite added a comment - - edited I'm unable to duplicate the problem, even with your config.xml file. I compared the config.xml you uploaded with the config.xml of my test job . The differences included: Your config.xml polls every minute. Mine is configured with polling but with no schedule so that the job will respond to notifyCommit without wasting the effort polling Your config.xml file is XML 1.0, mine is 1.1. That difference should be irrelevant Your config.xml references /path/to/my/repository.git. My config.xml references my bug check repository . I'm using a network based git server and I assume you are also using a network based git server. If instead you are using file system access to a repository on your Jenkins server, you might try using a network protocol to access the git repository rather than file system access. I rarely test direct file system access to git repositories because it would limit jobs to only run on the master Based on the differences between my config.xml and yours, I configured my job to poll every minute. I committed a change to the repository, waited one minute, and confirmed that the change was detected and the job ran. I committed two changes to the repository, waited for the next polling, and confirmed that the changes were detected and the job ran. I committed three changes to the repository, waited for the next poll, and confirmed the changes were detected. I also confirmed that if I use a notifyCommit hook in the central git repository to alert the Jenkins server that it should poll, the changes are detected as soon as they are applied, without the overhead of polling once a minute. My central git repository is a small computer hosting ssh based git repositories. In the hooks directory of that repository, I have an executable script named post-receive like this: #!/bin/bash curl -s http://mark-pc2.markwaite.net/git/notifyCommit?url=https://github.com/MarkEWaite/jenkins-bugs That script is automatically executed by git each time a commit arrives at that repository. That script is an example of the concept that Kohsuke Kawaguchi described in his blog post, " polling must die ". It may be worth experimenting with notifyCommit to avoid the overhead of polling and the delay of polling. Alternately, if you're using a git hosting system like GitHub, Bitbucket, GitLab, or Gitea, they offer web hooks which can do the same thing. The web hooks are often managed by Jenkins plugins focused on the specific provider. There is another possible difference between the job you're running and the job that I'm running. My Jenkinsfile is probably different from yours. I would hope that whatever differences there are between those two files, they would be irrelevant to this case. However, you might experiment with a variant of my Jenkinsfile in your environment, just to see if the contents of my Jenkinsfile make polling work, while yours does not.
          Hide
          markewaite Mark Waite added a comment -

          Matt G any more information you can share that would allow the issue to be duplicated by me (or others)?

          Show
          markewaite Mark Waite added a comment - Matt G any more information you can share that would allow the issue to be duplicated by me (or others)?
          Hide
          intelmatt Matt G added a comment -

          Darn... No, sorry, no additional info. We do try to use webhooks where possible, but in some cases are still relying on polling. No problem to close this issue as non-reproducible.

          Show
          intelmatt Matt G added a comment - Darn... No, sorry, no additional info. We do try to use webhooks where possible, but in some cases are still relying on polling. No problem to close this issue as non-reproducible.

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            intelmatt Matt G
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: