• Icon: Bug Bug
    • Resolution: Not A Defect
    • Icon: Major Major
    • core

      Mercurial scm polling stops after running once after restart.
      Jenkins version: 1.570 (can only choose 'current' in Jira)

      config.xml scm:

      <scm class="hudson.plugins.mercurial.MercurialSCM" plugin="mercurial@1.50">
          <installation>(Default)</installation>
          <source>http://host/path/project</source>
          <modules/>
          <revisionType>BRANCH</revisionType>
          <revision>stable</revision>
          <clean>true</clean>
          <browser class="hudson.plugins.mercurial.browser.HgWeb">
              <url>http://host/path/project/</url>
          </browser>
          <credentialsId/>
          <disableChangeLog>false</disableChangeLog>
      </scm>
      

      config.xml triggers:

      <triggers>
          <hudson.triggers.SCMTrigger>
              <spec>H 3 * * 1-6</spec>
              <ignorePostCommitHooks>false</ignorePostCommitHooks>
          </hudson.triggers.SCMTrigger>
      </triggers>
      

      scm-polling.log (today is 14-aug-2014, so it did not run for a couple of days already):

      Started on 9-aug-2014 3:27:01
      Polling SCM changes on master
      [workspace] $ hg pull --rev stable
      pulling from http://host/path/project
      no changes found
      [workspace] $ hg log --rev stable --template {node}
      [workspace] $ hg log --rev stable --template {rev}
      Done. Took 1,3 seconden
      No changes
      

          [JENKINS-24244] Scheduled polling not run

          I've checked the scm-polling.log from the project, but this was only containing info from the latest manual trigger performed by /job/foo/polling.

          Rick Oosterholt added a comment - I've checked the scm-polling.log from the project, but this was only containing info from the latest manual trigger performed by /job/foo/polling.

          Daniel Beck added a comment - - edited

          Alright, not a problem. If you were on FINER logging it'd tell you that it's not the correct time to poll...

          Check the following for all jobs that should get triggered but aren't:

          • Go to the job config page, enter the text field for the cron spec (H 3 * * 1-6), and leave it. Recent Jenkins versions tell you what the previous and next polling time is below the text field if you do that.
          • Shortly after the next (project specific) polling time (within two minutes) the polling log should have been updated, and the log should contain something like "cron triggered hudson.model.FreeStyleProject@deadbeef[Project Name]".

          Also add the logger hudson.triggers.SCMTrigger at FINE to your log recorder, there may be something interesting there as well.

          Daniel Beck added a comment - - edited Alright, not a problem. If you were on FINER logging it'd tell you that it's not the correct time to poll... Check the following for all jobs that should get triggered but aren't: Go to the job config page, enter the text field for the cron spec (H 3 * * 1-6), and leave it. Recent Jenkins versions tell you what the previous and next polling time is below the text field if you do that. Shortly after the next (project specific) polling time (within two minutes) the polling log should have been updated, and the log should contain something like "cron triggered hudson.model.FreeStyleProject@deadbeef [Project Name] ". Also add the logger hudson.triggers.SCMTrigger at FINE to your log recorder, there may be something interesting there as well.

          Rick Oosterholt added a comment - - edited

          Setting the hudson.triggers.Trigger log to FINER shows indeed:

          ...
          cron checking 20-8-14 2:03
          ...
          aug 20, 2014 2:03:01 PM FINE hudson.triggers.Trigger
          cron checking hudson.model.FreeStyleProject@64ae3cef[Build STABLE] with spec ‘H 3 * * 1-6’
          aug 20, 2014 2:02:01 PM FINER hudson.triggers.Trigger
          did not trigger hudson.model.FreeStyleProject@64ae3cef[Build STABLE]
          ...
          

          enter/leaving the cron schedule field in config shows the correct info (I've translated it to EN):

          Would last have run at Wednesday August 20 2014 3:00:08 CEST; would next run at Thursday 21 August 2014 3:00:08  CEST.
          

          I'll check for the hudson.triggers.SCMTrigger logging tomorrow (do not want to change the cron spec; might have something to to with the issue)

          Rick Oosterholt added a comment - - edited Setting the hudson.triggers.Trigger log to FINER shows indeed: ... cron checking 20-8-14 2:03 ... aug 20, 2014 2:03:01 PM FINE hudson.triggers.Trigger cron checking hudson.model.FreeStyleProject@64ae3cef[Build STABLE] with spec ‘H 3 * * 1-6’ aug 20, 2014 2:02:01 PM FINER hudson.triggers.Trigger did not trigger hudson.model.FreeStyleProject@64ae3cef[Build STABLE] ... enter/leaving the cron schedule field in config shows the correct info (I've translated it to EN): Would last have run at Wednesday August 20 2014 3:00:08 CEST; would next run at Thursday 21 August 2014 3:00:08 CEST. I'll check for the hudson.triggers.SCMTrigger logging tomorrow (do not want to change the cron spec; might have something to to with the issue)

          Jesse Glick added a comment -

          The SCMTrigger log is irrelevant; this is a problem in Trigger. There should have been a message at FINE about

          cron checking <some date here>

          before a list of pairs of messages per triggerable job as shown in your last comment. This may help explain why the spec was not considered to match at the given time.

          This whole block is logged every minute. You have to look at the check done at the exact minute that the config hint indicates the job should have been scheduled, to understand why it was not. Unfortunately the UI logger does not capture more than 100 messages, so this would be long gone by the time you wake up. So you need to install the Support Core plugin so that the persisted log file has the right record. JENKINS-22431 would help here.

          Jesse Glick added a comment - The SCMTrigger log is irrelevant; this is a problem in Trigger . There should have been a message at FINE about cron checking <some date here> before a list of pairs of messages per triggerable job as shown in your last comment. This may help explain why the spec was not considered to match at the given time. This whole block is logged every minute. You have to look at the check done at the exact minute that the config hint indicates the job should have been scheduled, to understand why it was not. Unfortunately the UI logger does not capture more than 100 messages, so this would be long gone by the time you wake up. So you need to install the Support Core plugin so that the persisted log file has the right record. JENKINS-22431 would help here.

          Well, that might be my bad. I just copied the relevant part of the log. I really should have mentioned that ...

          the log is quite large and does contain cron checking 20-8-14 2:03

          the hudson.triggers.SCMTrigger is completely empty.

          what is the next step to do to help you guys get the right info?

          Rick Oosterholt added a comment - Well, that might be my bad. I just copied the relevant part of the log. I really should have mentioned that ... the log is quite large and does contain cron checking 20-8-14 2:03 the hudson.triggers.SCMTrigger is completely empty. what is the next step to do to help you guys get the right info?

          Daniel Beck added a comment -

          While you should follow the advice by Jesse and install Support Core, the following will make it easier to access older log messages:


          Try setting the System property hudson.util.RingBufferLogHandler.defaultSize to a large value. Change the Jenkins java call (if using bundled Jetty container) to add it like so:

          java -Dhudson.util.RingBufferLogHandler.defaultSize=10000 -jar jenkins.war

          Then the UI will show more messages.

          Requires a full restart of Jenkins, and not via /safeRestart. Shut it down, then start again.


          The log entries that are relevant are those logged between 2:59 and 3:05 or so. The rest is expected to not matter. "did not trigger" is expected every minute of every day, except at 3.


          There's this weird behavior in Jenkins that skips any SCM polling while Jenkins is in "quiet down" mode (the red banner "Jenkins is preparing to shut down"). Make sure you're not in this mode (one of the backup plugins does this IIRC) at the scheduled time.

          Daniel Beck added a comment - While you should follow the advice by Jesse and install Support Core, the following will make it easier to access older log messages: Try setting the System property hudson.util.RingBufferLogHandler.defaultSize to a large value. Change the Jenkins java call (if using bundled Jetty container) to add it like so: java -Dhudson.util.RingBufferLogHandler.defaultSize=10000 -jar jenkins.war Then the UI will show more messages. Requires a full restart of Jenkins, and not via /safeRestart . Shut it down, then start again. The log entries that are relevant are those logged between 2:59 and 3:05 or so. The rest is expected to not matter. "did not trigger" is expected every minute of every day, except at 3. There's this weird behavior in Jenkins that skips any SCM polling while Jenkins is in "quiet down" mode (the red banner "Jenkins is preparing to shut down"). Make sure you're not in this mode (one of the backup plugins does this IIRC) at the scheduled time.

          There's this weird behavior in Jenkins that skips any SCM polling while Jenkins is in "quiet down" mode (the red banner "Jenkins is preparing to shut down"). Make sure you're not in this mode (one of the backup plugins does this IIRC) at the scheduled time.

          That's it! I'm using Exclusive Execution Plugin which puts Jenkins in "shutdown mode".

          Thanks for your help!
          Has anyone an idea to ensure a specific project is executed "exclusively" in Jenkins?

          Rick Oosterholt added a comment - There's this weird behavior in Jenkins that skips any SCM polling while Jenkins is in "quiet down" mode (the red banner "Jenkins is preparing to shut down"). Make sure you're not in this mode (one of the backup plugins does this IIRC) at the scheduled time. That's it! I'm using Exclusive Execution Plugin which puts Jenkins in "shutdown mode". Thanks for your help! Has anyone an idea to ensure a specific project is executed "exclusively" in Jenkins?

          Daniel Beck added a comment -

          Not a defect, as brain dead as the current polling behavior is. (My own Jenkins builds remove that part as it's too annoying.)

          Please direct further questions to the jenkinsci-users list, or #jenkins on Freenode.

          Daniel Beck added a comment - Not a defect, as brain dead as the current polling behavior is. (My own Jenkins builds remove that part as it's too annoying.) Please direct further questions to the jenkinsci-users list, or #jenkins on Freenode.

          Code changed in jenkins
          User: Jesse Glick
          Path:
          core/src/main/java/hudson/triggers/SCMTrigger.java
          http://jenkins-ci.org/commit/jenkins/608517e187cb5bd1566b1c3728a4df0f7ac4dd5c
          Log:
          JENKINS-24244 If we are skipping polling due to quietingDown, at least record this fact in the system log.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/hudson/triggers/SCMTrigger.java http://jenkins-ci.org/commit/jenkins/608517e187cb5bd1566b1c3728a4df0f7ac4dd5c Log: JENKINS-24244 If we are skipping polling due to quietingDown, at least record this fact in the system log.

          dogfood added a comment -

          Integrated in jenkins_main_trunk #3642
          JENKINS-24244 If we are skipping polling due to quietingDown, at least record this fact in the system log. (Revision 608517e187cb5bd1566b1c3728a4df0f7ac4dd5c)

          Result = SUCCESS
          Jesse Glick : 608517e187cb5bd1566b1c3728a4df0f7ac4dd5c
          Files :

          • core/src/main/java/hudson/triggers/SCMTrigger.java

          dogfood added a comment - Integrated in jenkins_main_trunk #3642 JENKINS-24244 If we are skipping polling due to quietingDown, at least record this fact in the system log. (Revision 608517e187cb5bd1566b1c3728a4df0f7ac4dd5c) Result = SUCCESS Jesse Glick : 608517e187cb5bd1566b1c3728a4df0f7ac4dd5c Files : core/src/main/java/hudson/triggers/SCMTrigger.java

            Unassigned Unassigned
            r_oosterholt Rick Oosterholt
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: