-
Bug
-
Resolution: Unresolved
-
Critical
-
Jenkins 1.588, CentOS release 6.3 (Final)
Plugins:
active-directory, version: 1.38
analysis-collector, version: 1.41
analysis-core, version: 1.65
ant, version: 1.2
antisamy-markup-formatter, version: 1.3
anything-goes-formatter, version: 1.0
audit-trail, version: 2.1
bugzilla, version: 1.5
build-blocker-plugin, version: 1.4.1
build-failure-analyzer, version: 1.10.3
build-flow-plugin, version: 0.16
buildgraph-view, version: 1.1.1
build-monitor-plugin, version: 1.6+build.132
build-pipeline-plugin, version: 1.4.4
build-timeout, version: 1.14
buildtriggerbadge, version: 1.3
build-user-vars-plugin, version: 1.4
chucknorris, version: 0.5
claim, version: 2.3
cobertura, version: 1.9.6
compact-columns, version: 1.10
conditional-buildstep, version: 1.3.3
console-column-plugin, version: 1.5
credentials, version: 1.18
cvs, version: 2.12
database, version: 1.3
discard-old-build, version: 1.05
disk-usage, version: 0.24
docker-plugin, version: 0.8
downstream-buildview, version: 1.9
durable-task, version: 0.7
email-ext, version: 2.38.2
emma, version: 1.29
emmacoveragecolumn, version: 0.0.4
envfile, version: 1.2
envinject, version: 1.90
environment-script, version: 1.1.2
extended-choice-parameter, version: 0.34
external-monitor-job, version: 1.2
extra-columns, version: 1.14
fail-the-build-plugin, version: 1.0
findbugs, version: 4.57
gatling, version: 1.0.3
git, version: 2.3
git-client, version: 1.11.1
global-build-stats, version: 1.3
greenballs, version: 1.14
groovy, version: 1.24
htmlpublisher, version: 1.3
http_request, version: 1.8.5
jacoco, version: 1.0.16
javadoc, version: 1.3
jdepend, version: 1.2.4
jenkins-multijob-plugin, version: 1.15
jenkins-reviewbot, version: 2.1.2
jobConfigHistory, version: 2.9
job-dsl, version: 1.26
job-restrictions, version: 0.3
jquery, version: 1.7.2-1
junit, version: 1.2
junit-attachments, version: 1.3
ldap, version: 1.11
mailer, version: 1.12
mapdb-api, version: 1.0.6.0
mask-passwords, version: 2.7.2
matrix-auth, version: 1.2
matrix-project, version: 1.4
maven-plugin, version: 2.7.1
measurement-plots, version: 0.1
mesos, version: 0.5.0
monitoring, version: 1.53.1
naginator, version: 1.12
pam-auth, version: 1.2
parameterized-trigger, version: 2.25
patch-parameter, version: 1.2
performance, version: 1.11
plot, version: 1.8
pmd, version: 3.39
postbuild-task, version: 1.8
progress-bar-column-plugin, version: 1.0
publish-over-ssh, version: 1.12
rich-text-publisher-plugin, version: 1.3
run-condition, version: 1.0
saferestart, version: 0.3
sbt, version: 1.4
sbt-vars, version: 1.5-SNAPSHOT (private-05/05/2014 16:31-dev)
scm-api, version: 0.2
scoverage, version: 1.0.2
scp, version: 1.8
show-build-parameters, version: 1.0
sitemonitor, version: 0.4
sonar, version: 2.1
ssh, version: 2.4
ssh-agent, version: 1.5
ssh-credentials, version: 1.10
ssh-slaves, version: 1.9
started-by-envvar, version: 1.0
subversion, version: 2.4.4
swarm, version: 1.21
template-project, version: 1.4.2
testInProgress, version: 1.3
testng-plugin, version: 1.9.1
test-stability, version: 1.0
text-finder, version: 1.10
timestamper, version: 1.5.14
token-macro, version: 1.10
translation, version: 1.12
view-job-filters, version: 1.26
vsphere-cloud, version: 1.1.12
warnings, version: 4.43
windows-slaves, version: 1.0
ws-cleanup, version: 0.24
xvfb, version: 1.0.13
xvnc, version: 1.18
Jenkins 1.588, CentOS release 6.3 (Final) Plugins: active-directory, version: 1.38 analysis-collector, version: 1.41 analysis-core, version: 1.65 ant, version: 1.2 antisamy-markup-formatter, version: 1.3 anything-goes-formatter, version: 1.0 audit-trail, version: 2.1 bugzilla, version: 1.5 build-blocker-plugin, version: 1.4.1 build-failure-analyzer, version: 1.10.3 build-flow-plugin, version: 0.16 buildgraph-view, version: 1.1.1 build-monitor-plugin, version: 1.6+build.132 build-pipeline-plugin, version: 1.4.4 build-timeout, version: 1.14 buildtriggerbadge, version: 1.3 build-user-vars-plugin, version: 1.4 chucknorris, version: 0.5 claim, version: 2.3 cobertura, version: 1.9.6 compact-columns, version: 1.10 conditional-buildstep, version: 1.3.3 console-column-plugin, version: 1.5 credentials, version: 1.18 cvs, version: 2.12 database, version: 1.3 discard-old-build, version: 1.05 disk-usage, version: 0.24 docker-plugin, version: 0.8 downstream-buildview, version: 1.9 durable-task, version: 0.7 email-ext, version: 2.38.2 emma, version: 1.29 emmacoveragecolumn, version: 0.0.4 envfile, version: 1.2 envinject, version: 1.90 environment-script, version: 1.1.2 extended-choice-parameter, version: 0.34 external-monitor-job, version: 1.2 extra-columns, version: 1.14 fail-the-build-plugin, version: 1.0 findbugs, version: 4.57 gatling, version: 1.0.3 git, version: 2.3 git-client, version: 1.11.1 global-build-stats, version: 1.3 greenballs, version: 1.14 groovy, version: 1.24 htmlpublisher, version: 1.3 http_request, version: 1.8.5 jacoco, version: 1.0.16 javadoc, version: 1.3 jdepend, version: 1.2.4 jenkins-multijob-plugin, version: 1.15 jenkins-reviewbot, version: 2.1.2 jobConfigHistory, version: 2.9 job-dsl, version: 1.26 job-restrictions, version: 0.3 jquery, version: 1.7.2-1 junit, version: 1.2 junit-attachments, version: 1.3 ldap, version: 1.11 mailer, version: 1.12 mapdb-api, version: 1.0.6.0 mask-passwords, version: 2.7.2 matrix-auth, version: 1.2 matrix-project, version: 1.4 maven-plugin, version: 2.7.1 measurement-plots, version: 0.1 mesos, version: 0.5.0 monitoring, version: 1.53.1 naginator, version: 1.12 pam-auth, version: 1.2 parameterized-trigger, version: 2.25 patch-parameter, version: 1.2 performance, version: 1.11 plot, version: 1.8 pmd, version: 3.39 postbuild-task, version: 1.8 progress-bar-column-plugin, version: 1.0 publish-over-ssh, version: 1.12 rich-text-publisher-plugin, version: 1.3 run-condition, version: 1.0 saferestart, version: 0.3 sbt, version: 1.4 sbt-vars, version: 1.5-SNAPSHOT (private-05/05/2014 16:31-dev) scm-api, version: 0.2 scoverage, version: 1.0.2 scp, version: 1.8 show-build-parameters, version: 1.0 sitemonitor, version: 0.4 sonar, version: 2.1 ssh, version: 2.4 ssh-agent, version: 1.5 ssh-credentials, version: 1.10 ssh-slaves, version: 1.9 started-by-envvar, version: 1.0 subversion, version: 2.4.4 swarm, version: 1.21 template-project, version: 1.4.2 testInProgress, version: 1.3 testng-plugin, version: 1.9.1 test-stability, version: 1.0 text-finder, version: 1.10 timestamper, version: 1.5.14 token-macro, version: 1.10 translation, version: 1.12 view-job-filters, version: 1.26 vsphere-cloud, version: 1.1.12 warnings, version: 4.43 windows-slaves, version: 1.0 ws-cleanup, version: 0.24 xvfb, version: 1.0.13 xvnc, version: 1.18
Jobs configured with a periodic trigger stop being triggered after Jenkins has been running for a few days.
There are no indications of a problem in the log. The last/next run times shown in the configure page look right. Manually triggering the job works, though the periodic trigger doesn't resume until Jenkins is restarted.
The Jenkins and plugin versions are what is currently installed though we've seen the problem for some months with different recent versions of Jenkins and various plugins. It might be that the problem has not been noticed more often because of more frequent Jenkins restarts.
I've attached a thread dump taken while periodic triggers haven't been working for over a day.
- is related to
-
-
- Closed
-
[JENKINS-25704] Scheduled triggering stops working until restart
David Resnick
created issue -
Could you set up a log recorder for the logger hudson.triggers.Trigger on level FINE to see what gets logged?
David Resnick
added a comment - The problem has finally recurred after I configured the logger. The logs show that the trigger just stops at some point.
This is one of the cron jobs:
Nov 26, 2014 2:36:00 AM FINE hudson.triggers.Trigger cron checking hudson.model.FreeStyleProject@4f4e362a[Reviewbot2] with spec ‘H/5 * * * *’ Nov 26, 2014 2:37:00 AM FINE hudson.triggers.Trigger cron checking hudson.model.FreeStyleProject@4f4e362a[Reviewbot2] with spec ‘H/5 * * * *’ Nov 26, 2014 2:38:00 AM FINE hudson.triggers.Trigger cron checking hudson.model.FreeStyleProject@4f4e362a[Reviewbot2] with spec ‘H/5 * * * *’ Nov 26, 2014 2:38:00 AM CONFIG hudson.triggers.Trigger cron triggered hudson.model.FreeStyleProject@4f4e362a[Reviewbot2] Nov 26, 2014 2:39:00 AM FINE hudson.triggers.Trigger cron checking hudson.model.FreeStyleProject@4f4e362a[Reviewbot2] with spec ‘H/5 * * * *’ Nov 26, 2014 2:40:00 AM FINE hudson.triggers.Trigger cron checking hudson.model.FreeStyleProject@4f4e362a[Reviewbot2] with spec ‘H/5 * * * *’ Nov 26, 2014 2:41:00 AM FINE hudson.triggers.Trigger cron checking hudson.model.FreeStyleProject@4f4e362a[Reviewbot2] with spec ‘H/5 * * * *’
David Resnick
added a comment - The problem has finally recurred after I configured the logger. The logs show that the trigger just stops at some point.
This is one of the cron jobs:
Nov 26, 2014 2:36:00 AM FINE hudson.triggers.Trigger
cron checking hudson.model.FreeStyleProject@4f4e362a[Reviewbot2] with spec ‘H/5 * * * *’
Nov 26, 2014 2:37:00 AM FINE hudson.triggers.Trigger
cron checking hudson.model.FreeStyleProject@4f4e362a[Reviewbot2] with spec ‘H/5 * * * *’
Nov 26, 2014 2:38:00 AM FINE hudson.triggers.Trigger
cron checking hudson.model.FreeStyleProject@4f4e362a[Reviewbot2] with spec ‘H/5 * * * *’
Nov 26, 2014 2:38:00 AM CONFIG hudson.triggers.Trigger
cron triggered hudson.model.FreeStyleProject@4f4e362a[Reviewbot2]
Nov 26, 2014 2:39:00 AM FINE hudson.triggers.Trigger
cron checking hudson.model.FreeStyleProject@4f4e362a[Reviewbot2] with spec ‘H/5 * * * *’
Nov 26, 2014 2:40:00 AM FINE hudson.triggers.Trigger
cron checking hudson.model.FreeStyleProject@4f4e362a[Reviewbot2] with spec ‘H/5 * * * *’
Nov 26, 2014 2:41:00 AM FINE hudson.triggers.Trigger
cron checking hudson.model.FreeStyleProject@4f4e362a[Reviewbot2] with spec ‘H/5 * * * *’
No. AFAIK there are no changes taking place in the environment. The last time triggers stopped was in the middle of the night.
David Resnick
added a comment - - edited No. AFAIK there are no changes taking place in the environment. The last time triggers stopped was in the middle of the night. No building is done on the master, though a couple jobs do run there (e.g. the Reviewbot2 one in the log above).
David Resnick
added a comment - - edited No building is done on the master, though a couple jobs do run there (e.g. the Reviewbot2 one in the log above). David Resnick
added a comment - I'll be able to check tomorrow. I did check some logs on the master today (e.g. cron) and saw that there were no jumps of the date/time. Timestamps in jenkins.log are also predictable, after the triggers stop too.
David Resnick
added a comment - I'll be able to check tomorrow. I did check some logs on the master today (e.g. cron) and saw that there were no jumps of the date/time. Timestamps in jenkins.log are also predictable, after the triggers stop too. David Resnick
added a comment - ntpd is running on the server. It seems to be functioning OK. Time on the server is synced, ntpd hasn't logged anything.
David Resnick
added a comment - ntpd is running on the server. It seems to be functioning OK. Time on the server is synced, ntpd hasn't logged anything.
Could you search the Jenkins log for "Timer task foo failed", foo being the name of a task?