-
Bug
-
Resolution: Fixed
-
Major
-
Jenkins 2.46.3 LTS (integrated Container) on Windows Server 2008 R2
Installed from WAR-File
Oracle Server JRE8u131 64Bit
JIRA v7.1.9
Plugins:
ace-editor 1.1
active-directory 2.4
analysis-core 1.87
ant 1.5
antisamy-markup-formatter 1.5
authentication-tokens 1.3
bouncycastle-api 2.16.1
branch-api 2.0.9
build-name-setter 1.6.5
cccc 0.6
chucknorris 1.0
cloudbees-folder 6.0.4
copy-to-slave 1.4.4
copyartifact 1.38.1
cppcheck 1.21
credentials 2.1.13
credentials-binding 1.11
cvs 2.13
disk-usage 0.28
display-url-api 2.0
docker-commons 1.6
docker-workflow 1.11
doxygen 0.18
durable-task 1.13
external-monitor-job 1.7
files-found-trigger 1.5
git 3.3.0
git-client 2.4.6
git-server 1.7
github 1.27.0
github-api 1.85.1
gradle 1.26
handlebars 1.1.1
htmlpublisher 1.13
icon-shim 2.0.3
jackson2-api 2.7.3
javadoc 1.4
jira 2.3.1
jobConfigHistory 2.16
jquery-detached 1.2.1
junit 1.20
lastfailureversioncolumn 1.1
lastsuccessversioncolumn 1.1
ldap 1.15
mailer 1.20
mapdb-api 1.0.9.0
matrix-auth 1.6
matrix-project 1.11
maven-plugin 2.16
mercurial 1.60
momentjs 1.1.1
monitoring 1.67.0
multiple-scms 0.6
notification 1.11
pam-auth 1.3
persona 2.4 false
pipeline-build-step 2.5
pipeline-graph-analysis 1.4
pipeline-input-step 2.7
pipeline-milestone-step 1.3.1
pipeline-model-api 1.1.5
pipeline-model-declarative-agent 1.1.1
pipeline-model-definition 1.1.5
pipeline-model-extensions 1.1.5
pipeline-rest-api 2.8
pipeline-stage-step 2.2
pipeline-stage-tags-metadata 1.1.5
pipeline-stage-view 2.8
plain-credentials 1.4
plot 1.11
project-description-setter 1.1
release 2.7
repo 1.10.7 false
resource-disposer 0.6
role-strategy 2.5.0
scm-api 2.1.1
script-security 1.28
ssh-credentials 1.13
ssh-slaves 1.17
structs 1.7
subversion 2.7.2
svn-release-mgr 1.2
throttle-concurrents 2.0.1
token-macro 2.1
tracking-svn 1.1
translation 1.15
viewVC 1.7
warnings 4.62
windows-slaves 1.3.1
workflow-aggregator 2.5
workflow-api 2.17
workflow-basic-steps 2.5
workflow-cps 2.34
workflow-cps-global-lib 2.8
workflow-durable-task-step 2.11
workflow-job 2.11
workflow-multibranch 2.15
workflow-scm-step 2.4
workflow-step-api 2.11
workflow-support 2.14
ws-cleanup 0.33
xunit 1.102Jenkins 2.46.3 LTS (integrated Container) on Windows Server 2008 R2 Installed from WAR-File Oracle Server JRE8u131 64Bit JIRA v7.1.9 Plugins: ace-editor 1.1 active-directory 2.4 analysis-core 1.87 ant 1.5 antisamy-markup-formatter 1.5 authentication-tokens 1.3 bouncycastle-api 2.16.1 branch-api 2.0.9 build-name-setter 1.6.5 cccc 0.6 chucknorris 1.0 cloudbees-folder 6.0.4 copy-to-slave 1.4.4 copyartifact 1.38.1 cppcheck 1.21 credentials 2.1.13 credentials-binding 1.11 cvs 2.13 disk-usage 0.28 display-url-api 2.0 docker-commons 1.6 docker-workflow 1.11 doxygen 0.18 durable-task 1.13 external-monitor-job 1.7 files-found-trigger 1.5 git 3.3.0 git-client 2.4.6 git-server 1.7 github 1.27.0 github-api 1.85.1 gradle 1.26 handlebars 1.1.1 htmlpublisher 1.13 icon-shim 2.0.3 jackson2-api 2.7.3 javadoc 1.4 jira 2.3.1 jobConfigHistory 2.16 jquery-detached 1.2.1 junit 1.20 lastfailureversioncolumn 1.1 lastsuccessversioncolumn 1.1 ldap 1.15 mailer 1.20 mapdb-api 1.0.9.0 matrix-auth 1.6 matrix-project 1.11 maven-plugin 2.16 mercurial 1.60 momentjs 1.1.1 monitoring 1.67.0 multiple-scms 0.6 notification 1.11 pam-auth 1.3 persona 2.4 false pipeline-build-step 2.5 pipeline-graph-analysis 1.4 pipeline-input-step 2.7 pipeline-milestone-step 1.3.1 pipeline-model-api 1.1.5 pipeline-model-declarative-agent 1.1.1 pipeline-model-definition 1.1.5 pipeline-model-extensions 1.1.5 pipeline-rest-api 2.8 pipeline-stage-step 2.2 pipeline-stage-tags-metadata 1.1.5 pipeline-stage-view 2.8 plain-credentials 1.4 plot 1.11 project-description-setter 1.1 release 2.7 repo 1.10.7 false resource-disposer 0.6 role-strategy 2.5.0 scm-api 2.1.1 script-security 1.28 ssh-credentials 1.13 ssh-slaves 1.17 structs 1.7 subversion 2.7.2 svn-release-mgr 1.2 throttle-concurrents 2.0.1 token-macro 2.1 tracking-svn 1.1 translation 1.15 viewVC 1.7 warnings 4.62 windows-slaves 1.3.1 workflow-aggregator 2.5 workflow-api 2.17 workflow-basic-steps 2.5 workflow-cps 2.34 workflow-cps-global-lib 2.8 workflow-durable-task-step 2.11 workflow-job 2.11 workflow-multibranch 2.15 workflow-scm-step 2.4 workflow-step-api 2.11 workflow-support 2.14 ws-cleanup 0.33 xunit 1.102
After we updated our Jenkins-Instance from 1.609.3 to 2.46.3 (both LTS) the Post-Build-Action "Update relevant issues" will stall many of our Jenkins-Builds. (100% CPU usage on Jenkins Master, the Builds would run hours and days if I didn't kill the thread manually).
The Action "Update relevant issues" is enabled in some of our Jobs and the Default Issue Selector is selected.
All of those Jobs are of the Type "Freestyle Build Job".
Our Sourcecode-Repository is a Subversion-Repository. Builds are triggered via a svn post commit hook. Our subversion project repositories uses 'svn externals' - there are around ~50 externals (libraries) included in a project-checkout.
Sometimes one and the same library is checked out multiple times for one build-job, e.g.
at Path /: svn:externals=/repos/Tools/tags/V0071/ tools
at Path /build: svn:externals=/repos/Tools/tags/V0071/yagarto/4.90/ yagarto
The problem does not occour on every build of those jobs - only if "Changes in dependency" are detected the problem occours.
There is nothing written on the Build-Console after
[WARNINGS] Parsing warnings in console log with parser GNU C Compiler 4 (gcc)
[WARNINGS] Computing warning deltas based on reference build #1204
Archiving artifacts
Recording fingerprints
At Jenkins-Logfile nothing abnormal is logged while the Thread is working and working.
Only at Java-Melody I can see something suspicious:
The Executor-Thread for this Build consumes loads fo CPU-time and seems to be in some kind of a recursion:
Executor #0 for HUDSONSL1 : executing {Jobname} #1205
java.util.StringTokenizer.scanToken(StringTokenizer.java:278)
java.util.StringTokenizer.nextToken(StringTokenizer.java:351)
jenkins.model.Jenkins.getItemByFullName(Jenkins.java:2876)
hudson.model.Fingerprint$BuildPtr.getJob(Fingerprint.java:152)
hudson.tasks.Fingerprinter$FingerprintAction.getDependencies(Fingerprinter.java:412)
hudson.model.AbstractBuild.getDependencyChanges(AbstractBuild.java:1271)
hudson.plugins.jira.RunScmChangeExtractor.getDependencyChanges(RunScmChangeExtractor.java:76)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesFromDependentBuilds(DefaultIssueSelector.java:132)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesRecursive(DefaultIssueSelector.java:110)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesFromDependentBuilds(DefaultIssueSelector.java:135)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesRecursive(DefaultIssueSelector.java:110)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesFromDependentBuilds(DefaultIssueSelector.java:135)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesRecursive(DefaultIssueSelector.java:110)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesFromDependentBuilds(DefaultIssueSelector.java:135)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesRecursive(DefaultIssueSelector.java:110)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesFromDependentBuilds(DefaultIssueSelector.java:135)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesRecursive(DefaultIssueSelector.java:110)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesFromDependentBuilds(DefaultIssueSelector.java:135)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesRecursive(DefaultIssueSelector.java:110)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesFromDependentBuilds(DefaultIssueSelector.java:135)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesRecursive(DefaultIssueSelector.java:110)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesFromDependentBuilds(DefaultIssueSelector.java:135)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesRecursive(DefaultIssueSelector.java:110)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesFromDependentBuilds(DefaultIssueSelector.java:135)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesRecursive(DefaultIssueSelector.java:110)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesFromDependentBuilds(DefaultIssueSelector.java:135)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesRecursive(DefaultIssueSelector.java:110)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesFromDependentBuilds(DefaultIssueSelector.java:135)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesRecursive(DefaultIssueSelector.java:110)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesFromDependentBuilds(DefaultIssueSelector.java:135)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesRecursive(DefaultIssueSelector.java:110)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesFromDependentBuilds(DefaultIssueSelector.java:135)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesRecursive(DefaultIssueSelector.java:110)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesFromDependentBuilds(DefaultIssueSelector.java:135)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesRecursive(DefaultIssueSelector.java:110)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesFromDependentBuilds(DefaultIssueSelector.java:135)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesRecursive(DefaultIssueSelector.java:110)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesFromDependentBuilds(DefaultIssueSelector.java:135)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesRecursive(DefaultIssueSelector.java:110)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesFromDependentBuilds(DefaultIssueSelector.java:135)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesRecursive(DefaultIssueSelector.java:110)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesFromDependentBuilds(DefaultIssueSelector.java:135)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesRecursive(DefaultIssueSelector.java:110)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesFromDependentBuilds(DefaultIssueSelector.java:135)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesRecursive(DefaultIssueSelector.java:110)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesFromDependentBuilds(DefaultIssueSelector.java:135)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesRecursive(DefaultIssueSelector.java:110)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesFromDependentBuilds(DefaultIssueSelector.java:135)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesRecursive(DefaultIssueSelector.java:110)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesFromDependentBuilds(DefaultIssueSelector.java:135)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesRecursive(DefaultIssueSelector.java:110)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesFromDependentBuilds(DefaultIssueSelector.java:135)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesRecursive(DefaultIssueSelector.java:110)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesFromDependentBuilds(DefaultIssueSelector.java:135)
hudson.plugins.jira.selector.DefaultIssueSelector.addIssuesRecursive(DefaultIssueSelector.java:110)
hudson.plugins.jira.selector.DefaultIssueSelector.findIssueIds(DefaultIssueSelector.java:48)
hudson.plugins.jira.Updater.perform(Updater.java:85)
hudson.plugins.jira.JiraIssueUpdater.perform(JiraIssueUpdater.java:64)
hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:78)
hudson.tasks.BuildStepMonitor$3.perform(BuildStepMonitor.java:45)
hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
hudson.model.Build$BuildExecution.post2(Build.java:186)
hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:665)
hudson.model.Run.execute(Run.java:1753)
hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
hudson.model.ResourceController.execute(ResourceController.java:98)
hudson.model.Executor.run(Executor.java:405
Another Problem we now have is (don't know if this problems are in connection), that the post build action "Update issues" does comment loads of issues... often also those, which hasn't been mentioned in a commit comment since years.
