[JENKINS-19741] Massive jobs execution causes issues in lastSuccessfulBuild permalink update

Type: Bug
Resolution: Unresolved
Priority: Major
Component/s: core
Labels:
- concurrent-build
Environment:
1) 1.509.3, Linux server (RHEL6)
2) 1.509.4 LTS, SL Linux 6.4

Similar Issues:
Powered by SuggestiMate

Show

We run massive parallel builds of the free-style job (~120 simultaneous runs). Most of runs take about 15 seconds, to job frequently updates the build statistics.

Several jobs print warnings to the log. I suppose that issue is caused by parallel updates of the file.

Sep 25, 2013 10:42:48 AM jenkins.model.PeepholePermalink updateCache
WARNING: Failed to update hudson.model.FreeStyleProject@3771fd63[SBC_Run_Any_Batch] lastSuccessfulBuild permalink for MY_JOB #3830
java.nio.file.NoSuchFileException: /var/lib/jenkins/jobs/MY_JOB/builds/lastSuccessfulBuild.tmp
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileSystemProvider.readSymbolicLink(UnixFileSystemProvider.java:495)
at java.nio.file.Files.readSymbolicLink(Files.java:1382)
at sun.reflect.GeneratedMethodAccessor249.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at hudson.Util.resolveSymlink(Util.java:1213)
at jenkins.model.PeepholePermalink.writeSymlink(PeepholePermalink.java:169)
at jenkins.model.PeepholePermalink.updateCache(PeepholePermalink.java:137)
at jenkins.model.PeepholePermalink.resolve(PeepholePermalink.java:114)
at jenkins.model.PeepholePermalink$RunListenerImpl.onCompleted(PeepholePermalink.java:212)
at hudson.model.listeners.RunListener.fireCompleted(RunListener.java:199)
at hudson.model.Run.execute(Run.java:1642)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:237)

- - Sort By Name
  - Sort By Date
  - Ascending
  - Descending
  - Thumbnails
  - List
  - Download All

140107-LZA-J19741-log-fragment.log
5 kB
2014-01-07 16:47

is duplicated by

JENKINS-15476 "Block until the triggered projects finish their builds" doesn't notify upstream project when complete.

Resolved

is related to

JENKINS-54863 Parallel builds in same workspace

Open

JENKINS-17681 LastSuccessful and LastStable symlinks are invalid under Windows

Resolved

Oleg Nenashev created issue - 2013-09-25 10:50

Oleg Nenashev made changes - 2013-09-25 10:50

Summary

Original: Massive

New: Massive jobs execution causes issues in lastSuccessfulBuild permalink update

Oleg Nenashev added a comment - 2013-09-25 10:51

Quite similar issue has been fixed for Windows in 1.509.3

Oleg Nenashev added a comment - 2013-09-25 10:51 Quite similar issue has been fixed for Windows in 1.509.3

Oleg Nenashev made changes - 2013-09-25 10:51

Link

New: This issue is related to ~~JENKINS-17681~~ [ ~~JENKINS-17681~~ ]

Oleg Nenashev made changes - 2013-09-26 13:39

Link

New: This issue duplicates ~~JENKINS-19377~~ [ ~~JENKINS-19377~~ ]

Oleg Nenashev made changes - 2013-09-26 13:39

Resolution		New: Duplicate [ 3 ]
Status	Original: Open [ 1 ]	New: Resolved [ 5 ]

Oleg Nenashev made changes - 2013-10-04 13:05

Link

New: This issue is duplicated by ~~JENKINS-15476~~ [ ~~JENKINS-15476~~ ]

Lukáš Zach added a comment - 2014-01-07 16:46

We are facing the same issue (on Linux as original reporter).

Jenkins ver	Jenkins ver. 1.509.4 LTS
OS ver	Scientific Linux release 6.4 (Carbon)

The same stacktrace as above.

Jan 01, 2014 7:05:37 PM jenkins.model.PeepholePermalink updateCache
WARNING: Failed to update hudson.model.FreeStyleProject@6d5d2cc[prototype load] lastStableBuild permalink for prototype load t2mg0i1lkvibf4v07yw184j0m0j6yk2m-1383593938@qa-9
8prototype-1.na.getgooddata.com
java.nio.file.NoSuchFileException: /var/lib/jenkins/jobs/prototype load/builds/lastStableBuild.tmp
        at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
        at sun.nio.fs.UnixFileSystemProvider.readSymbolicLink(UnixFileSystemProvider.java:495)
        at java.nio.file.Files.readSymbolicLink(Files.java:1382)
        at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at hudson.Util.resolveSymlink(Util.java:1216)
        at jenkins.model.PeepholePermalink.writeSymlink(PeepholePermalink.java:169)
        at jenkins.model.PeepholePermalink.updateCache(PeepholePermalink.java:137)
        at jenkins.model.PeepholePermalink$RunListenerImpl.onCompleted(PeepholePermalink.java:214)
        at hudson.model.listeners.RunListener.fireCompleted(RunListener.java:199)
        at hudson.model.Run.execute(Run.java:1642)
        at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
        at hudson.model.ResourceController.execute(ResourceController.java:88)
        at hudson.model.Executor.run(Executor.java:237)

Bigger log fragment here - 140107-LZA-J19741-log-fragment.log

Statistic per day on our Jenkins instance:

[root@jenkins-1509:/var/lib/jenkins/logs] zgrep -c "WARNING: Failed to update hudson.model.FreeStyleProject" jen* | sort -r
jenkins.log:23 (20140106)
jenkins.log-20140104.gz:26
jenkins.log-20140101.gz:22
jenkins.log-20131227.gz:18
jenkins.log-20131221.gz:23
jenkins.log-20131219.gz:1

IMO it has nothing to do with windows platform.

=> Reopening

Lukáš Zach added a comment - 2014-01-07 16:46 We are facing the same issue (on Linux as original reporter). Jenkins ver Jenkins ver. 1.509.4 LTS OS ver Scientific Linux release 6.4 (Carbon) The same stacktrace as above. Jan 01, 2014 7:05:37 PM jenkins.model.PeepholePermalink updateCache WARNING: Failed to update hudson.model.FreeStyleProject@6d5d2cc[prototype load] lastStableBuild permalink for prototype load t2mg0i1lkvibf4v07yw184j0m0j6yk2m-1383593938@qa-9 8prototype-1.na.getgooddata.com java.nio.file.NoSuchFileException: /var/lib/jenkins/jobs/prototype load/builds/lastStableBuild.tmp at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at sun.nio.fs.UnixFileSystemProvider.readSymbolicLink(UnixFileSystemProvider.java:495) at java.nio.file.Files.readSymbolicLink(Files.java:1382) at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at hudson.Util.resolveSymlink(Util.java:1216) at jenkins.model.PeepholePermalink.writeSymlink(PeepholePermalink.java:169) at jenkins.model.PeepholePermalink.updateCache(PeepholePermalink.java:137) at jenkins.model.PeepholePermalink$RunListenerImpl.onCompleted(PeepholePermalink.java:214) at hudson.model.listeners.RunListener.fireCompleted(RunListener.java:199) at hudson.model.Run.execute(Run.java:1642) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:237) Bigger log fragment here - 140107-LZA-J19741-log-fragment.log Statistic per day on our Jenkins instance: [root@jenkins-1509:/var/lib/jenkins/logs] zgrep -c "WARNING: Failed to update hudson.model.FreeStyleProject" jen* | sort -r jenkins.log:23 (20140106) jenkins.log-20140104.gz:26 jenkins.log-20140101.gz:22 jenkins.log-20131227.gz:18 jenkins.log-20131221.gz:23 jenkins.log-20131219.gz:1 IMO it has nothing to do with windows platform. => Reopening

Lukáš Zach made changes - 2014-01-07 16:46

Resolution	Original: Duplicate [ 3 ]
Status	Original: Resolved [ 5 ]	New: Reopened [ 4 ]

Lukáš Zach made changes - 2014-01-07 16:47

Attachment

New: 140107-LZA-J19741-log-fragment.log [ 24949 ]

Assignee:: Unassigned

Reporter:: Oleg Nenashev

Votes:: 7 Vote for this issue

Watchers:: 12 Start watching this issue

Created:: 2013-09-25 10:50

Updated:: 2018-12-06 08:40

Jenkins

Details

Description

Attachments

Attachments

Issue Links

Activity

Collapse comment: Oleg Nenashev added a comment - 2013-09-25 10:51

Expand comment: Oleg Nenashev added a comment - 2013-09-25 10:51

Collapse comment: Lukáš Zach added a comment - 2014-01-07 16:46

Expand comment: Lukáš Zach added a comment - 2014-01-07 16:46

People

Dates