[JENKINS-30909] queue.xml only gets persisted on successful shutdown - Jenkins Jira

XML

Word

Printable

Details

Type: Bug
Status: Resolved (View Workflow)
Priority: Major
Resolution: Fixed
Component/s: core
Labels:
- 2.107.2-fixed

Similar Issues:

Show

Description

So, if Jenkins doesn't shutdown properly, we lose the queueId. This is what we suspect as being the root case of the issue in ~~JENKINS-30899~~

Attachments

Issue Links

is blocking

JENKINS-30899 Build History order not sorted properly

Resolved

is related to

JENKINS-34281 Queue isn't saved on any shutdown

Resolved

JENKINS-32820 Queue is not persisted on Safe Restart when Jenkins is installed as Windows Service (clean up is not performed)

Resolved

JENKINS-33926 Jenkins no longer appears to shutdown correctly

Resolved

links to

CloudBees Internal CJP-1708

Activity

Ascending order - Click to sort in descending order

Tom FENNELLY created issue - 2015-10-12 18:16

Tom FENNELLY made changes - 2015-10-12 18:19

Field	Original Value	New Value
Link		This issue is blocking ~~JENKINS-30899~~ [ ~~JENKINS-30899~~ ]

Tom FENNELLY added a comment - 2015-10-12 20:16

https://github.com/jenkinsci/jenkins/pull/1865

Tom FENNELLY added a comment - 2015-10-12 20:16 https://github.com/jenkinsci/jenkins/pull/1865

Kristin Whetstone made changes - 2016-04-05 15:37

Link

This issue is related to ~~JENKINS-33926~~ [ ~~JENKINS-33926~~ ]

Kristin Whetstone added a comment - 2016-04-15 05:15

As part of an aside, I've been running tests on Jenkins 1.x and the 2 betas/rcs and am running into some interesting issues with saving the queue on Windows. If Jenkins is running as a process and it is stopped, it won't save a queue.xml. From looking at this pull request, this looks like it should have fixed the problem; are these cases related?

Kristin Whetstone added a comment - 2016-04-15 05:15 As part of an aside, I've been running tests on Jenkins 1.x and the 2 betas/rcs and am running into some interesting issues with saving the queue on Windows. If Jenkins is running as a process and it is stopped, it won't save a queue.xml . From looking at this pull request, this looks like it should have fixed the problem; are these cases related?

Kristin Whetstone made changes - 2016-04-15 15:06

Link

This issue is related to ~~JENKINS-34281~~ [ ~~JENKINS-34281~~ ]

Jesse Glick added a comment - 2016-06-02 16:44

Is this simply a duplicate of ~~JENKINS-34281~~, or is it discussing something else? I am not sure what precisely “we lose the queueId” means.

Jesse Glick added a comment - 2016-06-02 16:44 Is this simply a duplicate of JENKINS-34281 , or is it discussing something else? I am not sure what precisely “we lose the queueId” means.

R. Tyler Croy made changes - 2016-07-25 23:52

Workflow

JNJira [ 166036 ]

JNJira + In-Review [ 182257 ]

Tim Vail added a comment - 2016-08-01 18:54

Jesse – this appears to be an old issue. However, I'm finding a few issues with Jenkins and the queue persistence. Particularly on Windows.

If you run Jenkins as a service then you restart the service (as opposed to telling Jenkins to exit) then there is no queue.xml created and the build queue is therefore lost.
Prior situation means that if the machine is rebooted – we would probably lose the queue.
Even more interesting – it appears that your fix somehow isn't working for /safeRestart or /restart. It is like /exit (which does save the queue.xml) is actually safer than /safeRestart.

Considering all things – why don't we simply write queue.xml to the disk on every queue change? With today's HDD and SSD's – I'd think for most servers Jenkins is running on – the cost of doing so should be minimal. Compared to the writing of build logs to disk this is practically nothing. At least, why not make it an option to continuously persist the build queue (if so, then I'd think it should be on by default).

Tim Vail added a comment - 2016-08-01 18:54 Jesse – this appears to be an old issue. However, I'm finding a few issues with Jenkins and the queue persistence. Particularly on Windows. If you run Jenkins as a service then you restart the service (as opposed to telling Jenkins to exit) then there is no queue.xml created and the build queue is therefore lost. Prior situation means that if the machine is rebooted – we would probably lose the queue. Even more interesting – it appears that your fix somehow isn't working for /safeRestart or /restart. It is like /exit (which does save the queue.xml) is actually safer than /safeRestart. Considering all things – why don't we simply write queue.xml to the disk on every queue change? With today's HDD and SSD's – I'd think for most servers Jenkins is running on – the cost of doing so should be minimal. Compared to the writing of build logs to disk this is practically nothing. At least, why not make it an option to continuously persist the build queue (if so, then I'd think it should be on by default).

Oleg Nenashev made changes - 2017-03-29 08:38

Link

This issue is related to ~~JENKINS-32820~~ [ ~~JENKINS-32820~~ ]

Oleg Nenashev added a comment - 2017-03-29 08:39

Queue persistence on Windows has been fixed in ~~JENKINS-32820~~ (2.47, backported to 2.46.1). Not sure what else we can do in this ticket

Oleg Nenashev added a comment - 2017-03-29 08:39 Queue persistence on Windows has been fixed in JENKINS-32820 (2.47, backported to 2.46.1). Not sure what else we can do in this ticket

Jesse Glick added a comment - 2017-04-13 20:00

In general the queue will change once per second, so writing it to disk every time may be unwise, but I agree it could be automatically persisted every minute or so.

Jesse Glick added a comment - 2017-04-13 20:00 In general the queue will change once per second, so writing it to disk every time may be unwise, but I agree it could be automatically persisted every minute or so.

Jesse Glick added a comment - 2017-05-31 18:25

An idea from svanoort (IIRC) was to have Queue.MaintainTask.doRun compare a Snapshot before and after maintain, and if they differ, call Queue.save; this would ensure that any changes to the queue are saved within five seconds and would be restored after a crash. Probably queue.xml.bak could be considered obsolete too.

Jesse Glick added a comment - 2017-05-31 18:25 An idea from svanoort (IIRC) was to have Queue.MaintainTask.doRun compare a Snapshot before and after maintain , and if they differ, call Queue.save ; this would ensure that any changes to the queue are saved within five seconds and would be restored after a crash. Probably queue.xml.bak could be considered obsolete too.

CloudBees Inc. made changes - 2017-12-05 17:27

Remote Link

This issue links to "CloudBees Internal CJP-1708 (Web Link)" [ 19167 ]

Oliver Gondža made changes - 2018-01-18 15:19

Assignee

Tom FENNELLY [ tfennelly ]

Oliver Gondža [ olivergondza ]

Oliver Gondža added a comment - 2018-01-18 15:26

Proposing an alternative fix: https://github.com/jenkinsci/jenkins/pull/3244

Oliver Gondža added a comment - 2018-01-18 15:26 Proposing an alternative fix: https://github.com/jenkinsci/jenkins/pull/3244

Oliver Gondža made changes - 2018-02-20 12:48

Status

Open [ 1 ]

In Progress [ 3 ]

Oliver Gondža made changes - 2018-02-20 12:49

Status

In Progress [ 3 ]

In Review [ 10005 ]

SCM/JIRA link daemon added a comment - 2018-02-23 19:30

Code changed in jenkins
User: Oliver Gondža
Path:
core/src/main/java/hudson/model/Queue.java
test/pom.xml
test/src/test/java/hudson/model/QueueRestartTest.java
test/src/test/java/hudson/model/QueueTest.java
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/config.xml
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/a/config.xml
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/b/config.xml
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/c/config.xml
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/d/config.xml
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/e/config.xml
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/f/config.xml
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/g/config.xml
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/h/config.xml
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/i/config.xml
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/j/config.xml
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/k/config.xml
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/queue.xml
http://jenkins-ci.org/commit/jenkins/72a7529a119b2be6b3ff93d63688ee6973404236
Log:
[FIX JENKINS-30909] Make sure queue is persisted reliably (#3244)

[FIX JENKINS-30909] Make sure queue is persisted reliably

Fix tests on Windows

Ensure Saver is thread-safe

SCM/JIRA link daemon added a comment - 2018-02-23 19:30 Code changed in jenkins User: Oliver Gondža Path: core/src/main/java/hudson/model/Queue.java test/pom.xml test/src/test/java/hudson/model/QueueRestartTest.java test/src/test/java/hudson/model/QueueTest.java test/src/test/resources/hudson/model/QueueTest/load_queue_xml/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/a/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/b/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/c/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/d/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/e/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/f/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/g/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/h/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/i/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/j/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/k/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/queue.xml http://jenkins-ci.org/commit/jenkins/72a7529a119b2be6b3ff93d63688ee6973404236 Log: [FIX JENKINS-30909] Make sure queue is persisted reliably (#3244) [FIX JENKINS-30909] Make sure queue is persisted reliably Fix tests on Windows Ensure Saver is thread-safe

Oleg Nenashev added a comment - 2018-02-26 07:59

Fixed in 2.109. Likely it should be considered as a partial fix.
olivergondza do you consider it as LTS candidate?

Oleg Nenashev added a comment - 2018-02-26 07:59 Fixed in 2.109. Likely it should be considered as a partial fix. olivergondza do you consider it as LTS candidate?

Oleg Nenashev made changes - 2018-02-26 07:59

Resolution		Fixed [ 1 ]
Status	In Review [ 10005 ]	Resolved [ 5 ]

Oliver Gondža made changes - 2018-02-26 08:01

Labels

lts-candidate

Oliver Gondža added a comment - 2018-02-26 08:01

oleg_nenashev, certainly not for .1, adding the label to be reconsidered later.

Oliver Gondža added a comment - 2018-02-26 08:01 oleg_nenashev , certainly not for .1, adding the label to be reconsidered later.

Oliver Gondža made changes - 2018-03-23 13:32

Labels

lts-candidate

2.107.2-fixed

SCM/JIRA link daemon added a comment - 2018-03-23 17:25

Code changed in jenkins
User: Oliver Gondža
Path:
core/src/main/java/hudson/model/Queue.java
test/pom.xml
test/src/test/java/hudson/model/QueueRestartTest.java
test/src/test/java/hudson/model/QueueTest.java
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/config.xml
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/a/config.xml
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/b/config.xml
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/c/config.xml
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/d/config.xml
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/e/config.xml
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/f/config.xml
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/g/config.xml
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/h/config.xml
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/i/config.xml
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/j/config.xml
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/k/config.xml
test/src/test/resources/hudson/model/QueueTest/load_queue_xml/queue.xml
http://jenkins-ci.org/commit/jenkins/8c975f11c5a366b1693ed2be04e184a714cf2f84
Log:
[FIX JENKINS-30909] Make sure queue is persisted reliably (#3244)

[FIX JENKINS-30909] Make sure queue is persisted reliably

Fix tests on Windows

Ensure Saver is thread-safe

(cherry picked from commit 72a7529a119b2be6b3ff93d63688ee6973404236)

SCM/JIRA link daemon added a comment - 2018-03-23 17:25 Code changed in jenkins User: Oliver Gondža Path: core/src/main/java/hudson/model/Queue.java test/pom.xml test/src/test/java/hudson/model/QueueRestartTest.java test/src/test/java/hudson/model/QueueTest.java test/src/test/resources/hudson/model/QueueTest/load_queue_xml/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/a/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/b/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/c/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/d/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/e/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/f/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/g/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/h/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/i/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/j/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/k/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/queue.xml http://jenkins-ci.org/commit/jenkins/8c975f11c5a366b1693ed2be04e184a714cf2f84 Log: [FIX JENKINS-30909] Make sure queue is persisted reliably (#3244) [FIX JENKINS-30909] Make sure queue is persisted reliably Fix tests on Windows Ensure Saver is thread-safe (cherry picked from commit 72a7529a119b2be6b3ff93d63688ee6973404236)

People

Assignee:: Oliver Gondža

Reporter:: Tom FENNELLY

Votes:: 4 Vote for this issue

Watchers:: 12 Start watching this issue

Dates

Created:: 2015-10-12 18:16

Updated:: 2018-03-23 17:25

Resolved:: 2018-02-26 07:59