Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-30909

queue.xml only gets persisted on successful shutdown

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • core

      So, if Jenkins doesn't shutdown properly, we lose the queueId. This is what we suspect as being the root case of the issue in JENKINS-30899

          [JENKINS-30909] queue.xml only gets persisted on successful shutdown

          Jesse Glick added a comment -

          Is this simply a duplicate of JENKINS-34281, or is it discussing something else? I am not sure what precisely “we lose the queueId” means.

          Jesse Glick added a comment - Is this simply a duplicate of JENKINS-34281 , or is it discussing something else? I am not sure what precisely “we lose the queueId” means.

          Tim Vail added a comment -

          Jesse – this appears to be an old issue. However, I'm finding a few issues with Jenkins and the queue persistence. Particularly on Windows.

          1. If you run Jenkins as a service then you restart the service (as opposed to telling Jenkins to exit) then there is no queue.xml created and the build queue is therefore lost.
          2. Prior situation means that if the machine is rebooted – we would probably lose the queue.
          3. Even more interesting – it appears that your fix somehow isn't working for /safeRestart or /restart. It is like /exit (which does save the queue.xml) is actually safer than /safeRestart.

          Considering all things – why don't we simply write queue.xml to the disk on every queue change? With today's HDD and SSD's – I'd think for most servers Jenkins is running on – the cost of doing so should be minimal. Compared to the writing of build logs to disk this is practically nothing. At least, why not make it an option to continuously persist the build queue (if so, then I'd think it should be on by default).

          Tim Vail added a comment - Jesse – this appears to be an old issue. However, I'm finding a few issues with Jenkins and the queue persistence. Particularly on Windows. If you run Jenkins as a service then you restart the service (as opposed to telling Jenkins to exit) then there is no queue.xml created and the build queue is therefore lost. Prior situation means that if the machine is rebooted – we would probably lose the queue. Even more interesting – it appears that your fix somehow isn't working for /safeRestart or /restart. It is like /exit (which does save the queue.xml) is actually safer than /safeRestart. Considering all things – why don't we simply write queue.xml to the disk on every queue change? With today's HDD and SSD's – I'd think for most servers Jenkins is running on – the cost of doing so should be minimal. Compared to the writing of build logs to disk this is practically nothing. At least, why not make it an option to continuously persist the build queue (if so, then I'd think it should be on by default).

          Oleg Nenashev added a comment -

          Queue persistence on Windows has been fixed in JENKINS-32820 (2.47, backported to 2.46.1). Not sure what else we can do in this ticket

          Oleg Nenashev added a comment - Queue persistence on Windows has been fixed in JENKINS-32820 (2.47, backported to 2.46.1). Not sure what else we can do in this ticket

          Jesse Glick added a comment -

          In general the queue will change once per second, so writing it to disk every time may be unwise, but I agree it could be automatically persisted every minute or so.

          Jesse Glick added a comment - In general the queue will change once per second, so writing it to disk every time may be unwise, but I agree it could be automatically persisted every minute or so.

          Jesse Glick added a comment -

          An idea from svanoort (IIRC) was to have Queue.MaintainTask.doRun compare a Snapshot before and after maintain, and if they differ, call Queue.save; this would ensure that any changes to the queue are saved within five seconds and would be restored after a crash. Probably queue.xml.bak could be considered obsolete too.

          Jesse Glick added a comment - An idea from svanoort (IIRC) was to have Queue.MaintainTask.doRun compare a Snapshot before and after maintain , and if they differ, call Queue.save ; this would ensure that any changes to the queue are saved within five seconds and would be restored after a crash. Probably queue.xml.bak could be considered obsolete too.

          Proposing an alternative fix: https://github.com/jenkinsci/jenkins/pull/3244

          Oliver Gondža added a comment - Proposing an alternative fix: https://github.com/jenkinsci/jenkins/pull/3244

          Code changed in jenkins
          User: Oliver Gondža
          Path:
          core/src/main/java/hudson/model/Queue.java
          test/pom.xml
          test/src/test/java/hudson/model/QueueRestartTest.java
          test/src/test/java/hudson/model/QueueTest.java
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/config.xml
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/a/config.xml
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/b/config.xml
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/c/config.xml
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/d/config.xml
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/e/config.xml
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/f/config.xml
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/g/config.xml
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/h/config.xml
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/i/config.xml
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/j/config.xml
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/k/config.xml
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/queue.xml
          http://jenkins-ci.org/commit/jenkins/72a7529a119b2be6b3ff93d63688ee6973404236
          Log:
          [FIX JENKINS-30909] Make sure queue is persisted reliably (#3244)

          • [FIX JENKINS-30909] Make sure queue is persisted reliably
          • Fix tests on Windows
          • Ensure Saver is thread-safe

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oliver Gondža Path: core/src/main/java/hudson/model/Queue.java test/pom.xml test/src/test/java/hudson/model/QueueRestartTest.java test/src/test/java/hudson/model/QueueTest.java test/src/test/resources/hudson/model/QueueTest/load_queue_xml/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/a/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/b/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/c/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/d/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/e/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/f/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/g/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/h/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/i/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/j/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/k/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/queue.xml http://jenkins-ci.org/commit/jenkins/72a7529a119b2be6b3ff93d63688ee6973404236 Log: [FIX JENKINS-30909] Make sure queue is persisted reliably (#3244) [FIX JENKINS-30909] Make sure queue is persisted reliably Fix tests on Windows Ensure Saver is thread-safe

          Oleg Nenashev added a comment -

          Fixed in 2.109. Likely it should be considered as a partial fix.
          olivergondza do you consider it as LTS candidate?

          Oleg Nenashev added a comment - Fixed in 2.109. Likely it should be considered as a partial fix. olivergondza do you consider it as LTS candidate?

          oleg_nenashev, certainly not for .1, adding the label to be reconsidered later.

          Oliver Gondža added a comment - oleg_nenashev , certainly not for .1, adding the label to be reconsidered later.

          Code changed in jenkins
          User: Oliver Gondža
          Path:
          core/src/main/java/hudson/model/Queue.java
          test/pom.xml
          test/src/test/java/hudson/model/QueueRestartTest.java
          test/src/test/java/hudson/model/QueueTest.java
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/config.xml
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/a/config.xml
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/b/config.xml
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/c/config.xml
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/d/config.xml
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/e/config.xml
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/f/config.xml
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/g/config.xml
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/h/config.xml
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/i/config.xml
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/j/config.xml
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/k/config.xml
          test/src/test/resources/hudson/model/QueueTest/load_queue_xml/queue.xml
          http://jenkins-ci.org/commit/jenkins/8c975f11c5a366b1693ed2be04e184a714cf2f84
          Log:
          [FIX JENKINS-30909] Make sure queue is persisted reliably (#3244)

          • [FIX JENKINS-30909] Make sure queue is persisted reliably
          • Fix tests on Windows
          • Ensure Saver is thread-safe

          (cherry picked from commit 72a7529a119b2be6b3ff93d63688ee6973404236)

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oliver Gondža Path: core/src/main/java/hudson/model/Queue.java test/pom.xml test/src/test/java/hudson/model/QueueRestartTest.java test/src/test/java/hudson/model/QueueTest.java test/src/test/resources/hudson/model/QueueTest/load_queue_xml/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/a/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/b/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/c/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/d/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/e/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/f/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/g/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/h/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/i/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/j/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/jobs/k/config.xml test/src/test/resources/hudson/model/QueueTest/load_queue_xml/queue.xml http://jenkins-ci.org/commit/jenkins/8c975f11c5a366b1693ed2be04e184a714cf2f84 Log: [FIX JENKINS-30909] Make sure queue is persisted reliably (#3244) [FIX JENKINS-30909] Make sure queue is persisted reliably Fix tests on Windows Ensure Saver is thread-safe (cherry picked from commit 72a7529a119b2be6b3ff93d63688ee6973404236)

            olivergondza Oliver Gondža
            tfennelly Tom FENNELLY
            Votes:
            4 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated:
              Resolved: