Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-50552

Pipeline projects do not trigger libvirt VM snapshot reverts

    • Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Minor Minor
    • libvirt-slave-plugin
    • Ubuntu 16.04.3 LTS 64 bit (running directly on the hardware)
      OpenJDK 1.8.0_151
      Jenkins 2.107.1
      Libvirt-plugin 1.8.5
      No proxy.
      Connection method: "Launch Slave Agents via SSH"
      Firefox 59.0.1 and Chrome 64.0.3282.167

      Configuring a snapshot revert within the project via the "Use Snapshots" checkbox or via the "Revert Snapshot" and "Before Snapshot" options on the node configuration does not appear to do anything on pipeline projects. They run exactly as though those settings were completely ignored.

          [JENKINS-50552] Pipeline projects do not trigger libvirt VM snapshot reverts

          Michael Dirks added a comment -

          Can unfortunately confirm this happening. I wanted to switch off of AppVeyor and Travis and have builds in-house (faster and less costly) and ended up running into a situation where the build bots had old versions of the repository and configuration left around, resulting in faster later builds, but eventually resulting in failures due to configuration changes and the same directories being used.

          Jenkins Version: 2.141
          libvirt Slaves plugin Version: 1.8.5
          Remote VM Manager: KVM/QEMU

          Michael Dirks added a comment - Can unfortunately confirm this happening. I wanted to switch off of AppVeyor and Travis and have builds in-house (faster and less costly) and ended up running into a situation where the build bots had old versions of the repository and configuration left around, resulting in faster later builds, but eventually resulting in failures due to configuration changes and the same directories being used. Jenkins Version: 2.141 libvirt Slaves plugin Version: 1.8.5 Remote VM Manager: KVM/QEMU

          Michael Dirks added a comment -

          Work on a fix for this has begun. I have the plugin listening to the pipeline complete event now, though I don't think I can make the "Before Snapshot" option work with Pipeline projects.

          Michael Dirks added a comment - Work on a fix for this has begun. I have the plugin listening to the pipeline complete event now, though I don't think I can make the "Before Snapshot" option work with Pipeline projects.

          Michael Dirks added a comment -

          Fix is working. Cleaning up, then submitting a PR.

          Michael Dirks added a comment - Fix is working. Cleaning up, then submitting a PR.

          David Taylor added a comment -

          xaymar any update on the plugin fix? my team is waiting for the "Before Snapshot" and "After Build" options to work within the node config. Any updates?

          David Taylor added a comment - xaymar any update on the plugin fix? my team is waiting for the "Before Snapshot" and "After Build" options to work within the node config. Any updates?

          Michael Dirks added a comment -

          taylor01 Unfortunately my patch breaks when you have the pipeline project running on more than one machine. I have actually managed to make it watch for what machines are being used, but unfortunately the pipeline projects are a bit difficult to manage.

          I am still working on it though. I really want to get rid of Travis and AppVeyor as my primary CIs and instead adopt Jenkins CI, simple because of the massive speed difference that I have observed when building (Jenkins CI is usually 2x to 3x faster on nearly identical build machines).

          Michael Dirks added a comment - taylor01 Unfortunately my patch breaks when you have the pipeline project running on more than one machine. I have actually managed to make it watch for what machines are being used, but unfortunately the pipeline projects are a bit difficult to manage. I am still working on it though. I really want to get rid of Travis and AppVeyor as my primary CIs and instead adopt Jenkins CI, simple because of the massive speed difference that I have observed when building (Jenkins CI is usually 2x to 3x faster on nearly identical build machines).

          Michael Dirks added a comment -

          I officially give up. I can't seem to hook into enough points of the Pipeline to prevent it from re-using "dead" nodes, which means that builds will unexpectedly fail for no other reason than that the VM was reset to the original state. I'm sure there's a lot of hook points I've missed, but the documentation is a bit large and I got lost quite a few times in it trying to find how to even figure out what node of a pipeline is being executed on what VM node.

          The code I have right now restarts the VM if it was known to be used by the Pipeline build system during any pipeline projects, which is unfortunately not enough. I will likely revisit this once I have more knowledge, but for now it's up for grabs for anyone that wants to try their hand at it.

          Michael Dirks added a comment - I officially give up. I can't seem to hook into enough points of the Pipeline to prevent it from re-using "dead" nodes, which means that builds will unexpectedly fail for no other reason than that the VM was reset to the original state. I'm sure there's a lot of hook points I've missed, but the documentation is a bit large and I got lost quite a few times in it trying to find how to even figure out what node of a pipeline is being executed on what VM node. The code I have right now restarts the VM if it was known to be used by the Pipeline build system during any pipeline projects, which is unfortunately not enough. I will likely revisit this once I have more knowledge, but for now it's up for grabs for anyone that wants to try their hand at it.

          benoit guerin added a comment -

          Hello xaymar

          Sorry to reply to this issue 2 years after but is there any trace of your modifications somewhere ? I am also affected by this problem and must solve it

          I found this, which might be relevant : https://github.com/jenkinsci/libvirt-slave-plugin/compare/master...qczwcad:dev 

          Thanks for your answer !

          benoit guerin added a comment - Hello xaymar Sorry to reply to this issue 2 years after but is there any trace of your modifications somewhere ? I am also affected by this problem and must solve it I found this, which might be relevant : https://github.com/jenkinsci/libvirt-slave-plugin/compare/master...qczwcad:dev   Thanks for your answer !

          Michael Dirks added a comment -

          Hey benoit guerin,

          I had them in a local repository, but they did not end up working at all so I threw it away. I couldn't figure out how to prevent Jenkins from using currently offline nodes, or trying to re-use the same node unexpectedly with a completely new task.

          Michael Dirks added a comment - Hey benoit guerin, I had them in a local repository, but they did not end up working at all so I threw it away. I couldn't figure out how to prevent Jenkins from using currently offline nodes, or trying to re-use the same node unexpectedly with a completely new task.

          Jonathan Rajotte-Julien added a comment - - edited

          Hi,

          We currently have a pipeline job that uses nodes that are configured using the libvirt plugin and cannot seem to get neither the `Before job` or the option `Use Snapshot` to work.

          I validated the plugin version and we are using the 1.9.2 version (Jenkins Libvirt Agents plugin 1.9.2 from the plugin page).

          I'm open to any way to do this (code/hack) as this is quite an important use case for us. I might be missing something. We basically need this to ensure that we run a "stage" on a clean vm.  I'm open to change how we do thing as long as I can guarantee that we use a clean vm (snapshot) when the stage run.

          Thanks.

           

           

          Jonathan Rajotte-Julien added a comment - - edited Hi, We currently have a pipeline job that uses nodes that are configured using the libvirt plugin and cannot seem to get neither the `Before job` or the option `Use Snapshot` to work. I validated the plugin version and we are using the 1.9.2 version (Jenkins Libvirt Agents plugin 1.9.2 from the plugin page). I'm open to any way to do this (code/hack) as this is quite an important use case for us. I might be missing something. We basically need this to ensure that we run a "stage" on a clean vm.  I'm open to change how we do thing as long as I can guarantee that we use a clean vm (snapshot) when the stage run. Thanks.    

            bgermann Bastian Germann
            zetasyanthis Zeta Syanthis
            Votes:
            2 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: