Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-57258

Lost builds historical data after updating to Jenkins v2.175

    • Icon: Bug Bug
    • Resolution: Cannot Reproduce
    • Icon: Blocker Blocker
    • core, pipeline
    • None
    • Jenkins v2.175 / Unix master / Java 64bit 1.8.0_192

      I'm using Multibranch Pipeline to automatically build changes in my Enterprise GitHub Organization repositories.

      After updating Jenkins to 2.175 and restarting, ALL the successful builds of ALL the branches of ALL my GitHub org repositories (that's around 75 repositories and approximately 2,000 builds) have been retriggered and instantly failed. All the builds failed with the same error, see attached console log. Now, all the projects and branches and builds are marked as Failed, and some build information such as the build date appear to be lost (see screenshots).

      In the example log, we can see the build was successfully completed a week ago, but restarted and failed today after restarting Jenkins.  I can consistenly reproduce this issue by re-building a project and restarting Jenkins.

      The issue does not happen if I downgrade to Jenkins 2.174.

        1. failed_build_console_log.txt
          3 kB
        2. build_history.PNG
          build_history.PNG
          17 kB
        3. build_before.xml
          25 kB
        4. build_after.xml
          16 kB
        5. branches.PNG
          branches.PNG
          68 kB

          [JENKINS-57258] Lost builds historical data after updating to Jenkins v2.175

          Nicolas Echegut created issue -

          Daniel Beck added a comment -

          Did you also update any plugins?

          Daniel Beck added a comment - Did you also update any plugins?

          I did prior to updating Jenkins, but restarted the instance in between. The only plugin updates were related to Blue Ocean.

          Nicolas Echegut added a comment - I did prior to updating Jenkins, but restarted the instance in between. The only plugin updates were related to Blue Ocean.

          Similar issue here, only not on all repos and all branches. I can't find a common pattern unfortunately.

          Jeremy Mathevet added a comment - Similar issue here, only not on all repos and all branches. I can't find a common pattern unfortunately.

          Baptiste Mathus added a comment - - edited

          That would (loosely?) hint about the serialization format change of the workflow-support:3.x line, but if you're saying you had updates all plugins before, that doesn't look like it.
          To be sure: can you confirm you were already running with all latest plugins, specifically workflow-support was 3.2?
          cc dnusbaum

          Baptiste Mathus added a comment - - edited That would (loosely?) hint about the serialization format change of the workflow-support:3.x line, but if you're saying you had updates all plugins before, that doesn't look like it. To be sure: can you confirm you were already running with all latest plugins, specifically workflow-support was 3.2? cc dnusbaum

          Oleg Nenashev added a comment -

          After going through https://github.com/jenkinsci/jenkins/compare/jenkins-2.174...jenkins-2.175 , I do not see anything strictly related to this issue. I could assume a weird side effect of https://github.com/jenkinsci/jenkins/pull/3980 , but there is no data to confirm it.

           

          batmat dnusbaum .dat files and format changes should be related to Running builds only, I am not sure these files are used for completed builds. Any chance we have issues with build status persistency in  build.xml 

           

          nico_echegut If you have build.xml files and System logs after the upgade to 2.175, it would help a lot

          Oleg Nenashev added a comment - After going through  https://github.com/jenkinsci/jenkins/compare/jenkins-2.174...jenkins-2.175  , I do not see anything strictly related to this issue. I could assume a weird side effect of  https://github.com/jenkinsci/jenkins/pull/3980  , but there is no data to confirm it.   batmat dnusbaum  .dat files and format changes should be related to Running builds only, I am not sure these files are used for completed builds. Any chance we have issues with build status persistency in  build.xml    nico_echegut If you have build.xml files and System logs after the upgade to 2.175, it would help a lot

          Devin Nusbaum added a comment -

          Any chance we have issues with build status persistency in build.xml

          We have had issues with this kind of thing in the past (JENKINS-50199), but I am not aware of any related issues in recent plugin versions, although it does remind me of the issues some people are seeing where Pipeline Jobs seem to hold onto flyweight executors after they have apparently completed (discussed in JENKINS-45571). nico_echegut If you have build.xml files from before the upgrade and can tell us whether you updated the Pipeline Job, Pipeline Groovy, or Pipeline Supporting APIs plugins and if so, what versions they were previously and what versions they are now, that would help.

          Devin Nusbaum added a comment - Any chance we have issues with build status persistency in build.xml We have had issues with this kind of thing in the past ( JENKINS-50199 ), but I am not aware of any related issues in recent plugin versions, although it does remind me of the issues some people are seeing where Pipeline Jobs seem to hold onto flyweight executors after they have apparently completed (discussed in JENKINS-45571 ). nico_echegut If you have build.xml files from before the upgrade and can tell us whether you updated the Pipeline Job, Pipeline Groovy, or Pipeline Supporting APIs plugins and if so, what versions they were previously and what versions they are now, that would help.

          I grabbed the build.xml from a build before and after restart. 

          I updated workflow-support from 3.2 to 3.3 right before upgrading Jenkins, but restarted Jenkins in-between. I was already on the latest version of workflow-job-plugin and workflow-cps-plugin.

          Nicolas Echegut added a comment - I grabbed the build.xml from a build before and after restart.  I updated workflow-support from 3.2 to 3.3 right before upgrading Jenkins, but restarted Jenkins in-between. I was already on the latest version of workflow-job-plugin and workflow-cps-plugin.
          Nicolas Echegut made changes -
          Attachment New: build_before.xml [ 46965 ]
          Nicolas Echegut made changes -
          Attachment New: build_after.xml [ 46966 ]

            Unassigned Unassigned
            nico_echegut Nicolas Echegut
            Votes:
            2 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: