• Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • integrity-plugin
    • None
    • Windows server 2012
      Jenkins 2.40
      PTC integrity Plugin 2.0.2

      I just upgraded system from Jenkins 1.644 to 2.40
      At the same time I have upgraded PTC integrity Plugin from 1.3.6 to 2.0.2
      I have updated config of some build job (at timeout condition) and started to test.
      Unfortunately, synchronizing workspace failed buy timeout did not happened after specified time!

      Below part of the corresponding log:

      Time Elapsed 00:00:02.45
      Success build fororg.jenkinsci.plugins.conditionalbuildstep.singlestep.SingleConditionalBuilder@f00640
      Change Log: http://ds-jenk2-1.biotronik.com:8080/job/TDE.Common_Libraries-ECM_Utilities/343/changes
      Build Log: http://ds-jenk2-1.biotronik.com:8080/job/TDE.Common_Libraries-ECM_Utilities/343/console
      Preparing to execute si projectinfo for /repository/TDE/Common_Libraries/ECM_Storage/project.pj
      Preparing to execute si viewproject for #/repository/TDE#Common_Libraries/ECM_Storage
      Checkout directory is D:\Jenkins_ROOT\TDE\Common_Libraries\ECM_Storage
      A clean copy is requested; deleting contents of D:\Jenkins_ROOT\TDE\Common_Libraries\ECM_Storage
      Populating clean workspace...
      A Timeout Exception was caught. Failed to checkout contents of file!
      null
      ERROR: Failed to synchronize workspace!
      Retrying after 10 seconds
      Change Log: http://ds-jenk2-1.biotronik.com:8080/job/TDE.Common_Libraries-ECM_Utilities/343/changes
      Build Log: http://ds-jenk2-1.biotronik.com:8080/job/TDE.Common_Libraries-ECM_Utilities/343/console
      Preparing to execute si projectinfo for /repository/TDE/Common_Libraries/ECM_Storage/project.pj
      Preparing to execute si viewproject for #/repository/TDE#Common_Libraries/ECM_Storage
      Checkout directory is D:\Jenkins_ROOT\TDE\Common_Libraries\ECM_Storage
      A clean copy is requested; deleting contents of D:\Jenkins_ROOT\TDE\Common_Libraries\ECM_Storage
      Populating clean workspace...
      A Timeout Exception was caught. Failed to checkout contents of file!
      null
      ERROR: Failed to synchronize workspace!
      Retrying after 10 seconds
      Change Log: http://ds-jenk2-1.biotronik.com:8080/job/TDE.Common_Libraries-ECM_Utilities/343/changes
      Build Log: http://ds-jenk2-1.biotronik.com:8080/job/TDE.Common_Libraries-ECM_Utilities/343/console
      Preparing to execute si projectinfo for /repository/TDE/Common_Libraries/ECM_Storage/project.pj
      Preparing to execute si viewproject for #/repository/TDE#Common_Libraries/ECM_Storage
      A SQL Exception was caught!
      Table/View 'SCM_C380AD5E_FCBF_4350_8B88_0FE304E3C602' does not exist.
      ERROR: Caught Derby SQLException!
      Retrying after 10 seconds
      Build was aborted
      Aborted by Grigoriy Milman
      Archiving artifacts
      Recording fingerprints
      Build failed! Skipping Integrity Checkpoint step!
      Started calculate disk usage of build
      Finished Calculation of disk usage of build in 0 seconds
      Started calculate disk usage of workspace
      Finished Calculation of disk usage of workspace in 0 seconds
      Warning: you have no plugins providing access control for builds, so falling back to legacy behavior of permitting any downstream builds to be triggered
      Finished: ABORTED

          [JENKINS-41054] PTC plugin failed to timeout

          PTC ALM added a comment -

          Hi Grigoriy,
          Have you re-created the build jobs post the upgrade to 2.0.2?

          PTC ALM added a comment - Hi Grigoriy, Have you re-created the build jobs post the upgrade to 2.0.2?

          Grigoriy Milman added a comment - - edited

          It first I did re-saved not all of the build jobs but just some of them with new PTC timeout settings.
          At that moment I have seen the issue I have reported for one of the re-saved job.

          Now I have all of the build jobs re-saved with new PTC timeout settings. I can't say for sure that I have seen failure to timeout. what I have found today that one build job did perform "si" resync successfully but at checkpoint failed:

          Preparing to execute si checkpoint for #/repository/Operations#ESS/Application Software/SWCode/Scanner/pcProx
          FATAL: Failed to checkpoint /repository/Operations/ESS/Application Software/SWCode/Scanner/pcProx/project.pj: MKS125210: The project history is being changed by another user.
          Build step 'Integrity - CM Checkpoint' marked build as failure

          By the way, for PTC plugin upgrade 2.0.2 is it enough to re-save build job config with new PTC timeout parameter or build job should be re-created from scratch? Can it be recreated based on the original one of absolutely brand new?
          If re-created, that means job history will be lost... May be it it is possible just to add something to job config manually?
          The attached file config.xml is config for some of ours build job re-saved after upgrade.

          Grigoriy Milman added a comment - - edited It first I did re-saved not all of the build jobs but just some of them with new PTC timeout settings. At that moment I have seen the issue I have reported for one of the re-saved job. Now I have all of the build jobs re-saved with new PTC timeout settings. I can't say for sure that I have seen failure to timeout. what I have found today that one build job did perform "si" resync successfully but at checkpoint failed: Preparing to execute si checkpoint for #/repository/Operations#ESS/Application Software/SWCode/Scanner/pcProx FATAL: Failed to checkpoint /repository/Operations/ESS/Application Software/SWCode/Scanner/pcProx/project.pj: MKS125210: The project history is being changed by another user. Build step 'Integrity - CM Checkpoint' marked build as failure By the way, for PTC plugin upgrade 2.0.2 is it enough to re-save build job config with new PTC timeout parameter or build job should be re-created from scratch? Can it be recreated based on the original one of absolutely brand new? If re-created, that means job history will be lost... May be it it is possible just to add something to job config manually? The attached file config.xml is config for some of ours build job re-saved after upgrade.

          PTC ALM added a comment -

          Its recommended to recreate the jobs, since we changed some of the code that relate to how the jobs are referred to in the Derby database. Not doing so may result in the "Table/View 'SCM_C380AD5E_FCBF_4350_8B88_0FE304E3C602' does not exist." sort of errors, that was there in the log you posted. We did put some precaution in for this error not to happen, if we did just a re-save, but not completely guaranteed.

          The checkpoint error looks to be because because si checkpoint is happening at the time of some other operation that might be affecting the repository at the same time. The stacktrace on the Integrity Server would be more helpful in this case to diagnose (if this happening frequently).

          PTC ALM added a comment - Its recommended to recreate the jobs, since we changed some of the code that relate to how the jobs are referred to in the Derby database. Not doing so may result in the "Table/View 'SCM_C380AD5E_FCBF_4350_8B88_0FE304E3C602' does not exist." sort of errors, that was there in the log you posted. We did put some precaution in for this error not to happen, if we did just a re-save, but not completely guaranteed. The checkpoint error looks to be because because si checkpoint is happening at the time of some other operation that might be affecting the repository at the same time. The stacktrace on the Integrity Server would be more helpful in this case to diagnose (if this happening frequently).

          1. We can't recreate all jobs because in this case the jobs history will be lost.
          2. Something new: It looks like after creation of the new build job based on the one which already exit does not work any more.
          Both jobs original and new fail "si" step:
          Original job related log:

          Preparing to execute si projectinfo for /repository/CorporateMeasurementTechnologies/SWProduction/Tools/BuildProcess/JenkinsMSEI/GetBuildResults/project.pj
          Preparing to execute si viewproject for #/repository/CorporateMeasurementTechnologies#s=SWProduction/SWProduction.pj#s=Tools/Tools.pj#BuildProcess/JenkinsMSEI/GetBuildResults
          Checkout directory is D:\Jenkins_ROOT\NGMP.DIRECT.Sandbox\SWProduction\Tools\BuildProcess\JenkinsMSEI\GetBuildResults
          A clean copy is requested; deleting contents of D:\Jenkins_ROOT\NGMP.DIRECT.Sandbox\SWProduction\Tools\BuildProcess\JenkinsMSEI\GetBuildResults
          Populating clean workspace...
          A Timeout Exception was caught. Failed to checkout contents of file!
          null
          ERROR: Failed to synchronize workspace!

          New build job related log entry:

          Preparing to execute si projectinfo for /repository/CorporateMeasurementTechnologies/SWProduction/Tools/BuildProcess/JenkinsMSEI/GetBuildResults4.6/project.pj
          Preparing to execute si viewproject for #/repository/CorporateMeasurementTechnologies#s=SWProduction/SWProduction.pj#s=Tools/Tools.pj#BuildProcess/JenkinsMSEI/GetBuildResults4.6
          A SQL Exception was caught!
          Table/View 'SCM_6A71D9B4_FDA4_41C2_9E2E_B2B64C882F9E' does not exist.
          ERROR: Caught Derby SQLException!
          Retrying after 10 seconds

          Grigoriy Milman added a comment - 1. We can't recreate all jobs because in this case the jobs history will be lost. 2. Something new: It looks like after creation of the new build job based on the one which already exit does not work any more. Both jobs original and new fail "si" step: Original job related log: Preparing to execute si projectinfo for /repository/CorporateMeasurementTechnologies/SWProduction/Tools/BuildProcess/JenkinsMSEI/GetBuildResults/project.pj Preparing to execute si viewproject for #/repository/CorporateMeasurementTechnologies#s=SWProduction/SWProduction.pj#s=Tools/Tools.pj#BuildProcess/JenkinsMSEI/GetBuildResults Checkout directory is D:\Jenkins_ROOT\NGMP.DIRECT.Sandbox\SWProduction\Tools\BuildProcess\JenkinsMSEI\GetBuildResults A clean copy is requested; deleting contents of D:\Jenkins_ROOT\NGMP.DIRECT.Sandbox\SWProduction\Tools\BuildProcess\JenkinsMSEI\GetBuildResults Populating clean workspace... A Timeout Exception was caught. Failed to checkout contents of file! null ERROR: Failed to synchronize workspace! New build job related log entry: Preparing to execute si projectinfo for /repository/CorporateMeasurementTechnologies/SWProduction/Tools/BuildProcess/JenkinsMSEI/GetBuildResults4.6/project.pj Preparing to execute si viewproject for #/repository/CorporateMeasurementTechnologies#s=SWProduction/SWProduction.pj#s=Tools/Tools.pj#BuildProcess/JenkinsMSEI/GetBuildResults4.6 A SQL Exception was caught! Table/View 'SCM_6A71D9B4_FDA4_41C2_9E2E_B2B64C882F9E' does not exist. ERROR: Caught Derby SQLException! Retrying after 10 seconds

          PTC ALM added a comment -

          Based on the first exception(original job), can you try increasing the Checkout thread timeout parameter in the old job configuration to something greater than the default 5 mins? Say 10 mins or 15 mins and recheck.

          I am not entirely sure why a new job would throw a Derby SQL exception. Just a couple of things for my understanding: did you 'copy' the existing job or created a brand new one from scratch? Also can you send across the IntegritySCM log for this scenario, if you can. The IntegritySCM logger setup can be found here: https://wiki.jenkins-ci.org/display/JENKINS/PTC+Integrity+Plugin#PTCIntegrityPlugin-Troubleshooting

          PTC ALM added a comment - Based on the first exception(original job), can you try increasing the Checkout thread timeout parameter in the old job configuration to something greater than the default 5 mins? Say 10 mins or 15 mins and recheck. I am not entirely sure why a new job would throw a Derby SQL exception. Just a couple of things for my understanding: did you 'copy' the existing job or created a brand new one from scratch? Also can you send across the IntegritySCM log for this scenario, if you can. The IntegritySCM logger setup can be found here: https://wiki.jenkins-ci.org/display/JENKINS/PTC+Integrity+Plugin#PTCIntegrityPlugin-Troubleshooting

          Some clarifications. After upgrade
          1. When all old jobs are re-saved with new "Timeout parameter" and no new job is created, all of them works.
          2. When a new build job is created based on the re-saved old one, all build jobs started to failed "si" command, including just created and and re-saved ones.
          3. When a new build job is created from scratch, all old re-saved build jobs started to failed "si" command, but brand new build job works.

          Grigoriy Milman added a comment - Some clarifications. After upgrade 1. When all old jobs are re-saved with new "Timeout parameter" and no new job is created, all of them works. 2. When a new build job is created based on the re-saved old one, all build jobs started to failed "si" command, including just created and and re-saved ones. 3. When a new build job is created from scratch, all old re-saved build jobs started to failed "si" command, but brand new build job works.

          Paul Sharpe added a comment -

           I have exactly the same problem when upgrading my jobs from 1.36 to 2.2.

          As mentioned, it is not acceptable to create new jobs as we need to keep the history.

          But as Grigoriy has stated, just updating the timeout parameter in the old jobs, seems to allow the checkout to proceed on the old jobs, without creating new ones.

          Thanks Grigoriy!

           

          So the solution to this issue, could be: Update the upgrade documentation and change the default timeout to not be 0?

          Paul Sharpe added a comment -  I have exactly the same problem when upgrading my jobs from 1.36 to 2.2. As mentioned, it is not acceptable to create new jobs as we need to keep the history. But as Grigoriy has stated, just updating the timeout parameter in the old jobs, seems to allow the checkout to proceed on the old jobs, without creating new ones. Thanks Grigoriy!   So the solution to this issue, could be: Update the upgrade documentation and change the default timeout to not be 0?

            integrity_jenkins_plugin PTC ALM
            gremlm Grigoriy Milman
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: