• Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • dimensionsscm-plugin
    • None
    • Jenkins 2.401.1, Dimensions Plugin Version 0.9.3.1, Windows 10, Chrome

      We have almost 150 projects setup in Jenkins, and only one of them gives me this checkout error. This one project is massive and when it builds it takes almost 10 minutes to run and complete. It'll take about 5 minutes before the build errors on the checkout process and then fails. After it fails, I can run the same build again with no changes and it will run successfully. I'm wondering if because the build takes so long and the application is so large that it actually times out on the checkout process, since it will run successfully right after failing and its the only project I have this issue with?

      Here is the error messages I receive when it fails:

       
      Running as SYSTEM
      [EnvInject] - Loading node environment variables.
      Building in workspace D:\.jenkins\workspace\JobNet_Maintenance_Stream
      [DIMENSIONS] Running checkout on master...
      [DIMENSIONS] Running build in 'D:\.jenkins\workspace\JobNet_Maintenance_Stream'...
      [DIMENSIONS] Removing 'file:/D:/.jenkins/workspace/JobNet_Maintenance_Stream/'...
      [DIMENSIONS] Checking out project "JBOSS:JOBNET_MAINT2019"...
      [DIMENSIONS] Checking out directory 'Development\jobnet'...
      FATAL: Unable to run checkout callout (IOException: Exception during checkout (DimensionsRuntimeException: Dimensions command 'UPDATE /BRIEF /DIR="Development\jobnet"/WORKSET="JBOSS:JOBNET_MAINT2019" /USER_DIR="D:\.jenkins\workspace\JobNet_Maintenance_Stream" /NOMETADATA' failed (DimensionsConnectionException: java.net.SocketException: Connection reset)))
      java.io.IOException: Unable to run checkout callout (IOException: Exception during checkout (DimensionsRuntimeException: Dimensions command 'UPDATE /BRIEF /DIR="Development\jobnet"/WORKSET="JBOSS:JOBNET_MAINT2019" /USER_DIR="D:\.jenkins\workspace\JobNet_Maintenance_Stream" /NOMETADATA' failed (DimensionsConnectionException: java.net.SocketException: Connection reset)))
      at hudson.plugins.dimensionsscm.CheckOutAPITask.execute(CheckOutAPITask.java:220)
      at hudson.plugins.dimensionsscm.GenericAPITask.invoke(GenericAPITask.java:53)
      Caused: java.io.IOException: Exception during login (IOException: Unable to run checkout callout (IOException: Exception during checkout (DimensionsRuntimeException: Dimensions command 'UPDATE /BRIEF /DIR="Development\jobnet"/WORKSET="JBOSS:JOBNET_MAINT2019" /USER_DIR="D:\.jenkins\workspace\JobNet_Maintenance_Stream" /NOMETADATA' failed (DimensionsConnectionException: java.net.SocketException: Connection reset))))
      at hudson.plugins.dimensionsscm.GenericAPITask.invoke(GenericAPITask.java:56)
      at hudson.plugins.dimensionsscm.CheckOutAPITask.invoke(CheckOutAPITask.java:20)
      at hudson.plugins.dimensionsscm.GenericAPITask.invoke(GenericAPITask.java:14)
      at hudson.FilePath.act(FilePath.java:1198)
      at hudson.FilePath.act(FilePath.java:1181)
      at hudson.plugins.dimensionsscm.DimensionsSCM.checkout(DimensionsSCM.java:761)
      at hudson.scm.SCM.checkout(SCM.java:540)
      at hudson.model.AbstractProject.checkout(AbstractProject.java:1240)
      at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:649)
      at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:85)
      at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:521)
      at hudson.model.Run.execute(Run.java:1900)
      at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
      at hudson.model.ResourceController.execute(ResourceController.java:101)
      at hudson.model.Executor.run(Executor.java:442)

          [JENKINS-71694] Dimensions checkout fails

          Hi #brumms, it does sound like some kind of timeout. However, 5 or 10 minutes doesn't actually sound that long - I've seen jobs that fetch for 20 or 30 minutes on very large streams.

          One thing to consider is if there is any software on your network that might be cutting the TCP network connection (like a firewall or other network utility) between Jenkins and the Dimensions CM server. This seems a possible explanation given the abrupt 'Connection reset' message you are seeing.

          If you follow the instructions at https://github.com/jenkinsci/dimensionsscm-plugin/blob/master/docs/logging.md (clear the log before a failed build, run a failing build, then copy-and-paste the log contents here) then there may be additional information in the diagnostic logs that helps understand what's going on.

          If we can't get anywhere with it, then there is an option used when uploading artifacts to force the use of *dmcli.exe* instead of the Java API, which I could extend to apply to downloads as well - although of course it's likely dmcli will be affected the same way as the Java API if this is due to some network hardware cutting the network connections. You could also try setting up a Jenkins agent, which would use that option (to use *dmcli.exe* for download) by default

          David Conneely added a comment - Hi # brumms , it does sound like some kind of timeout. However, 5 or 10 minutes doesn't actually sound that long - I've seen jobs that fetch for 20 or 30 minutes on very large streams. One thing to consider is if there is any software on your network that might be cutting the TCP network connection (like a firewall or other network utility) between Jenkins and the Dimensions CM server. This seems a possible explanation given the abrupt 'Connection reset' message you are seeing. If you follow the instructions at https://github.com/jenkinsci/dimensionsscm-plugin/blob/master/docs/logging.md (clear the log before a failed build, run a failing build, then copy-and-paste the log contents here) then there may be additional information in the diagnostic logs that helps understand what's going on. If we can't get anywhere with it, then there is an option used when uploading artifacts to force the use of * dmcli.exe * instead of the Java API, which I could extend to apply to downloads as well - although of course it's likely dmcli will be affected the same way as the Java API if this is due to some network hardware cutting the network connections. You could also try setting up a Jenkins agent, which would use that option (to use * dmcli.exe * for download) by default

          Sara Brumm added a comment -

          Thank you for the tips dconneely , I've added the log you suggested and will keep an eye. The issue is sporadic so I never know when it will happen, it never occurs a second time in a row either, it always works when run it again. I'll update you when I have log information and check into the network connections as well.

          Sara Brumm added a comment - Thank you for the tips dconneely , I've added the log you suggested and will keep an eye. The issue is sporadic so I never know when it will happen, it never occurs a second time in a row either, it always works when run it again. I'll update you when I have log information and check into the network connections as well.

          Sara Brumm added a comment -

          Hi dconneely, the build finally had a failure with this same error. Here is the log information I grabbed from it. Please let me know if you see anything that stands out causing this issue. Thank you!
          Aug 21, 2023 9:05:55 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login
          DimensionsAPI.login - build number: "816", project: "JobNet_Maintenance_Stream"
          Aug 21, 2023 9:05:55 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login
          Checking Dimensions login parameters...
          Aug 21, 2023 9:05:55 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login
          Logging into Dimensions: dmsys mdotdimensions.state.mi.us dba_pcms@cmtl
          Aug 21, 2023 9:05:55 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login
          Getting Dimensions connection...
          Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login
          Connection map key is 12
          Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login
          Connection map size before putIfAbsent is 0
          Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login
          Connection map size after putIfAbsent is 1
          Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login
          key: "12"
          Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsSCM checkout
          Login worked.
          Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI logout
          DimensionsAPI.logout - build number: "816", project: "JobNet_Maintenance_Stream"
          Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI getCon
          Looking for key 12
          Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI getCon
          Found database
          Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI logout
          Closing connection to Dimensions for key "12"...
          Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI logout
          Now have 0 connections in use...
          Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsSCM checkout
          Checking if master or slave...
          Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.GenericAPITask
          Creating task - hudson.plugins.dimensionsscm.CheckOutAPITask
          Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.CheckOutAPITask
          Creating task - hudson.plugins.dimensionsscm.CheckOutAPITask
          Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login
          DimensionsAPI.login - build number: "816", project: "JobNet_Maintenance_Stream"
          Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login
          Checking Dimensions login parameters...
          Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login
          Logging into Dimensions: dmsys mdotdimensions.state.mi.us dba_pcms@cmtl
          Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login
          Getting Dimensions connection...
          Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login
          Connection map key is 13
          Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login
          Connection map size before putIfAbsent is 0
          Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login
          Connection map size after putIfAbsent is 1
          Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login
          Server information detected -3
          Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login
          0 - 14.6
          Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login
          1 - 2
          Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login
          2 - 14.6.0 Build 9.548
          Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login
          Detected server version: 14.6.0 Build 9.548
          Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login
          Version to process set to 2009
          Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login
          key: "13"
          Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.GenericAPITask invoke
          Login worked.
          Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.CheckOutAPITask execute
          Deleting 'file:/D:/.jenkins/workspace/JobNet_Maintenance_Stream/'...
          Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.CheckOutAPITask execute
          Extra parameters - null null
          Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.CheckOutAPITask execute
          Checking out 'Development/jobnet/'...
          Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI getCon
          Looking for key 13
          Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI getCon
          Found database
          Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI run
          Running the command 'UPDATE /BRIEF /DIR="Development\jobnet"/WORKSET="JBOSS:JOBNET_MAINT2019" /USER_DIR="D:\.jenkins\workspace\JobNet_Maintenance_Stream" /NOMETADATA'...

          Sara Brumm added a comment - Hi dconneely , the build finally had a failure with this same error. Here is the log information I grabbed from it. Please let me know if you see anything that stands out causing this issue. Thank you! Aug 21, 2023 9:05:55 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login DimensionsAPI.login - build number: "816", project: "JobNet_Maintenance_Stream" Aug 21, 2023 9:05:55 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login Checking Dimensions login parameters... Aug 21, 2023 9:05:55 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login Logging into Dimensions: dmsys mdotdimensions.state.mi.us dba_pcms@cmtl Aug 21, 2023 9:05:55 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login Getting Dimensions connection... Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login Connection map key is 12 Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login Connection map size before putIfAbsent is 0 Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login Connection map size after putIfAbsent is 1 Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login key: "12" Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsSCM checkout Login worked. Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI logout DimensionsAPI.logout - build number: "816", project: "JobNet_Maintenance_Stream" Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI getCon Looking for key 12 Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI getCon Found database Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI logout Closing connection to Dimensions for key "12"... Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI logout Now have 0 connections in use... Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsSCM checkout Checking if master or slave... Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.GenericAPITask Creating task - hudson.plugins.dimensionsscm.CheckOutAPITask Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.CheckOutAPITask Creating task - hudson.plugins.dimensionsscm.CheckOutAPITask Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login DimensionsAPI.login - build number: "816", project: "JobNet_Maintenance_Stream" Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login Checking Dimensions login parameters... Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login Logging into Dimensions: dmsys mdotdimensions.state.mi.us dba_pcms@cmtl Aug 21, 2023 9:05:56 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login Getting Dimensions connection... Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login Connection map key is 13 Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login Connection map size before putIfAbsent is 0 Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login Connection map size after putIfAbsent is 1 Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login Server information detected -3 Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login 0 - 14.6 Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login 1 - 2 Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login 2 - 14.6.0 Build 9.548 Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login Detected server version: 14.6.0 Build 9.548 Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login Version to process set to 2009 Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI login key: "13" Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.GenericAPITask invoke Login worked. Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.CheckOutAPITask execute Deleting 'file:/D:/.jenkins/workspace/JobNet_Maintenance_Stream/'... Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.CheckOutAPITask execute Extra parameters - null null Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.CheckOutAPITask execute Checking out 'Development/jobnet/'... Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI getCon Looking for key 13 Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI getCon Found database Aug 21, 2023 9:05:57 AM FINE hudson.plugins.dimensionsscm.DimensionsAPI run Running the command 'UPDATE /BRIEF /DIR="Development\jobnet"/WORKSET="JBOSS:JOBNET_MAINT2019" /USER_DIR="D:\.jenkins\workspace\JobNet_Maintenance_Stream" /NOMETADATA'...

          brumms, the only odd thing in there is that that the plugin thinks that it's connected to Dimenisons CM 2009 (between 10 and 12), but that's probably just a minor bug. I don't think it's affected the command-line used adversely. Everything else looks fine, and there's no error information to work with.

          One thing you could try is to check that `darius.jar`, `dmnet.jar`, `dmclient.jar`, etc. that you installed into the plugin are all the correct versions from the Dimensions CM server version you are now running (it's easy to forget to update them when updating the server), but this is grasping at straws.

          If that doesn't help, then all i can do is spend some time to add a flag in a new version of the Jenkins Dimensions plugin to use `dmcli` command to run a local UPDATE instead of the Java API, so you can see if that helps.

          David Conneely added a comment - brumms , the only odd thing in there is that that the plugin thinks that it's connected to Dimenisons CM 2009 (between 10 and 12), but that's probably just a minor bug. I don't think it's affected the command-line used adversely. Everything else looks fine, and there's no error information to work with. One thing you could try is to check that `darius.jar`, `dmnet.jar`, `dmclient.jar`, etc. that you installed into the plugin are all the correct versions from the Dimensions CM server version you are now running (it's easy to forget to update them when updating the server), but this is grasping at straws. If that doesn't help, then all i can do is spend some time to add a flag in a new version of the Jenkins Dimensions plugin to use `dmcli` command to run a local UPDATE instead of the Java API, so you can see if that helps.

          Sara Brumm added a comment -

          Hi dconneely , I checked and verified the jar files are up to date and using the same ones from Dimensions CM and they are. This hasn't been a huge issue (although would be nice to have fixed), as normally we can just kick the build off again and then it'll work the 2nd time. Today the same project build has now errored 3 times in a row, which has never happened before.

          Sara Brumm added a comment - Hi dconneely , I checked and verified the jar files are up to date and using the same ones from Dimensions CM and they are. This hasn't been a huge issue (although would be nice to have fixed), as normally we can just kick the build off again and then it'll work the 2nd time. Today the same project build has now errored 3 times in a row, which has never happened before.

            Unassigned Unassigned
            brumms Sara Brumm
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: