Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-59754

Jenkins Micro Focus Application Automation Tools Plugin losing connection with Octane after running pipelines

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      We have configured Octane / Jenkins integration by installing ALM Octane plugin in Jenkins. The connection works for a day and then drops.

      Jenkins is setup as a CI server in Octane, we have found that if we run a pipeline everyday with a Jenkins build that the connection between Jenkins and Octane drops and the Jenkins plugin for Octane cannot get a new connection to the Octane server.

      A few things we have noticed:

      The issue only happens if we run a build once a day, if we have a pipeline running every hour the issue does not seem to happen. We believe that the plugin may not be managing the sessions (cookies/tokens) with Octane correctly and that by running a build every hour might be keeping the session "alive"

      When this happens our only fix seems to be restarting the Jenkins server which we think might make the plugin renegotiate the session with Octane

      We tried the newest plugin 5.9 to see if it might resolve the issue but it did not

        Attachments

          Activity

          Hide
          radislavb Radi Berkovich added a comment - - edited

          Hi.

          Session to Octane is valid for 3 hours (be default). Plugin is expected to renew the connection if it is required to communicate with Octane and recognize that session is over.

          Please explain what happen with plugin when it "drop" the connection.

          Do you see some exception in logs?

           You can check plugin log here : <jenkins_url>/userContent/nga/logs/nga.log

           

          Show
          radislavb Radi Berkovich added a comment - - edited Hi. Session to Octane is valid for 3 hours (be default). Plugin is expected to renew the connection if it is required to communicate with Octane and recognize that session is over. Please explain what happen with plugin when it "drop" the connection. Do you see some exception in logs?  You can check plugin log here : <jenkins_url>/userContent/nga/logs/nga.log  
          Hide
          jmyates86 Joshua Yates added a comment -

          Attached are logs from a few weeks back (ignore the exceptions from a long time back)

          We see alot of lines like this:

          Tests:177:STARTED] event/s ...
          28/09/2019 01:19:43,990 INFO [EventsServiceWorker-57 ] OctaneRestClientImpl : https://ServerURL:8443?p=1001 doing RE-LOGIN due to status 401 received while calling https://serverURL:8443/internal-api/shared_spaces/1001/analytics/ci/events?ci_server_identity=365fbec0-6279-4978-840d-b6964f56bab1
          28/09/2019 01:19:44,033 INFO [EventsServiceWorker-57 ] OctaneRestClientImpl : https://ServerURL:8443?p=1001 re-attempting the original request (https://serverURL:8443/internal-api/shared_spaces/1001/analytics/ci/events?ci_server_identity=365fbec0-6279-4978-840d-b6964f56bab1) having successful RE-LOGIN

          The problem is I don't believe that the plugin ever successfully renegotiates the session with octane, not sure if there are any settings we should try....

          Thanks

          Show
          jmyates86 Joshua Yates added a comment - Attached are logs from a few weeks back (ignore the exceptions from a long time back) We see alot of lines like this: Tests:177:STARTED] event/s ... 28/09/2019 01:19:43,990 INFO [EventsServiceWorker-57 ] OctaneRestClientImpl : https://ServerURL:8443?p=1001 doing RE-LOGIN due to status 401 received while calling https://serverURL:8443/internal-api/shared_spaces/1001/analytics/ci/events?ci_server_identity=365fbec0-6279-4978-840d-b6964f56bab1 28/09/2019 01:19:44,033 INFO [EventsServiceWorker-57 ] OctaneRestClientImpl : https://ServerURL:8443?p=1001 re-attempting the original request ( https://serverURL:8443/internal-api/shared_spaces/1001/analytics/ci/events?ci_server_identity=365fbec0-6279-4978-840d-b6964f56bab1 ) having successful RE-LOGIN The problem is I don't believe that the plugin ever successfully renegotiates the session with octane, not sure if there are any settings we should try.... Thanks
          Hide
          radislavb Radi Berkovich added a comment - - edited

          I actually see that relogin succedded - "having successful RE-LOGIN".

          Please explain what happen with plugin when it "drop" the connection, what exactly happen.

          Show
          radislavb Radi Berkovich added a comment - - edited I actually see that relogin succedded - "having successful RE-LOGIN". Please explain what happen with plugin when it "drop" the connection, what exactly happen.
          Hide
          radislavb Radi Berkovich added a comment -

          HI.

          I need more information - what is not working after some time?

          In log file , I see that relogins are succedded. 

          The best will be , if you encounter you  problem - take logs file and give me the time it happen, so it will be easier to search relevant log rows.

           

          Thanks.

          Radi.

           

          Show
          radislavb Radi Berkovich added a comment - HI. I need more information - what is not working after some time? In log file , I see that relogins are succedded.  The best will be , if you encounter you  problem - take logs file and give me the time it happen, so it will be easier to search relevant log rows.   Thanks. Radi.  
          Hide
          jmyates86 Joshua Yates added a comment -

          Hello Radi,

          What happens is if they go more than a few hours without running a build It will not run a build from Octane. At the time that this is happening if you check Octane it does not show the Jenkins CI server as connected during that period, even though we can ping the jenkins server from Octane at the time that Octane is saying that it cannot get to jenkins.

          The log files that we sent had a period in there where it could not connect but that is the weird thing, the plugin seems to think that it is creating a new session but we do not think that this is actually happening..... And as soon as Jenkins is restarted the issue goes away

          Thanks

          Show
          jmyates86 Joshua Yates added a comment - Hello Radi, What happens is if they go more than a few hours without running a build It will not run a build from Octane. At the time that this is happening if you check Octane it does not show the Jenkins CI server as connected during that period, even though we can ping the jenkins server from Octane at the time that Octane is saying that it cannot get to jenkins. The log files that we sent had a period in there where it could not connect but that is the weird thing, the plugin seems to think that it is creating a new session but we do not think that this is actually happening..... And as soon as Jenkins is restarted the issue goes away Thanks
          Hide
          radislavb Radi Berkovich added a comment -

          Hi.

          Unfortunately we did not succeeded to reproduce the issue in our environment.

          Can I create plugin version with additional log message and you will use it for several days?

           

          Thanks.  

          Show
          radislavb Radi Berkovich added a comment - Hi. Unfortunately we did not succeeded to reproduce the issue in our environment. Can I create plugin version with additional log message and you will use it for several days?   Thanks.  
          Hide
          jmyates86 Joshua Yates added a comment -

          Yes that would be fine, send us the modified plugin and we can install that and get the logs

          Show
          jmyates86 Joshua Yates added a comment - Yes that would be fine, send us the modified plugin and we can install that and get the logs
          Show
          radislavb Radi Berkovich added a comment - Please use v5.9.2-beta :  http://repo.jenkins-ci.org/releases/org/jenkins-ci/plugins/hp-application-automation-tools-plugin/5.9.2-beta/hp-application-automation-tools-plugin-5.9.2-beta.hpi  
          Hide
          radislavb Radi Berkovich added a comment -

          please update me if you already used 5.9.2

          .

          Show
          radislavb Radi Berkovich added a comment - please update me if you already used 5.9.2 .
          Hide
          radislavb Radi Berkovich added a comment -

          If you still didn't installed previous link, please you this :

          (its the same code, it just official beta version)

          http://repo.jenkins-ci.org/releases/org/jenkins-ci/plugins/hp-application-automation-tools-plugin/5.9.3-beta/hp-application-automation-tools-plugin-5.9.3-beta.hpi

           

          Show
          radislavb Radi Berkovich added a comment - If you still didn't installed previous link, please you this : (its the same code, it just official beta version) http://repo.jenkins-ci.org/releases/org/jenkins-ci/plugins/hp-application-automation-tools-plugin/5.9.3-beta/hp-application-automation-tools-plugin-5.9.3-beta.hpi  
          Hide
          radislavb Radi Berkovich added a comment -

          any updates?

          Show
          radislavb Radi Berkovich added a comment - any updates?
          Hide
          radislavb Radi Berkovich added a comment - - edited

          Official version 6.0 is released .

          Please check if issue is reproduced on this version.

          Show
          radislavb Radi Berkovich added a comment - - edited Official version 6.0 is released . Please check if issue is reproduced on this version.

            People

            Assignee:
            radislavb Radi Berkovich
            Reporter:
            jmyates86 Joshua Yates
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: