Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-13032

Updating cvs plugin to version 2.0 causes "java.lang.RuntimeException: CVS authentication failure while running rlog command"

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Blocker Blocker
    • cvs-plugin
    • CentOS release 5.4 64-bit, Tomcat 6.0.20, Jenkins 1.454

      On version 1.6 everything was ok. Now every job using cvs fails.

      After updating cvs plugin from version 1.6 to 2.0 I get:

      FATAL: CVS authentication failure while running rlog command
      java.lang.RuntimeException: CVS authentication failure while running rlog command
      at hudson.scm.CVSSCM.getRemoteLogForModule(CVSSCM.java:529)
      at hudson.scm.CVSSCM.calculateChangeLog(CVSSCM.java:409)
      at hudson.scm.CVSSCM.checkout(CVSSCM.java:829)
      at hudson.model.AbstractProject.checkout(AbstractProject.java:1195)
      at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:579)
      at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:468)
      at hudson.model.Run.run(Run.java:1408)
      at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
      at hudson.model.ResourceController.execute(ResourceController.java:88)
      at hudson.model.Executor.run(Executor.java:238)
      Caused by: org.netbeans.lib.cvsclient.connection.AuthenticationException: Timeout, no response from server.
      at org.netbeans.lib.cvsclient.Client.ensureConnection(Client.java:418)
      at org.netbeans.lib.cvsclient.command.log.RlogCommand.execute(RlogCommand.java:357)
      at org.netbeans.lib.cvsclient.Client.executeCommand(Client.java:710)
      at hudson.scm.CVSSCM.getRemoteLogForModule(CVSSCM.java:521)
      ... 9 more

          [JENKINS-13032] Updating cvs plugin to version 2.0 causes "java.lang.RuntimeException: CVS authentication failure while running rlog command"

          Joël Royer added a comment - - edited

          I've got the same problem since version 2.0 of the plugin.
          I've got timeouts randomly. However, the CVS server is available, I can ping it from the jenkins server.

          Windows Server 2003 / Glassfish 3.1.2.2 / Oracle SDK 1.7.0_07 / Jenkins 1.486 / CVS Plugin 2.0

          Here is the log in the console:

          Building in workspace C:\PIC\work\jenkins-work\jobs\OX1_V011703 - etape 02 - TI - CAP\workspace
          Using locally configured password for connection to :pserver:hudson@srvcvs1:/cvsroot
          cvs checkout -P -r OX1_V011703 -d web src/web
          ERROR: CVS Authentication failed: Timeout, no response from server.
          org.netbeans.lib.cvsclient.connection.AuthenticationException: Timeout, no response from server.
          at org.netbeans.lib.cvsclient.Client.ensureConnection(Client.java:418)
          at org.netbeans.lib.cvsclient.command.checkout.CheckoutCommand.execute(CheckoutCommand.java:299)
          at org.netbeans.lib.cvsclient.Client.executeCommand(Client.java:710)
          at hudson.scm.AbstractCvs$1.invoke(AbstractCvs.java:243)
          at hudson.scm.AbstractCvs$1.invoke(AbstractCvs.java:223)
          at hudson.FilePath.act(FilePath.java:851)
          at hudson.FilePath.act(FilePath.java:824)
          at hudson.scm.AbstractCvs.perform(AbstractCvs.java:223)
          at hudson.scm.AbstractCvs.checkout(AbstractCvs.java:189)
          at hudson.scm.CVSSCM.checkout(CVSSCM.java:318)
          at hudson.model.AbstractProject.checkout(AbstractProject.java:1308)
          at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:679)
          at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88)
          at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:584)
          at hudson.model.Run.execute(Run.java:1516)
          at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:477)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:236)
          ERROR: Cvs task failed
          Archiving artifacts
          Finished: FAILURE

          Joël Royer added a comment - - edited I've got the same problem since version 2.0 of the plugin. I've got timeouts randomly. However, the CVS server is available, I can ping it from the jenkins server. Windows Server 2003 / Glassfish 3.1.2.2 / Oracle SDK 1.7.0_07 / Jenkins 1.486 / CVS Plugin 2.0 Here is the log in the console: Building in workspace C:\PIC\work\jenkins-work\jobs\OX1_V011703 - etape 02 - TI - CAP\workspace Using locally configured password for connection to :pserver:hudson@srvcvs1:/cvsroot cvs checkout -P -r OX1_V011703 -d web src/web ERROR: CVS Authentication failed: Timeout, no response from server. org.netbeans.lib.cvsclient.connection.AuthenticationException: Timeout, no response from server. at org.netbeans.lib.cvsclient.Client.ensureConnection(Client.java:418) at org.netbeans.lib.cvsclient.command.checkout.CheckoutCommand.execute(CheckoutCommand.java:299) at org.netbeans.lib.cvsclient.Client.executeCommand(Client.java:710) at hudson.scm.AbstractCvs$1.invoke(AbstractCvs.java:243) at hudson.scm.AbstractCvs$1.invoke(AbstractCvs.java:223) at hudson.FilePath.act(FilePath.java:851) at hudson.FilePath.act(FilePath.java:824) at hudson.scm.AbstractCvs.perform(AbstractCvs.java:223) at hudson.scm.AbstractCvs.checkout(AbstractCvs.java:189) at hudson.scm.CVSSCM.checkout(CVSSCM.java:318) at hudson.model.AbstractProject.checkout(AbstractProject.java:1308) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:679) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:584) at hudson.model.Run.execute(Run.java:1516) at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:477) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:236) ERROR: Cvs task failed Archiving artifacts Finished: FAILURE

          See https://github.com/jenkinsci/cvsclient/blob/master/src/main/java/org/netbeans/lib/cvsclient/Client.java#L361

          This is the method that's causing the error you're seeing whilst it's trying to do it open a connection to the server, with a timeout of 1 minute before failing the connection. If the server fails to accept the connection within this time then the exception you see is thrown. This method doesn't issue any CVS commands, only looks for a network port connection so would be impacted by proxy servers and/or firewall restrictions. Do you have either of these between your Jenkins server and your CVS server?

          Note: Pinging your server only shows the network stack is responding, CVS Server sits ontop of this and could be failing to respond whilst the ping command shows the server as being available.

          Michael Clarke added a comment - See https://github.com/jenkinsci/cvsclient/blob/master/src/main/java/org/netbeans/lib/cvsclient/Client.java#L361 This is the method that's causing the error you're seeing whilst it's trying to do it open a connection to the server, with a timeout of 1 minute before failing the connection. If the server fails to accept the connection within this time then the exception you see is thrown. This method doesn't issue any CVS commands, only looks for a network port connection so would be impacted by proxy servers and/or firewall restrictions. Do you have either of these between your Jenkins server and your CVS server? Note: Pinging your server only shows the network stack is responding, CVS Server sits ontop of this and could be failing to respond whilst the ping command shows the server as being available.

          Joël Royer added a comment -

          Yes, we have a proxy. I setted up Jenkins to use this proxy in the plugin management, but I added the CVS server hostname in exception list ("No proxy host"). Is the plugin using those settings?

          Moreover, we have a strange behavior. When we got this error, if we restart the application server, CVS server is reachable.

          Joël Royer added a comment - Yes, we have a proxy. I setted up Jenkins to use this proxy in the plugin management, but I added the CVS server hostname in exception list ("No proxy host"). Is the plugin using those settings? Moreover, we have a strange behavior. When we got this error, if we restart the application server, CVS server is reachable.

          CVS plugin doesn't use Jenkins network settings and will only ever attempt a direct connection unless you set values in the CVSROOT entry (CVSROOT supports proxy settings from what I remember but this would be difficult to create automatically from the Jenkins configuration given exclude rules etc).

          By 'application server', is this the app server hosting Jenkins? At what point does your issue re-occur (is this random or do you see other issues before hand (hanging jobs, failing jobs etc)? Once the issue does re-occur, do your jobs keep failing until restart or do they recover themselves and, if so, is there a pattern to the recovery (a set time between fail and first recovery, killing selected process on the Jenkins server etc)?

          Michael Clarke added a comment - CVS plugin doesn't use Jenkins network settings and will only ever attempt a direct connection unless you set values in the CVSROOT entry (CVSROOT supports proxy settings from what I remember but this would be difficult to create automatically from the Jenkins configuration given exclude rules etc). By 'application server', is this the app server hosting Jenkins? At what point does your issue re-occur (is this random or do you see other issues before hand (hanging jobs, failing jobs etc)? Once the issue does re-occur, do your jobs keep failing until restart or do they recover themselves and, if so, is there a pattern to the recovery (a set time between fail and first recovery, killing selected process on the Jenkins server etc)?

          Joël Royer added a comment -

          I'm using GlassFish App Server v3.1.2.2.
          The issue re-occurs from time to time but when it reoccurs, the only solution we've found for the moment is to stop et start GlassFish. There is no automatic recover.

          Joël Royer added a comment - I'm using GlassFish App Server v3.1.2.2. The issue re-occurs from time to time but when it reoccurs, the only solution we've found for the moment is to stop et start GlassFish. There is no automatic recover.

          Next time you have the failure, could you go to /threadDump on your Jenkins host (e.g. http://localhost:8080/threadDump) before and after your restart so I can see if there's any rogue threads hanging around.

          Michael Clarke added a comment - Next time you have the failure, could you go to /threadDump on your Jenkins host (e.g. http://localhost:8080/threadDump ) before and after your restart so I can see if there's any rogue threads hanging around.

          Joël Royer added a comment -

          The bug re-occurs today.
          I joined a zip file which contains thread dumps before and after server reboot.

          Joël Royer added a comment - The bug re-occurs today. I joined a zip file which contains thread dumps before and after server reboot.

          Joël Royer added a comment - - edited

          The bug re-occurs today.
          Here is a zip file which contains thread dumps: 1 before restart and 1 after restart

          (See joined file threadDumps.zip)

          Joël Royer added a comment - - edited The bug re-occurs today. Here is a zip file which contains thread dumps: 1 before restart and 1 after restart (See joined file threadDumps.zip)

          Code changed in jenkins
          User: Michael Clarke
          Path:
          src/main/java/org/netbeans/lib/cvsclient/connection/PServerConnection.java
          http://jenkins-ci.org/commit/cvsclient/44ec053fafb65c44fc6fd4d88ce1d3e13e0da5d9
          Log:
          JENKINS-13032: Force timeout on initial connection to CVS Server

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Michael Clarke Path: src/main/java/org/netbeans/lib/cvsclient/connection/PServerConnection.java http://jenkins-ci.org/commit/cvsclient/44ec053fafb65c44fc6fd4d88ce1d3e13e0da5d9 Log: JENKINS-13032 : Force timeout on initial connection to CVS Server

          Code changed in jenkins
          User: Michael Clarke
          Path:
          pom.xml
          http://jenkins-ci.org/commit/cvs-plugin/443d3a1d640432d530cca78f6b777f966f2197a0
          Log:
          [FIXED JENKINS-13032] Upgrade to latest CVS Client with socket timeout

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Michael Clarke Path: pom.xml http://jenkins-ci.org/commit/cvs-plugin/443d3a1d640432d530cca78f6b777f966f2197a0 Log: [FIXED JENKINS-13032] Upgrade to latest CVS Client with socket timeout

            mc1arke Michael Clarke
            jacekw jacekw
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: