Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-27538

SCM polling error due to CVS authentication failure while running rlog command ($HOME variable not expanded for private key paths)

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Critical Critical
    • cvs-plugin
    • None
    • Environment : Refer to attched pdf

      Jenkins Version: Jenkins ver. 1.596.1
      Operating System : CentOS 6 (86_x64)
      CVS plugin versions : 2.12
      Concurrent Versions System (CVS) 1.11.23 (client/server)
      Running Jenkins directly

      After upgrading to CVS Plugin 2.12 (from 1.6), the SCM polling is failing.

      1. CVS checkout/update Works fine with same settings (Both on Master & Slaves)
        1. CVS Authentication values : Refer to attached screenshot
      2. Only SCM polling is affected and looking at the error it seems the $HOME Variable not expanded (Though CVS checkout works fine)

      Any help would be greatly appreciated.

      CVS Polling Log
      
      Started on Mar 21, 2015 10:08:00 PM
      cvs rlog -S -d21 Mar 2015 19:13:12 +1100<21 Mar 2015 22:08:00 +1100 lendfast-core-root 
      ERROR: Failed to record SCM polling for hudson.maven.MavenModuleSet@5b0973cf[lendfast-core-root-HEAD-Build]
      java.lang.RuntimeException: CVS authentication failure while running rlog command
      	at hudson.scm.AbstractCvs.executeRlog(AbstractCvs.java:704)
      	at hudson.scm.AbstractCvs.getRemoteLogForModule(AbstractCvs.java:660)
      	at hudson.scm.AbstractCvs.calculateRepositoryState(AbstractCvs.java:611)
      	at hudson.scm.AbstractCvs.compareRemoteRevisionWith(AbstractCvs.java:515)
      	at hudson.scm.CVSSCM.compareRemoteRevisionWith(CVSSCM.java:224)
      	at hudson.scm.SCM.poll(SCM.java:397)
      	at hudson.model.AbstractProject._poll(AbstractProject.java:1446)
      	at hudson.model.AbstractProject.poll(AbstractProject.java:1349)
      	at jenkins.triggers.SCMTriggerItem$SCMTriggerItems$Bridge.poll(SCMTriggerItem.java:119)
      	at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:515)
      	at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:544)
      	at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: org.netbeans.lib.cvsclient.connection.AuthenticationException: SSH connection failed.
      	at org.netbeans.lib.cvsclient.connection.SSHConnection.open(SSHConnection.java:141)
      	at org.netbeans.lib.cvsclient.Client$1.run(Client.java:374)
      	... 1 more
      Caused by: com.jcraft.jsch.JSchException: java.io.FileNotFoundException: $HOME/.ssh/id_dsa (No such file or directory)
      	at com.jcraft.jsch.KeyPair.load(KeyPair.java:524)
      	at com.jcraft.jsch.IdentityFile.newInstance(IdentityFile.java:40)
      	at com.jcraft.jsch.JSch.addIdentity(JSch.java:389)
      	at com.jcraft.jsch.JSch.addIdentity(JSch.java:370)
      	at org.netbeans.lib.cvsclient.connection.SSHConnection.open(SSHConnection.java:135)
      	... 2 more
      Caused by: java.io.FileNotFoundException: $HOME/.ssh/id_dsa (No such file or directory)
      	at java.io.FileInputStream.open0(Native Method)
      	at java.io.FileInputStream.open(FileInputStream.java:195)
      	at java.io.FileInputStream.<init>(FileInputStream.java:138)
      	at java.io.FileInputStream.<init>(FileInputStream.java:93)
      	at com.jcraft.jsch.Util.fromFile(Util.java:492)
      	at com.jcraft.jsch.KeyPair.load(KeyPair.java:521)
      	... 6 more
      

          [JENKINS-27538] SCM polling error due to CVS authentication failure while running rlog command ($HOME variable not expanded for private key paths)

          Sandstone added a comment -

          Not sure if its the same issue (but same symptoms)

          Sandstone added a comment - Not sure if its the same issue (but same symptoms)

          After further investigation, here is a summary of my analysis (Please feel free to comment)

          Symptom

          CVS Polling fails due to missing private Key, even though the CVS checkout/update as part of a manually triggered build works fine. The path to private key (as part of SCM polling) contains the unexpanded $HOME variable, as indicated in the log.

          Cause

          I think the issue is caused by the following line of code in AbstractCvs.java (Line 509)

          AbstractCvs.java
          final EnvVars envVars = project.getLastBuild().getEnvironment(listener);
          

          As the envVars now doesnt seem to contains HOME variable.

          Note: While doing the CVS update/checkout, the envVars seem to be obtained from the current instance of build run containing HOME variable. And that's why the CVS update/checkout works fine.

          Line 106 at AbstractCvs.java
          final EnvVars envVars = build.getEnvironment(listener);
          

          Fix

          I am not sure yet what the best way to approach this problem is. Few things I am exploring are

          1. Investigate why getLastBuild().getEnvironment(listener) doesnt seem to contain HOME variable
          2. Modifying protected PollingResult compareRemoteRevisionWith to obtain complete list of envVars

          Any help or pointers in the right direction would be highly appreciated.

          Chaitanya Pinnamaneni added a comment - After further investigation, here is a summary of my analysis (Please feel free to comment) Symptom CVS Polling fails due to missing private Key, even though the CVS checkout/update as part of a manually triggered build works fine. The path to private key (as part of SCM polling) contains the unexpanded $HOME variable, as indicated in the log. Cause I think the issue is caused by the following line of code in AbstractCvs.java (Line 509) AbstractCvs.java final EnvVars envVars = project.getLastBuild().getEnvironment(listener); As the envVars now doesnt seem to contains HOME variable. Note: While doing the CVS update/checkout, the envVars seem to be obtained from the current instance of build run containing HOME variable. And that's why the CVS update/checkout works fine. Line 106 at AbstractCvs.java final EnvVars envVars = build.getEnvironment(listener); Fix I am not sure yet what the best way to approach this problem is. Few things I am exploring are Investigate why getLastBuild().getEnvironment(listener) doesnt seem to contain HOME variable Modifying protected PollingResult compareRemoteRevisionWith to obtain complete list of envVars Any help or pointers in the right direction would be highly appreciated.

          Carsten Kuhl added a comment -

          Are there any updates on this issue?
          I recently updated to the latest Jenkins version (1.609.2) from an older instance and now all build jobs must be started manually if they are triggered via cvs.
          This is not an optimal situation.

          Carsten Kuhl added a comment - Are there any updates on this issue? I recently updated to the latest Jenkins version (1.609.2) from an older instance and now all build jobs must be started manually if they are triggered via cvs. This is not an optimal situation.

          Carsten Kuhl added a comment -

          So I found a workaround for me. The envVar JENKINS_HOME is used and set in both ways. So I changed the location of the ssh key to $JENKINS_HOME/.ssh/id_dsa and set the envVar $JENKINS_HOME on all nodes.
          This way the CI works with the CVS and the build runs fine. Not perfect but it works

          Carsten Kuhl added a comment - So I found a workaround for me. The envVar JENKINS_HOME is used and set in both ways. So I changed the location of the ssh key to $JENKINS_HOME/.ssh/id_dsa and set the envVar $JENKINS_HOME on all nodes. This way the CI works with the CVS and the build runs fine. Not perfect but it works

          Shane Turner added a comment -

          I updated my Jenkins 1.609.3 LTS installation to use newer versions of several plugins, and now I'm seeing the following in the CVS Polling Log:

          Started on 15-Oct-2015 2:07:00 PM
          Repository not found in workspace state, scheduling build
          Done. Took 2 ms
          Changes found

          The plugins were:
          Credentials 1.23 -> 1.24
          Matrix Project 1.4.1 -> 1.6
          SSH Agent 1.7 -> 1.8

          Shane Turner added a comment - I updated my Jenkins 1.609.3 LTS installation to use newer versions of several plugins, and now I'm seeing the following in the CVS Polling Log: Started on 15-Oct-2015 2:07:00 PM Repository not found in workspace state, scheduling build Done. Took 2 ms Changes found The plugins were: Credentials 1.23 -> 1.24 Matrix Project 1.4.1 -> 1.6 SSH Agent 1.7 -> 1.8

            justchai Chaitanya Pinnamaneni
            sst_lfe_build Sandstone
            Votes:
            3 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: