Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-59034

Shared library loading with credentials failed when a tag is used

XMLWordPrintable

      Loading a shared library at a tag does not use the provided credentials:

      lib = library(
    identifier: "mylib@tag",
    retriever: modernSCM(
        [$class: 'GitSCMSource',
        remote: "GITLAB_URL/mylib.git",
        credentialsId: 'MY_SSH_KEY']
    )
)

      When using a branch name as version (identifier: "mylib@branch_name"), the library is well loaded:

      Attempting to resolve branch_name from remote references...
 > git --version # timeout=10
using GIT_SSH to set credentials Cle SSH pour la connexion au gitlab
 > git ls-remote git@gitlab/mylib.git # timeout=10
Found match: refs/heads/branch_name revision 1dd58015a5a5b9cb4a44404871b81cce6e5e18bb
using credential MY_SSH_KEY
Cloning the remote Git repository
Cloning with configured refspecs honoured and without tags
Cloning repository git@gitlab/mylib.git
 > git init /jenkins/workspace@libs/mylib # timeout=10
Fetching upstream changes from git@gitlab/mylib.git
 > git --version # timeout=10
using GIT_SSH to set credentials Cle SSH pour la connexion au gitlab
 > git fetch --no-tags --progress git@gitlab/mylib.git +refs/heads/*:refs/remotes/origin/*
 > git config remote.origin.url git@gitlab/mylib.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url git@gitlab/mylib.git # timeout=10
Fetching without tags
Fetching upstream changes from git@gitlab/mylib.git
using GIT_SSH to set credentials Cle SSH pour la connexion au gitlab
 > git fetch --no-tags --progress git@gitlab/mylib.git +refs/heads/*:refs/remotes/origin/*
Checking out Revision 1dd58015a5a5b9cb4a44404871b81cce6e5e18bb (branch_name)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 1dd58015a5a5b9cb4a44404871b81cce6e5e18bb
Commit message: "message"
 > git rev-list --no-walk 1dd58015a5a5b9cb4a44404871b81cce6e5e18bb # timeout=10

      But when using a tag, a permission denied occured:

      Loading library mylib@tag
Attempting to resolve tag from remote references...
 > git --version # timeout=10
using GIT_SSH to set credentials Cle SSH pour la connexion au gitlab
 > git ls-remote git@gitlab/mylib.git.git # timeout=10
Found match: refs/tags/tag revision 1dd58015a5a5b9cb4a44404871b81cce6e5e18bb
Resolving tag commit... (remote references may be a lightweight tag or an annotated tag)
 > git rev-parse --is-inside-work-tree # timeout=10
Setting origin to git@gitlab/mylib.git.git
 > git config remote.origin.url git@gitlab/mylib.git.git # timeout=10
Fetching origin...
Fetching upstream changes from origin
 > git --version # timeout=10
 > git config --get remote.origin.url # timeout=10
 > git fetch --tags --progress origin +refs/heads/*:refs/remotes/origin/*
ERROR: Checkout failed
hudson.plugins.git.GitException: Command "git fetch --tags --progress origin +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout: 
stderr: Permission denied, please try again.
Permission denied, please try again.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
fatal: Could not read from remote repository.

      The credentials MY_SSH_KEY seems to not be used in this case.

        1. jenkins-59034-config.xml
          2 kB

          [JENKINS-59034] Shared library loading with credentials failed when a tag is used

          Mark Waite added a comment - - edited

          I can't duplicate the problem you're describing. I've used the `@Liibrary` syntax to load a library by tag at Jenkinsfile compile time and I've used the `lib = library(...)` syntax to load the library by tag at runtime. Both cases work to load the library by tag. I've used that syntax with modern SCM through git (as you did in your example), modern SCM through GitHub, and legacy SCM through git.

          However, I suspect that the issue may be already described in the log file. The log file says:

          using GIT_SSH to set credentials Cle SSH pour la connexion au gitlab
          > git fetch --no-tags --progress git@gitlab/mylib.git +refs/heads/:refs/remotes/origin/
          > git config remote.origin.url git@gitlab/mylib.git # timeout=10
          > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10
          > git config remote.origin.url git@gitlab/mylib.git # timeout=10
          Fetching without tags

          If it is cloning the shared library repository without tags, you may need to adjust the library loading to include tags in the library.

          Have you tried a syntax like:

          // Dynamically loads library - cannot use import to reference classes
lib = library(
    identifier: 'globalPipelineLibraryMarkEWaiteSSH@v1.2',
    retriever: modernSCM(
        [$class: 'GitSCMSource',
        remote: 'git@github.com:MarkEWaite/jenkins-pipeline-utils-private.git',
        credentialsId: 'MarkEWaite-github-rsa-private-key',
        traits: [gitBranchDiscovery(), gitTagDiscovery()]]
    )
)

          That syntax came from the "Pipeline Syntax" helper.

          Can you provide a job definition that will allow me to duplicate the problem?

          Mark Waite added a comment - - edited I can't duplicate the problem you're describing. I've used the `@Liibrary` syntax to load a library by tag at Jenkinsfile compile time and I've used the `lib = library(...)` syntax to load the library by tag at runtime. Both cases work to load the library by tag. I've used that syntax with modern SCM through git (as you did in your example), modern SCM through GitHub, and legacy SCM through git. However, I suspect that the issue may be already described in the log file. The log file says: using GIT_SSH to set credentials Cle SSH pour la connexion au gitlab > git fetch --no-tags --progress git@gitlab/mylib.git +refs/heads/ :refs/remotes/origin/ > git config remote.origin.url git@gitlab/mylib.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/ :refs/remotes/origin/ # timeout=10 > git config remote.origin.url git@gitlab/mylib.git # timeout=10 Fetching without tags If it is cloning the shared library repository without tags, you may need to adjust the library loading to include tags in the library. Have you tried a syntax like: // Dynamically loads library - cannot use import to reference classes lib = library( identifier: 'globalPipelineLibraryMarkEWaiteSSH@v1.2', retriever: modernSCM( [$class: 'GitSCMSource', remote: 'git@github.com:MarkEWaite/jenkins-pipeline-utils-private.git', credentialsId: 'MarkEWaite-github-rsa-private-key', traits: [gitBranchDiscovery(), gitTagDiscovery()]] ) ) That syntax came from the "Pipeline Syntax" helper. Can you provide a job definition that will allow me to duplicate the problem?

          L V added a comment - - edited

          Hi Mark,

          thanks for your answer.

          I attached a file with my job definition.

          I've also done a test using "traits" (same job definition but with traits: [gitBranchDiscovery(), gitTagDiscovery()]added), but it failed with the exception:

          java.lang.NoSuchMethodError: No such DSL method 'gitBranchDiscovery' found
...
	at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:200)
	at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
	at sun.reflect.GeneratedMethodAccessor129.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:158)
	at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:157)
	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:142)
	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:156)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:160)
	at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
	at WorkflowScript.run(WorkflowScript:8)
	at ___cps.transform___(Native Method)
	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:84)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixName(FunctionCallBlock.java:78)
	at sun.reflect.GeneratedMethodAccessor184.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
	at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
	at com.cloudbees.groovy.cps.Next.step(Next.java:83)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
	at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
	at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:186)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:370)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:93)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:282)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:270)
	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:66)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

           btw, the job runs well using legacySCM:

          import jenkins.plugins.git.*
lib = library(
    identifier: 'mylib@tag',
    retriever: legacySCM(
        [$class: 'GitSCM',
        userRemoteConfigs: [[url: "GITLAB_URL/mylib.git", credentialsId: 'MY_SSH_KEY']]
        ]
    )
)

          NoSuchMethodError fixed using the syntax provided by the pipeline syntax helper:

          library(
    changelog: false,
    identifier: 'mylib@tag',
    retriever: modernSCM(
        [$class: 'GitSCMSource',
        credentialsId: 'MY_SSH_KEY',
        remote: 'GITLAB_URL/mylib.git',
        traits: [[$class: 'BranchDiscoveryTrait'], [$class: 'TagDiscoveryTrait']]]
    )
)

          but come back to the initial problem.
          The credential is used for first git commands (git ls-remote is ok), but "git fetch --tags --progress origin +refs/heads/:refs/remotes/origin/" still fails with permission denied.

          L V added a comment - - edited Hi Mark, thanks for your answer. I attached a file with my job definition. I've also done a test using "traits" (same job definition but with traits: [gitBranchDiscovery(), gitTagDiscovery()] added), but it failed with the exception: java.lang.NoSuchMethodError: No such DSL method 'gitBranchDiscovery' found ... at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:200) at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122) at sun.reflect.GeneratedMethodAccessor129.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:158) at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:157) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:142) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:156) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:160) at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17) at WorkflowScript.run(WorkflowScript:8) at ___cps.transform___(Native Method) at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:84) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixName(FunctionCallBlock.java:78) at sun.reflect.GeneratedMethodAccessor184.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21) at com.cloudbees.groovy.cps.Next.step(Next.java:83) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163) at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129) at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:186) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:370) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:93) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:282) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:270) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:66) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang. Thread .run( Thread .java:748)  btw, the job runs well using legacySCM: import jenkins.plugins.git.* lib = library( identifier: 'mylib@tag' , retriever: legacySCM( [$class: 'GitSCM' , userRemoteConfigs: [[url: "GITLAB_URL/mylib.git" , credentialsId: 'MY_SSH_KEY' ]] ] ) ) NoSuchMethodError fixed using the syntax provided by the pipeline syntax helper: library( changelog: false , identifier: 'mylib@tag' , retriever: modernSCM( [$class: 'GitSCMSource' , credentialsId: 'MY_SSH_KEY' , remote: 'GITLAB_URL/mylib.git' , traits: [[$class: 'BranchDiscoveryTrait' ], [$class: 'TagDiscoveryTrait' ]]] ) ) but come back to the initial problem. The credential is used for first git commands (git ls-remote is ok), but "git fetch --tags --progress origin +refs/heads/ :refs/remotes/origin/ " still fails with permission denied.

          Mark Waite added a comment - - edited

          If the config file content matches your current system, then your Pipeline plugins are outdated. The config file lists that it was saved with the workflow-job plugin 2.17, script security plugin 1.60, and workflow-cps 2.71.

          Plugin Config file version Current version
          script-security 1.60 - May 2019 1.62
          workflow-cps 2.71 - Jul 2019 2.73
          workflow-job 2.17 - Jan 2018 2.34

          Since legacy SCM runs as desired, you can either switch to legacy SCM or you can upgrade the plugins and try again.

          lviolon can you confirm the releases of the pipeline plugins that you are using?

          Mark Waite added a comment - - edited If the config file content matches your current system, then your Pipeline plugins are outdated. The config file lists that it was saved with the workflow-job plugin 2.17, script security plugin 1.60, and workflow-cps 2.71. Plugin Config file version Current version script-security 1.60 - May 2019 1.62 workflow-cps 2.71 - Jul 2019 2.73 workflow-job 2.17 - Jan 2018 2.34 Since legacy SCM runs as desired, you can either switch to legacy SCM or you can upgrade the plugins and try again. lviolon can you confirm the releases of the pipeline plugins that you are using?

          L V added a comment -

          Pipeline plugins releases:

          Pipeline Aggregator                         -> 1.4.1
Pipeline Graph Analysis Plugin              -> 1.1
Pipeline Utility Steps                      -> 2.3.0
Pipeline: API                               -> 2.36
Pipeline: Basic Steps                       -> 2.3
Pipeline: Build Step                        -> 2.4
Pipeline: Declarative Agent API             -> 1.1.1
Pipeline: Declarative Extension Points API  -> 1.1.8
Pipeline: Groovy                            -> 2.73
Pipeline: Input Step                        -> 2.5
Pipeline: Job                               -> 2.34
Pipeline: Milestone Step                    -> 1.3
Pipeline: Model API                         -> 1.1.8
Pipeline: Multibranch                       -> 2.17
Pipeline: REST API Plugin                   -> 2.4
Pipeline: SCM Step                          -> 2.7
Pipeline: Shared Groovy Libraries           -> 2.14
Pipeline: Stage Step                        -> 2.3
Pipeline: Stage Tags Metadata               -> 1.1.8
Pipeline: Stage View Plugin                 -> 2.4
Pipeline: Step API                          -> 2.20		
Pipeline: Supporting APIs                   -> 3.3

          Upgrade of the 3 plugins as suggested does not solve the problem.

          L V added a comment - Pipeline plugins releases: Pipeline Aggregator -> 1.4.1 Pipeline Graph Analysis Plugin -> 1.1 Pipeline Utility Steps -> 2.3.0 Pipeline: API -> 2.36 Pipeline: Basic Steps -> 2.3 Pipeline: Build Step -> 2.4 Pipeline: Declarative Agent API -> 1.1.1 Pipeline: Declarative Extension Points API -> 1.1.8 Pipeline: Groovy -> 2.73 Pipeline: Input Step -> 2.5 Pipeline: Job -> 2.34 Pipeline: Milestone Step -> 1.3 Pipeline: Model API -> 1.1.8 Pipeline: Multibranch -> 2.17 Pipeline: REST API Plugin -> 2.4 Pipeline: SCM Step -> 2.7 Pipeline: Shared Groovy Libraries -> 2.14 Pipeline: Stage Step -> 2.3 Pipeline: Stage Tags Metadata -> 1.1.8 Pipeline: Stage View Plugin -> 2.4 Pipeline: Step API -> 2.20 Pipeline: Supporting APIs -> 3.3 Upgrade of the 3 plugins as suggested does not solve the problem.

          Mark Waite added a comment -

          Did you restart Jenkins after you updated the plugins?

          Can you duplicate the same problem on a Jenkins instance running the current plugin versions? For example, the "SCM Step" pipeline plugin version that you are using is 2.7 from Oct 2018. Two versions have been released since then. I assume similar version gaps for other plugins in your list.

          Since I can't duplicate the problem with current plugin versions and you're not running current plugin versions, the first step seems to be to upgrade to current plugin versions.

          Mark Waite added a comment - Did you restart Jenkins after you updated the plugins? Can you duplicate the same problem on a Jenkins instance running the current plugin versions? For example, the "SCM Step" pipeline plugin version that you are using is 2.7 from Oct 2018. Two versions have been released since then. I assume similar version gaps for other plugins in your list. Since I can't duplicate the problem with current plugin versions and you're not running current plugin versions, the first step seems to be to upgrade to current plugin versions.

          L V added a comment -

          Yes, I restarted Jenkins after upgrade.
          I will try to upgrade all plugins as soon as possible.

          L V added a comment - Yes, I restarted Jenkins after upgrade. I will try to upgrade all plugins as soon as possible.

            Unassigned Unassigned
            lviolon L V
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: