Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-60195

Failed to load shared library when pipeline is triggered by hook in different repo

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • None
    • Jenkins: 2.204
      OS: Ubuntu 16.04.1
      Java Version: 1.8.0_222
      Plugins (selected):
      git: 4.0.0
      Pipeline: Shared Groovy Libraries: 2.15

      I am using the Pipeline: Shared Groovy Libraries to load a shared library from a repo named jenkins_configuration.

      In the pipeline I am loading the library with the usual @Library('shared-pipeline-library')
      and then importing some functions using import.

      The pipeline Jenkinsfile is also saved in the same repo with the shared library (jenkins_configuration)

      The pipeline is checking out code from a 2nd repo (named myservice) that contains the actual sources to be built (docker images)

      When I am building manually (clicking on the build button) the library is loaded correctly and its content is available in the pipeline.

      I am creating a hook in the myservice repo so as to trigger the build of the pipeline whenever a change is made in the source code.

      I have selected "Poll SCM" with no schedule inside the Job.

      When the pipeline is triggered by the hook it fails to load the shared library.

      The problem is that Jenkins tries to fetch the library from the jenkins_configuration repo using the git change id sent by Bitbucket that actually refers to the myservice repo.

      commit notification b68a13f8cc0fe283ab7f05daa7a49b12bb53f7b6
      Obtained jobs/build_service_image_pipeline/Jenkinsfile from git ssh://git@git.mycompany.com:7999/vos/jenkins_configuration.git
      Running in Durability level: MAX_SURVIVABILITY
      Loading library shared-pipeline-library@master
      Attempting to resolve master from remote references...
      > git --version # timeout=10
      using GIT_SSH to set credentials Credentials for accessing jenkins_configuration repo
      > git ls-remote -h – ssh://git@git.mycompany.com:7999/vos/jenkins_configuration.git # timeout=10
      Found match: refs/heads/master revision 325f891257bae26c47ecb630d1af2cf5231c786a
      using credential id_rsa_jenkins_configuration
      > git rev-parse --is-inside-work-tree # timeout=10
      Fetching changes from the remote Git repository
      > git config remote.origin.url ssh://git@git.mycompany.com:7999/vos/jenkins_configuration.git # timeout=10
      Fetching upstream changes from ssh://git@git.mycompany.com:7999/vos/jenkins_configuration.git
      > git --version # timeout=10
      using GIT_SSH to set credentials Credentials for accessing jenkins_configuration repo
      > git fetch --no-tags --force --progress – ssh://git@git.mycompany.com:7999/vos/jenkins_configuration.git +refs/heads/:refs/remotes/origin/ # timeout=10
      > git rev-parse b68a13f8cc0fe283ab7f05daa7a49b12bb53f7b6^{commit} # timeout=10
      ERROR: Checkout failed
      hudson.plugins.git.GitException: Command "git rev-parse b68a13f8cc0fe283ab7f05daa7a49b12bb53f7b6^{commit}" returned status code 128:
      stdout: b68a13f8cc0fe283ab7f05daa7a49b12bb53f7b6^{commit}

      stderr: fatal: ambiguous argument 'b68a13f8cc0fe283ab7f05daa7a49b12bb53f7b6^{commit}': unknown revision or path not in the working tree.
      Use '--' to separate paths from revisions, like this:
      'git <command> [<revision>...][<file>...]'

      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2372)
      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2302)
      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2298)
      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1857)
      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1869)
      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.revParse(CliGitAPIImpl.java:994)
      at hudson.plugins.git.GitAPI.revParse(GitAPI.java:316)
      at hudson.plugins.git.RevisionParameterAction.toRevision(RevisionParameterAction.java:98)
      at hudson.plugins.git.GitSCM.determineRevisionToBuild(GitSCM.java:1029)
      at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1146)
      at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:124)
      at org.jenkinsci.plugins.workflow.libs.SCMSourceRetriever.lambda$doRetrieve$1(SCMSourceRetriever.java:154)
      at org.jenkinsci.plugins.workflow.libs.SCMSourceRetriever.retrySCMOperation(SCMSourceRetriever.java:104)
      at org.jenkinsci.plugins.workflow.libs.SCMSourceRetriever.doRetrieve(SCMSourceRetriever.java:153)
      at org.jenkinsci.plugins.workflow.libs.SCMSourceRetriever.retrieve(SCMSourceRetriever.java:93)
      at org.jenkinsci.plugins.workflow.libs.LibraryAdder.retrieve(LibraryAdder.java:157)
      at org.jenkinsci.plugins.workflow.libs.LibraryAdder.add(LibraryAdder.java:138)
      at org.jenkinsci.plugins.workflow.libs.LibraryDecorator$1.call(LibraryDecorator.java:125)
      at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1065)
      at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
      at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
      at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
      at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
      at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
      at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
      at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
      at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:142)
      at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127)
      at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:561)
      at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:522)
      at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:327)
      at hudson.model.ResourceController.execute(ResourceController.java:97)
      at hudson.model.Executor.run(Executor.java:427)
      Retrying after 10 seconds

        1. bitbucket_hook.jpg
          bitbucket_hook.jpg
          749 kB
        2. bucket_hook_extra_options.jpg
          bucket_hook_extra_options.jpg
          743 kB
        3. pipeline_scm_configuration.jpg
          pipeline_scm_configuration.jpg
          531 kB
        4. poll_enabled.png
          poll_enabled.png
          70 kB
        5. shared_library_plugin.jpg
          shared_library_plugin.jpg
          471 kB

          [JENKINS-60195] Failed to load shared library when pipeline is triggered by hook in different repo

          Allan BURDAJEWICZ added a comment - - edited

          I noticed something while troubleshooting an issue with the GitLab plugin push trigger that I think is similar to what is happening here.

          When a RevisionParameterAction is injected to a triggered build, the Git SCMs that track the same repository of the passed RevisionsActionParameter are forced to build the commit that this Action holds. Even the Pipeline libraries GitSCM.

          In other words, if a hook on s shared library repo triggers a build that also loads the library repository with @Library. The Library will be loaded at the revision passed in the RevisionParameterAction instead of the revision/branch configured in the @Library annotation of globally.

          Note: Regardless of whether the Library is included in the changelog or not.

          ****

          For example, the GitLab Plugin PushHookTriggerHandlerImpl like the GitStatus#onNotifyCommit /git/notifyCommit create a RevisionParameterAction to specify the revision to build when scheduling the build. And that is forcing a revision for all SCMs in the pipeline that tracks the same repository...

          Pipeline Library relies on the GitSCMBuilder that properly injects a SpecificRevisionBuildChooser extension for the library branch.

          The RevisionParameterAction, attached to the build when scheduled, seems to take precedence. Down the line, the RevisionParameterAction will be used by the various GitSCMs in the pipeline, including the one from the @Library. The GitSCM checkout will use this to determine the revision to build:

          If a revision is found, the SpecificRevisionBuildChooser will not be used at all to determine the library revision:

          The RevisionParameterAction forces library checkouts...

          Pipeline shared library seems to do the right thing ? It does not really have a way to force a revision. It uses SCMSOurce#build and pass it the revision. Maybe the GitSCMBuilder should force this (with maybe an extension that overrides GitSCMExtensions#decorateRevisionToBuild). But would that impact other scenarios.... Not sure.
          In any case, I don't think there is any scenario where you would like the Library SCM to be controller by a build action. Git it looks like GitSCM allows that.

          cc jglick markwaite Does that makes sense to you as a Git issue rather than a Git plugin issue or something else ?

          Allan BURDAJEWICZ added a comment - - edited I noticed something while troubleshooting an issue with the GitLab plugin push trigger that I think is similar to what is happening here. When a RevisionParameterAction is injected to a triggered build, the Git SCMs that track the same repository of the passed RevisionsActionParameter are forced to build the commit that this Action holds. Even the Pipeline libraries GitSCM. In other words, if a hook on s shared library repo triggers a build that also loads the library repository with @Library. The Library will be loaded at the revision passed in the RevisionParameterAction instead of the revision/branch configured in the @Library annotation of globally. Note: Regardless of whether the Library is included in the changelog or not. **** For example, the GitLab Plugin PushHookTriggerHandlerImpl like the GitStatus#onNotifyCommit /git/notifyCommit create a RevisionParameterAction to specify the revision to build when scheduling the build. And that is forcing a revision for all SCMs in the pipeline that tracks the same repository... Pipeline Library relies on the GitSCMBuilder that properly injects a SpecificRevisionBuildChooser extension for the library branch. The RevisionParameterAction , attached to the build when scheduled, seems to take precedence. Down the line, the RevisionParameterAction will be used by the various GitSCMs in the pipeline, including the one from the @Library . The GitSCM checkout will use this to determine the revision to build: https://github.com/jenkinsci/git-plugin/blob/git-4.12.1/src/main/java/hudson/plugins/git/GitSCM.java#L1137-L1150 If a revision is found, the SpecificRevisionBuildChooser will not be used at all to determine the library revision: https://github.com/jenkinsci/git-plugin/blob/git-4.12.1/src/main/java/hudson/plugins/git/GitSCM.java#L1152-L1157 The RevisionParameterAction forces library checkouts... Pipeline shared library seems to do the right thing ? It does not really have a way to force a revision. It uses SCMSOurce#build and pass it the revision. Maybe the GitSCMBuilder should force this (with maybe an extension that overrides GitSCMExtensions#decorateRevisionToBuild ). But would that impact other scenarios.... Not sure. In any case, I don't think there is any scenario where you would like the Library SCM to be controller by a build action. Git it looks like GitSCM allows that. cc jglick markwaite Does that makes sense to you as a Git issue rather than a Git plugin issue or something else ?

          A simple question I have also is: Should we still consider the RevisionParameterAction when we have a SpecificRevisionBuildChooser ?

          It seems to me that no. The build chooser is used by GitSCMSources / GitSCMBuilders mainly. That may have narrow refspecs and traits and are very specific in what to fetch and build..

          Allan BURDAJEWICZ added a comment - A simple question I have also is: Should we still consider the RevisionParameterAction when we have a SpecificRevisionBuildChooser ? It seems to me that no. The build chooser is used by GitSCMSources / GitSCMBuilders mainly. That may have narrow refspecs and traits and are very specific in what to fetch and build..

          Jesse Glick added a comment -

          I am not very familiar with RevisionParameterAction or its usage. From a quick read this sounds like a git plugin bug indeed.

          Jesse Glick added a comment - I am not very familiar with RevisionParameterAction or its usage. From a quick read this sounds like a git plugin bug indeed.

            Unassigned Unassigned
            huskarl Ilias huskarl
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: