Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-64000

Git tag setting ignored by repository fetch in multibranch project created by GitHub org folder

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Minor
    • Resolution: Unresolved
    • Component/s: git-plugin
    • Labels:
      None
    • Environment:
      Jenkins 2.249.2
      Git plugin 4.4.4
    • Similar Issues:

      Description

      As requested in allowSecondFetch ** documentation I am opening a ticket.

      After upgrading git plugin to version 4.4.4 (can't recall the previous version), my Jenkins job started to fail on the checkout step because it requires the version tags, defined as bellow:

      stage('Checkout') {
        retry(3) {
          scmvars = checkout([
            $class                           : 'GitSCM',
            branches                         : scm.branches,
            doGenerateSubmoduleConfigurations: scm.doGenerateSubmoduleConfigurations,
            extensions                       : scm.extensions +
              [$class: 'CloneOption', noTags: false, reference: '~/git/service-sourcing/', shallow: false] +
              [$class: 'LocalBranch', localBranch: "**"] +
              [$class: 'UserIdentity', email: 'dev@nezasa.com', name: 'Jenkins CI'],
            userRemoteConfigs                : scm.userRemoteConfigs
          ])
        }
      }

       

      Before the update the job used to output this during checkout:
      **

      05:16:32   > git fetch --no-tags --force --progress -- git@github.com:nezasa/tb-service-sourcing.git +refs/heads/develop:refs/remotes/origin/develop +refs/heads/*:refs/remotes/origin/* # timeout=10
      05:16:35  Fetching without tags
      05:16:35   > git config remote.origin.url git@github.com:nezasa/tb-service-sourcing.git # timeout=10
      05:16:35   > git config --add remote.origin.fetch +refs/heads/develop:refs/remotes/origin/develop # timeout=10
      05:16:35   > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
      05:16:35   > git config remote.origin.url git@github.com:nezasa/tb-service-sourcing.git # timeout=10
      05:16:35  Fetching upstream changes from git@github.com:nezasa/tb-service-sourcing.git
      05:16:35  using GIT_SSH to set credentials Github SSH Key for common libraries access
      05:16:35   > git fetch --tags --force --progress -- git@github.com:nezasa/tb-service-sourcing.git +refs/heads/develop:refs/remotes/origin/develop +refs/heads/*:refs/remotes/origin/* # timeout=10

       

      After the update the output is (Git is now Avoiding second fetch):
      **

      05:16:59   > git fetch --no-tags --force --progress -- git@github.com:nezasa/tb-service-sourcing.git +refs/heads/develop:refs/remotes/origin/develop +refs/heads/*:refs/remotes/origin/* # timeout=10
      05:17:01  Avoid second fetch
      05:17:01  Checking out Revision 68628022e4553cacb97377dfd35d54c3d3d3734f (develop)
      05:17:01   > git config remote.origin.url git@github.com:nezasa/tb-service-sourcing.git # timeout=10
      05:17:01   > git config --add remote.origin.fetch +refs/heads/develop:refs/remotes/origin/develop # timeout=10
      05:17:01   > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
      05:17:01   > git config core.sparsecheckout # timeout=10
      05:17:01   > git checkout -f 68628022e4553cacb97377dfd35d54c3d3d3734f # timeout=10
      05:17:01   > git branch -a -v --no-abbrev # timeout=10
      05:17:02   > git checkout -b develop 68628022e4553cacb97377dfd35d54c3d3d3734f # timeout=10

       

      Adding allowSecondFetch flag and setting it to true on the global settings seems to fix the issue but I was expecting that the Advanced Clone Setting on the job would be enough.

        Attachments

          Activity

          Hide
          markewaite Mark Waite added a comment - - edited

          Thanks for the report!

          I believe you are saying that your build started failing with git plugin 4.4.4 because it requires that the checkout must retrieve the git tags from the remote repository.

          Since your pipeline checkout definition declares noTags: false, the first fetch operation should have included the --tags argument. It included the incorrect --no-tags argument instead. When the second fetch was removed, the incorrect --no-tags argument in the first fetch meant that tags would not be retrieved.

          Can you confirm that I have understood your bug report correctly?

          Show
          markewaite Mark Waite added a comment - - edited Thanks for the report! I believe you are saying that your build started failing with git plugin 4.4.4 because it requires that the checkout must retrieve the git tags from the remote repository. Since your pipeline checkout definition declares noTags: false , the first fetch operation should have included the --tags argument. It included the incorrect --no-tags argument instead. When the second fetch was removed, the incorrect --no-tags argument in the first fetch meant that tags would not be retrieved. Can you confirm that I have understood your bug report correctly?
          Hide
          jose_pedro_sa Jose Sa added a comment -

          Yes that is correct Mark Waite, thanks.

          Show
          jose_pedro_sa Jose Sa added a comment - Yes that is correct Mark Waite , thanks.
          Hide
          markewaite Mark Waite added a comment - - edited

          Jose Sa I can't reliably duplicate the problem. I've seen a single failure in the tests I've run. That failure was the 6th run of a multibranch pipeline that used GitHub as the provider. Builds 1-5 passed. Builds 7 and beyond passed. Only build 6 failed and reported that the --no-tags argument was used in the git fetch. I don't know what was different about that build.

          No other multi-branch pipeline failed. The job with the git plugin as provider passed. The job using JGit as the implementation passed. Various permutations of job settings all passed.

          Refer to my job definition that I used in my attempt to duplicate the problem. That attempt automatically ran the job in several different configurations of multi-branch pipeline.

          Please provide additional details that will allow others to duplicate the issue.

          Some of the questions to be answered include:

          • Version of command line git you're using
          • Operating system on the Jenkins controller
          • Operating system on the Jenkins agent
          • Context where the Pipeline is run (is it a Pipeline job with script entered directly in the Jenkins UI, a Pipeline job with script from SCM, a muilti-branch pipeline job, or a job created by an organization folder)
          • Which scm.extensions are enabled in the multi-branch pipeline
          Show
          markewaite Mark Waite added a comment - - edited Jose Sa I can't reliably duplicate the problem. I've seen a single failure in the tests I've run. That failure was the 6th run of a multibranch pipeline that used GitHub as the provider. Builds 1-5 passed. Builds 7 and beyond passed. Only build 6 failed and reported that the --no-tags argument was used in the git fetch. I don't know what was different about that build. No other multi-branch pipeline failed. The job with the git plugin as provider passed. The job using JGit as the implementation passed. Various permutations of job settings all passed. Refer to my job definition that I used in my attempt to duplicate the problem. That attempt automatically ran the job in several different configurations of multi-branch pipeline. Please provide additional details that will allow others to duplicate the issue. Some of the questions to be answered include: Version of command line git you're using Operating system on the Jenkins controller Operating system on the Jenkins agent Context where the Pipeline is run (is it a Pipeline job with script entered directly in the Jenkins UI, a Pipeline job with script from SCM, a muilti-branch pipeline job, or a job created by an organization folder) Which scm.extensions are enabled in the multi-branch pipeline
          Hide
          jose_pedro_sa Jose Sa added a comment - - edited

          We have Jenkinsdeployed into a Kubernetes cluster using helm and the official Jenkins chart.

          • Version of command line git you're using: git version 2.20.1
          • Operating system on the Jenkins controller:  Linux/UNIX (Amazon Linux 2 image)
          • Operating system on the Jenkins agent:  Linux/UNIX (Amazon Linux 2 image) - "docker image jenkins/inbound-agent:4.3-4"
          • Context where the Pipeline is run (is it a Pipeline job with script entered directly in the Jenkins UI, a Pipeline job with script from SCM, a muilti-branch pipeline job, or a job created by an organization folder): github organization folder, configured with multibranch pipeline
          • Which scm.extensions are enabled in the multi-branch pipeline: CloneOption, LocalBranch, UserIdentity

          This might also be useful:

          Show
          jose_pedro_sa Jose Sa added a comment - - edited We have Jenkinsdeployed into a Kubernetes cluster using helm and the official Jenkins chart. Version of command line git you're using: git version   2.20.1 Operating system on the Jenkins controller:   Linux/UNIX (Amazon Linux 2 image) Operating system on the Jenkins agent:   Linux/UNIX   (Amazon Linux 2 image) - "docker image jenkins/inbound-agent:4.3-4" Context where the Pipeline is run (is it a Pipeline job with script entered directly in the Jenkins UI, a Pipeline job with script from SCM, a muilti-branch pipeline job, or a job created by an organization folder): github organization folder, configured with multibranch pipeline Which scm.extensions are enabled in the multi-branch pipeline:  CloneOption, LocalBranch, UserIdentity This might also be useful:
          Hide
          markewaite Mark Waite added a comment - - edited

          Thanks Jose Sa. I have a GitHub organization folder with a multibranch pipeline that is showing the issue consistently.

          Your near term alternatives include:

          • Use a multibranch project created separate from the GitHub organization folder
          • Retain the second fetch globally so that the second fetch provides tags even though the first fetch incorrectly does not provide tags
          • Define the organization folder so that it retrieves tags. That should then be used by the multibranch projects

          It appears that a difference in tag retrieval configuration between the first and second fetch is not being detected by the code that removes the (potentially) redundant fetch.

          Problem is also only visible on the initial creation of a workspace. Later updates of an existing workspace do not have the issue. Since most of us are moving to ephemeral agents, it will be more and more common that the job is always performing initial creation of a workspace.

          I'm also able to consistently show the failure in a multibranch project that uses the GitHub provider and is configured to not fetch tags in the multibranch definition. My two cases that consistently pass are using the git provider and have tag fetching enabled. I suspect the issue is that the setting inside the job definition to fetch tags is being ignored or overridden on the first fetch and not ignored on the second fetch.

          Show
          markewaite Mark Waite added a comment - - edited Thanks Jose Sa . I have a GitHub organization folder with a multibranch pipeline that is showing the issue consistently. Your near term alternatives include: Use a multibranch project created separate from the GitHub organization folder Retain the second fetch globally so that the second fetch provides tags even though the first fetch incorrectly does not provide tags Define the organization folder so that it retrieves tags. That should then be used by the multibranch projects It appears that a difference in tag retrieval configuration between the first and second fetch is not being detected by the code that removes the (potentially) redundant fetch. Problem is also only visible on the initial creation of a workspace. Later updates of an existing workspace do not have the issue. Since most of us are moving to ephemeral agents, it will be more and more common that the job is always performing initial creation of a workspace. I'm also able to consistently show the failure in a multibranch project that uses the GitHub provider and is configured to not fetch tags in the multibranch definition. My two cases that consistently pass are using the git provider and have tag fetching enabled. I suspect the issue is that the setting inside the job definition to fetch tags is being ignored or overridden on the first fetch and not ignored on the second fetch.
          Hide
          jose_pedro_sa Jose Sa added a comment - - edited

          Thank you for the feedback Mark Waite, setting the flag allowSecondFetch to true on global settings solved our problem*.*

          Show
          jose_pedro_sa Jose Sa added a comment - - edited Thank you for the feedback Mark Waite , setting the flag  allowSecondFetch to true on global settings solved our problem*.*

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            jose_pedro_sa Jose Sa
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated: