Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-70012

Perform lightweight checkout on tags

XMLWordPrintable

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Major Major
    • atlassian-bitbucket-server-integration-plugin
    • None

      It seems like Bitbucket branches follow this convention but not Bitbucket tags. I create a Multibranch Pipeline job with the Branch Source set to Bitbucket Server. I have Discover Branches and Discover Tags set as well as the Build Strategies for building all regular branches and all tags regardless of age. 

      If I look in /var/lib/jenkins on the Jenkins Built-in node after a branch job runs I see a workspace_libs/ directory with the shared libraries that job imports cloned down. However, on tag jobs, I see that workspace_libs/ folder as well as a workspace_script/ folder. That workspace_script/ folder has the entire repo cloned down in order to access the Jenkinsfile it seems.

      To cut down on clone size I have changed my job configuration to sparse checkout just the Jenkinsfile but I still get the full .git/ folder under workspace_script/ which for some of our repos can be pretty big (~380 MB - it’s a large C++ project so git-lfs won’t help us here). I can change the clone depth as well but that has the consequences of not populating the “changes” field correctly if the shallow history depth of the clone is smaller than the number of commits since the last job run (not the end of the world there but not ideal). 

      For what it’s worth, our pipelines have their own stage in a node{} block that perform their own clone on the builder node the job chooses so there’s no need for us to use that workspace_scripts/ folder. We cleanup on the builder nodes after the jobs run so that’s not a problem. The Built-in node is where the growth seems to happen that we can’t control as well as I’d hope. 

      I would imagine branches and tags should work the same right? Branches don’t seem to need that workspace_script/ folder at all in order to access and use the Jenkinsfile. Is this a bug in how the tag jobs are processed or is there a reason behind that difference in behavior?

      Thanks! Im on mobile now so can’t provide server details or screenshots but will do so first thing tomorrow morning. 

      Note: workspace_libs/ and workspace_scripts/ actually have a double underscore in their name but formatting issues here preventing me from typing that out. I called them out above with the single underscore but just know that the server shows double as expected.

          [JENKINS-70012] Perform lightweight checkout on tags

          Michael Steppe created issue -
          Michael Steppe made changes -
          Description Original: It seems like Bitbucket branches follow this convention but not Bitbucket tags. I create a Multibranch Pipeline job with the Branch Source set to Bitbucket Server. I have Discover Branches and Discover Tags set as well as the Build Strategies for building all regular branches and all tags regardless of age. 

          If I look in /var/lib/jenkins on the Jenkins Built-in node after a branch job runs I see a “workspace__libs/” directory with the shared libraries that job imports cloned down. However, on tag jobs, I see that “workspace__libs/” folder as well as a “workspace__script/” folder. That workspace__script/ folder has the entire repo cloned down in order to access the Jenkinsfile it seems. 

          To cut down on clone size I have changed my job configuration to sparse checkout just the Jenkinsfile but I still get the full .git/ folder under “workspace__script/“ which for some of our repos can be pretty big (~380 MB - it’s a large C++ project so git-lfs won’t help us here). I can change the clone depth as well but that has the consequences of not populating the “changes” field correctly if the shallow history depth of the clone is smaller than the number of commits since the last job run (not the end of the world there but not ideal). 

          For what it’s worth, our pipelines have their own stage in a node{} block that perform their own clone on the builder node the job chooses so there’s no need for us to use that workspace__scripts/ folder. We cleanup on the builder nodes after the jobs run so that’s not a problem. The Built-in node is where the growth seems to happen that we can’t control as well as I’d hope. 

          I would imagine branches and tags should work the same right? Branches don’t seem to need that “workspace__script/“ folder at all in order to access and use the Jenkinsfile. Is this a bug in how the tag jobs are processed or is there a reason behind that difference in behavior?

          Thanks! Im on mobile now so can’t provide server details or screenshots but will do so first thing tomorrow morning.            		New: It seems like Bitbucket branches follow this convention but not Bitbucket tags. I create a Multibranch Pipeline job with the Branch Source set to Bitbucket Server. I have Discover Branches and Discover Tags set as well as the Build Strategies for building all regular branches and all tags regardless of age. 

          If I look in /var/lib/jenkins on the Jenkins Built-in node after a branch job runs I see a workspace__libs/ directory with the shared libraries that job imports cloned down. However, on tag jobs, I see that workspace__libs/ folder as well as a workspace__script/ folder. That workspace__script/ folder has the entire repo cloned down in order to access the Jenkinsfile it seems.

          To cut down on clone size I have changed my job configuration to sparse checkout just the Jenkinsfile but I still get the full .git/ folder under workspace__script/ which for some of our repos can be pretty big (~380 MB - it’s a large C++ project so git-lfs won’t help us here). I can change the clone depth as well but that has the consequences of not populating the “changes” field correctly if the shallow history depth of the clone is smaller than the number of commits since the last job run (not the end of the world there but not ideal). 

          For what it’s worth, our pipelines have their own stage in a node{} block that perform their own clone on the builder node the job chooses so there’s no need for us to use that workspace__scripts/ folder. We cleanup on the builder nodes after the jobs run so that’s not a problem. The Built-in node is where the growth seems to happen that we can’t control as well as I’d hope. 

          I would imagine branches and tags should work the same right? Branches don’t seem to need that workspace__script/ folder at all in order to access and use the Jenkinsfile. Is this a bug in how the tag jobs are processed or is there a reason behind that difference in behavior?

          Thanks! Im on mobile now so can’t provide server details or screenshots but will do so first thing tomorrow morning. 
          Michael Steppe made changes -
          Description Original: It seems like Bitbucket branches follow this convention but not Bitbucket tags. I create a Multibranch Pipeline job with the Branch Source set to Bitbucket Server. I have Discover Branches and Discover Tags set as well as the Build Strategies for building all regular branches and all tags regardless of age. 

          If I look in /var/lib/jenkins on the Jenkins Built-in node after a branch job runs I see a workspace__libs/ directory with the shared libraries that job imports cloned down. However, on tag jobs, I see that workspace__libs/ folder as well as a workspace__script/ folder. That workspace__script/ folder has the entire repo cloned down in order to access the Jenkinsfile it seems.

          To cut down on clone size I have changed my job configuration to sparse checkout just the Jenkinsfile but I still get the full .git/ folder under workspace__script/ which for some of our repos can be pretty big (~380 MB - it’s a large C++ project so git-lfs won’t help us here). I can change the clone depth as well but that has the consequences of not populating the “changes” field correctly if the shallow history depth of the clone is smaller than the number of commits since the last job run (not the end of the world there but not ideal). 

          For what it’s worth, our pipelines have their own stage in a node{} block that perform their own clone on the builder node the job chooses so there’s no need for us to use that workspace__scripts/ folder. We cleanup on the builder nodes after the jobs run so that’s not a problem. The Built-in node is where the growth seems to happen that we can’t control as well as I’d hope. 

          I would imagine branches and tags should work the same right? Branches don’t seem to need that workspace__script/ folder at all in order to access and use the Jenkinsfile. Is this a bug in how the tag jobs are processed or is there a reason behind that difference in behavior?

          Thanks! Im on mobile now so can’t provide server details or screenshots but will do so first thing tomorrow morning.            		New: It seems like Bitbucket branches follow this convention but not Bitbucket tags. I create a Multibranch Pipeline job with the Branch Source set to Bitbucket Server. I have Discover Branches and Discover Tags set as well as the Build Strategies for building all regular branches and all tags regardless of age. 

          If I look in /var/lib/jenkins on the Jenkins Built-in node after a branch job runs I see a workspace_libs/ directory with the shared libraries that job imports cloned down. However, on tag jobs, I see that workspace_libs/ folder as well as a workspace_script/ folder. That workspace_script/ folder has the entire repo cloned down in order to access the Jenkinsfile it seems.

          To cut down on clone size I have changed my job configuration to sparse checkout just the Jenkinsfile but I still get the full .git/ folder under workspace_script/ which for some of our repos can be pretty big (~380 MB - it’s a large C++ project so git-lfs won’t help us here). I can change the clone depth as well but that has the consequences of not populating the “changes” field correctly if the shallow history depth of the clone is smaller than the number of commits since the last job run (not the end of the world there but not ideal). 

          For what it’s worth, our pipelines have their own stage in a node{} block that perform their own clone on the builder node the job chooses so there’s no need for us to use that workspace_scripts/ folder. We cleanup on the builder nodes after the jobs run so that’s not a problem. The Built-in node is where the growth seems to happen that we can’t control as well as I’d hope. 

          I would imagine branches and tags should work the same right? Branches don’t seem to need that workspace_script/ folder at all in order to access and use the Jenkinsfile. Is this a bug in how the tag jobs are processed or is there a reason behind that difference in behavior?

          Thanks! Im on mobile now so can’t provide server details or screenshots but will do so first thing tomorrow morning. 
          Michael Steppe made changes -
          Description Original: It seems like Bitbucket branches follow this convention but not Bitbucket tags. I create a Multibranch Pipeline job with the Branch Source set to Bitbucket Server. I have Discover Branches and Discover Tags set as well as the Build Strategies for building all regular branches and all tags regardless of age. 

          If I look in /var/lib/jenkins on the Jenkins Built-in node after a branch job runs I see a workspace_libs/ directory with the shared libraries that job imports cloned down. However, on tag jobs, I see that workspace_libs/ folder as well as a workspace_script/ folder. That workspace_script/ folder has the entire repo cloned down in order to access the Jenkinsfile it seems.

          To cut down on clone size I have changed my job configuration to sparse checkout just the Jenkinsfile but I still get the full .git/ folder under workspace_script/ which for some of our repos can be pretty big (~380 MB - it’s a large C++ project so git-lfs won’t help us here). I can change the clone depth as well but that has the consequences of not populating the “changes” field correctly if the shallow history depth of the clone is smaller than the number of commits since the last job run (not the end of the world there but not ideal). 

          For what it’s worth, our pipelines have their own stage in a node{} block that perform their own clone on the builder node the job chooses so there’s no need for us to use that workspace_scripts/ folder. We cleanup on the builder nodes after the jobs run so that’s not a problem. The Built-in node is where the growth seems to happen that we can’t control as well as I’d hope. 

          I would imagine branches and tags should work the same right? Branches don’t seem to need that workspace_script/ folder at all in order to access and use the Jenkinsfile. Is this a bug in how the tag jobs are processed or is there a reason behind that difference in behavior?

          Thanks! Im on mobile now so can’t provide server details or screenshots but will do so first thing tomorrow morning.            		New: It seems like Bitbucket branches follow this convention but not Bitbucket tags. I create a Multibranch Pipeline job with the Branch Source set to Bitbucket Server. I have Discover Branches and Discover Tags set as well as the Build Strategies for building all regular branches and all tags regardless of age. 

          If I look in /var/lib/jenkins on the Jenkins Built-in node after a branch job runs I see a workspace_libs/ directory with the shared libraries that job imports cloned down. However, on tag jobs, I see that workspace_libs/ folder as well as a workspace_script/ folder. That workspace_script/ folder has the entire repo cloned down in order to access the Jenkinsfile it seems.

          To cut down on clone size I have changed my job configuration to sparse checkout just the Jenkinsfile but I still get the full .git/ folder under workspace_script/ which for some of our repos can be pretty big (~380 MB - it’s a large C++ project so git-lfs won’t help us here). I can change the clone depth as well but that has the consequences of not populating the “changes” field correctly if the shallow history depth of the clone is smaller than the number of commits since the last job run (not the end of the world there but not ideal). 

          For what it’s worth, our pipelines have their own stage in a node{} block that perform their own clone on the builder node the job chooses so there’s no need for us to use that workspace_scripts/ folder. We cleanup on the builder nodes after the jobs run so that’s not a problem. The Built-in node is where the growth seems to happen that we can’t control as well as I’d hope. 

          I would imagine branches and tags should work the same right? Branches don’t seem to need that workspace_script/ folder at all in order to access and use the Jenkinsfile. Is this a bug in how the tag jobs are processed or is there a reason behind that difference in behavior?

          Thanks! Im on mobile now so can’t provide server details or screenshots but will do so first thing tomorrow morning. 

          Note: workspace_libs/ and workspace_scripts/ actually have a double underscore in their name but formatting issues here preventing me from typing that out. I called them out above with the single underscore but just know that the server shows double as expected.
          Michael Steppe made changes -
          Environment New: Jenkins: 2.361.2
          OS: Linux - 3.10.0-1160.59.1.el7.x86_64
          ---
          ace-editor:1.1
          active-directory:2.26
          analysis-model-api:10.17.0
          anchore-container-scanner:1.0.25
          ansible:1.1
          ansicolor:1.0.2
          ant:481.v7b_09e538fcca
          antisamy-markup-formatter:2.7
          apache-httpcomponents-client-4-api:4.5.13-138.v4e7d9a_7b_a_e61
          artifactdeployer:1.3
          atlassian-bitbucket-server-integration:3.3.1
          audit-trail:3.11
          authentication-tokens:1.4
          basic-branch-build-strategies:1.3.2
          bitbucket:223.vd12f2bca5430
          bitbucket-build-status-notifier:1.4.2
          bitbucket-oauth:0.12
          bitbucket-push-and-pull-request:2.8.3
          blueocean:1.25.8
          blueocean-autofavorite:1.2.5
          blueocean-bitbucket-pipeline:1.25.8
          blueocean-commons:1.25.8
          blueocean-config:1.25.8
          blueocean-core-js:1.25.8
          blueocean-dashboard:1.25.8
          blueocean-display-url:2.4.1
          blueocean-events:1.25.8
          blueocean-git-pipeline:1.25.8
          blueocean-github-pipeline:1.25.8
          blueocean-i18n:1.25.8
          blueocean-jira:1.25.8
          blueocean-jwt:1.25.8
          blueocean-personalization:1.25.8
          blueocean-pipeline-api-impl:1.25.8
          blueocean-pipeline-editor:1.25.8
          blueocean-pipeline-scm-api:1.25.8
          blueocean-rest:1.25.8
          blueocean-rest-impl:1.25.8
          blueocean-web:1.25.8
          bootstrap4-api:4.6.0-5
          bootstrap5-api:5.2.1-3
          bouncycastle-api:2.26
          branch-api:2.1046.v0ca_37783ecc5
          build-pipeline-plugin:1.5.8
          build-timeout:1.24
          build-timestamp:1.0.3
          build-user-vars-plugin:1.9
          built-on-column:1.1
          caffeine-api:2.9.3-65.v6a_47d0f4d1fe
          checks-api:1.7.5
          clearcase:1.6.8
          cloudbees-bitbucket-branch-source:791.vb_eea_a_476405b
          cloudbees-disk-usage-simple:178.v1a_4d2f6359a_8
          cloudbees-folder:6.758.vfd75d09eea_a_1
          cobertura:1.17
          code-coverage-api:3.2.0
          command-launcher:90.v669d7ccb_7c31
          commons-lang3-api:3.12.0-36.vd97de6465d5b_
          commons-text-api:1.9-19.v8df45c678366
          conditional-buildstep:1.4.2
          config-file-provider:3.11.1
          configuration-as-code:1512.vb_79d418d5fc8
          copyartifact:1.47
          credentials:1189.vf61b_a_5e2f62e
          credentials-binding:523.vd859a_4b_122e6
          cvs:2.19.1
          dashboard-view:2.447.vda_124dd35f11
          data-tables-api:1.12.1-4
          declarative-pipeline-migration-assistant:1.5.2
          declarative-pipeline-migration-assistant-api:1.5.2
          description-setter:1.10
          disk-usage:0.28
          display-url-api:2.3.6
          docker-build-publish:1.4.0
          docker-build-step:2.8
          docker-commons:1.21
          docker-traceability:1.2
          docker-workflow:521.v1a_a_dd2073b_2e
          downstream-build-cache:1.7
          dtkit-api:3.0.2
          durable-task:501.ve5d4fc08b0be
          echarts-api:5.4.0-1
          email-ext:2.92
          emailext-template:1.5
          embeddable-build-status:255.va_d2370ee8fde
          envinject:2.881.v37c62073ff97
          envinject-api:1.199.v3ce31253ed13
          extended-choice-parameter:346.vd87693c5a_86c
          extended-read-permission:3.2
          external-monitor-job:203.v683c09d993b_9
          favorite:2.4.1
          file-operations:1.11
          font-awesome-api:6.2.0-3
          forensics-api:1.16.0
          fortify:22.1.38
          fortify-on-demand-uploader:7.1.1
          fstrigger:1.00
          generic-webhook-trigger:1.85.1
          git:4.12.1
          git-client:3.12.1
          git-forensics:1.11.0
          git-server:99.va_0826a_b_cdfa_d
          github:1.35.0
          github-api:1.303-400.v35c2d8258028
          github-branch-source:1695.v88de84e9f6b_9
          golang:1.4
          gradle:1.40
          handlebars:3.0.8
          handy-uri-templates-2-api:2.1.8-22.v77d5b_75e6953
          htmlpublisher:1.31
          http_request:1.16
          instance-identity:116.vf8f487400980
          ionicons-api:28.va_f3a_84439e5f
          jackson2-api:2.13.4-293.vee957901b_6fb
          jakarta-activation-api:2.0.1-2
          jakarta-mail-api:2.0.1-2
          javadoc:226.v71211feb_e7e9
          javax-activation-api:1.2.0-5
          javax-mail-api:1.6.2-8
          jaxb:2.3.6-2
          jdk-tool:55.v1b_32b_6ca_f9ca
          jenkins-design-language:1.25.8
          jenkins-multijob-plugin:623.v03401733c9a_9
          jersey2-api:2.37-1
          jira:3.8
          jjwt-api:0.11.5-77.v646c772fddb_0
          jnr-posix-api:3.1.15-2
          job-dsl:1.77
          job-restrictions:0.8
          jobConfigHistory:1176.v1b_4290db_41a_5
          jqs-monitoring:1.4
          jquery:1.12.4-1
          jquery-detached:1.2.1
          jquery3-api:3.6.1-2
          jsch:0.1.55.61.va_e9ee26616e7
          junit:1150.v5c2848328b_60
          kubernetes:3724.v0920c1e0ec69
          kubernetes-client-api:5.12.2-193.v26a_6078f65a_9
          kubernetes-credentials:0.9.0
          label-linked-jobs:6.0.1
          ldap:2.12
          lockable-resources:2.18
          log-parser:2.2
          mailer:438.v02c7f0a_12fa_4
          mapdb-api:1.0.9-28.vf251ce40855d
          matlab:2.9.0
          matrix-auth:2.6.8
          matrix-project:785.v06b_7f47b_c631
          mattermost:3.1.1
          maven-plugin:3.19
          mentor-questa-vrm:1.14
          mercurial:1251.va_b_121f184902
          metrics:4.2.10-389.v93143621b_050
          mina-sshd-api-common:2.9.1-44.v476733c11f82
          mina-sshd-api-core:2.9.1-44.v476733c11f82
          momentjs:1.1.1
          monitoring:1.91.0
          msbuild:1.30
          mstest:1.0.0
          mstestrunner:1.5.0
          multiple-scms:0.8
          next-build-number:1.8
          nexus-artifact-uploader:2.13
          nexus-jenkins-plugin:3.16.449.v50228c7ca_222
          nodejs:1.5.1
          nodelabelparameter:1.11.0
          okhttp-api:4.9.3-108.v0feda04578cf
          pam-auth:1.10
          parameterized-trigger:2.45
          parasoft-findings:10.5.3
          pipeline-build-step:2.18
          pipeline-github-lib:38.v445716ea_edda_
          pipeline-githubnotify-step:49.vf37bf92d2bc8
          pipeline-graph-analysis:195.v5812d95a_a_2f9
          pipeline-groovy-lib:612.v84da_9c54906d
          pipeline-input-step:451.vf1a_a_4f405289
          pipeline-milestone-step:101.vd572fef9d926
          pipeline-model-api:2.2114.v2654ca_721309
          pipeline-model-definition:2.2114.v2654ca_721309
          pipeline-model-extensions:2.2114.v2654ca_721309
          pipeline-rest-api:2.26
          pipeline-stage-step:296.v5f6908f017a_5
          pipeline-stage-tags-metadata:2.2114.v2654ca_721309
          pipeline-stage-view:2.26
          pipeline-utility-steps:2.13.0
          plain-credentials:139.ved2b_9cf7587b
          plot:2.1.11
          plugin-util-api:2.18.0
          popper-api:1.16.1-3
          popper2-api:2.11.6-2
          postbuild-task:1.9
          powershell:1.7
          prism-api:1.29.0-1
          publish-over:0.22
          publish-over-ssh:1.24
          pubsub-light:1.17
          read-only-configurations:1.10
          rebuild:1.34
          release:2.14
          resource-disposer:0.20
          robot:3.3.0
          run-condition:1.5
          saml:2.333.vc81e525974a_c
          scm-api:621.vda_a_b_055e58f7
          script-security:1183.v774b_0b_0a_a_451
          snakeyaml-api:1.32-86.ve3f030a_75631
          sonar:2.14
          sse-gateway:1.26
          ssh-agent:295.v9ca_a_1c7cc3a_a_
          ssh-credentials:305.v8f4381501156
          ssh-slaves:1.821.vd834f8a_c390e
          ssh-steps:2.0.39.v831c5e6468b_c
          sshd:3.249.v2dc2ea_416e33
          stashNotifier:1.28
          structs:324.va_f5d6774f3a_d
          subversion:2.16.0
          swarm:3.36
          synopsys-coverity:3.0.1
          text-finder:1.20
          thinBackup:1.10
          thycotic-secret-server:1.0.2
          timestamper:1.20
          token-macro:308.v4f2b_ed62b_b_16
          translation:1.16
          trilead-api:1.67.vc3938a_35172f
          validating-string-parameter:2.8
          variant:59.vf075fe829ccb
          versionnumber:1.10
          vmanager-plugin:3.1.6
          warnings-ng:9.20.1
          windows-slaves:1.8.1
          workflow-aggregator:590.v6a_d052e5a_a_b_5
          workflow-api:1198.v4596ea_5329b_6
          workflow-basic-steps:994.vd57e3ca_46d24
          workflow-cps:2802.v5ea_628154b_c2
          workflow-cps-global-lib:588.v576c103a_ff86
          workflow-cps-global-lib-http:2.15.0
          workflow-durable-task-step:1199.v02b_9244f8064
          workflow-job:1239.v71b_b_a_124a_725
          workflow-multibranch:716.vc692a_e52371b_
          workflow-scm-step:400.v6b_89a_1317c9a_
          workflow-step-api:639.v6eca_cd8c04a_a_
          workflow-support:838.va_3a_087b_4055b
          ws-cleanup:0.43
          xtrigger-api:0.4
          xunit:3.1.1
          xvfb:1.2
          yet-another-build-visualizer:1.16

            Unassigned Unassigned
            j17359 Michael Steppe
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: