-
Bug
-
Resolution: Unresolved
-
Major
-
None
When we have thousands of pipelines jobs using same version of library and also using git tags to load library than there is no meaning to check for update in git remote repo on every pipeline run, also there is no meaning to store separate library in every pipeline jobs workspace and also under every build directory. Calls to git remote repo on every pipeline run add extra loads on master when you have always large number of builds running all time. And extra storage on master makes maintenance(backup, restore in critical time) of jenkins master difficult and more time consuming.
- depends on
-
JENKINS-38992 Cache libraries specified by a permanent revision string
-
- Resolved
-
- relates to
-
JENKINS-65612 Support for immediately cleaning @libs workspaces
-
- Open
-
-
-
- Resolved
-
-
-
- Closed
-
[JENKINS-55341] Unnecessary git fetch calls to remote repository and storing separate library in every pipeline workspace and in every build dir when using git tags
Partial duplicate of JENKINS-38992 (this focuses additionally on disk use).
Basil Crow
added a comment - The problem with regard to disk utilization from my perspective isn't so much that each Pipeline job clones the Git repository in a ${JOB_NAME}@libs workspace but rather that the workspace is never cleaned up. If that job is then deleted, the workspace remains, along with the Git checkout. For large Git repositories this can leave large amounts of tiny files strewn all over the place, which are a pain for system administrators.
To me the fix is relatively straightforward: allow that workspace to be freed as soon as we're done with it rather than keeping it around. I filed JENKINS-65612 to cover that and opened jenkinsci/workflow-cps-global-lib-plugin#110 with a fix.
Basil Crow
added a comment - The problem with regard to disk utilization from my perspective isn't so much that each Pipeline job clones the Git repository in a ${JOB_NAME}@libs workspace but rather that the workspace is never cleaned up. If that job is then deleted, the workspace remains, along with the Git checkout. For large Git repositories this can leave large amounts of tiny files strewn all over the place, which are a pain for system administrators.
To me the fix is relatively straightforward: allow that workspace to be freed as soon as we're done with it rather than keeping it around. I filed JENKINS-65612 to cover that and opened jenkinsci/workflow-cps-global-lib-plugin#110 with a fix. Basil Crow
added a comment - To me the fix is relatively straightforward: allow that workspace to be freed as soon as we're done with it rather than keeping it around. I filed JENKINS-65612 to cover that and opened jenkinsci/workflow-cps-global-lib-plugin#110 with a fix.
Or alternatively, delete the leaked @libs directory whenever the job is deleted or its location changes, as proposed in JENKINS-66629 and implemented in jenkinsci/workflow-cps-global-lib-plugin#115.
Basil Crow
added a comment - To me the fix is relatively straightforward: allow that workspace to be freed as soon as we're done with it rather than keeping it around. I filed JENKINS-65612 to cover that and opened jenkinsci/workflow-cps-global-lib-plugin#110 with a fix.
Or alternatively, delete the leaked @libs directory whenever the job is deleted or its location changes, as proposed in JENKINS-66629 and implemented in jenkinsci/workflow-cps-global-lib-plugin#115 . 
PR raised for same: https://github.com/jenkinsci/workflow-cps-global-lib-plugin/pull/58