Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-46721

Expect library sources in custom subdirectory

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Hi.

      The current design of the shared library plugin doesn't play well with monolithic repositories. It forces a specific directory structure on the top level of the repository used for the library, which makes it very hard when a team want to keep all their sources (including the ones used by Jenkins) versioned in a single repository (following the monorepo paradigm).

      Our workflow might be a little strange, because we are a single team that uses a single Jenkins instance/cluster, but I am sure there are others who have the same setup.

      Basically, what we wanted to have is a single multi-branch Pipeline and a single global library, the multi-branch Pipeline delegates work to (because global libraries can use @Grab for example). In such a "simple" scenario, it makes sense to use a single repository for both the Jenkinsfile and the shared library.

      The way we have it set up is that the multi-branch Pipeline executes a simple Jenkinsfile that only determines what branch (or ref) it is running, then uses the `library` step to load the shared library with the same branch (or ref) and calls it, delegating all work to the shared library.

      This system, at least for us, is great, because when we work on, review and test out pull requests, every change can be done as part of a single pull request to a single repository.

      Anyway, I've actually gone ahead and implemented the possibility of specifying a custom directory from which to check out the library's sources. We've been using it for some time and I'd like to see if it could be merged upstream, so we could drop our fork of this great plugin.

      Here's the PR: https://github.com/jenkinsci/workflow-cps-global-lib-plugin/pull/40

        Attachments

          Issue Links

            Activity

            Hide
            petr_motejlek Petr Motejlek added a comment -

            Hi,

            I think it's important to give some closure here. Especially to people coming, wanting to do this.

            I agree with Jesse Glick, that there is too much work being done. Ultimately, people should use Jenkins as a tool that can checkout their sources and trigger their process. Having Jenkins do too many things will become hard to maintain over time and will definitely be brittle. Not to mention that it's very easy to work with external script(s) that do(es) not depend on Jenkins at all, rather than only being to execute them through Jenkins.

            The next time I work on a project that requires extensive programming to orchestrate our infrastructure, I will advocate for using Jenkins as minimally as possible.

            To Jesse Glick's point

            This is a bad idea and you are doing way too much work. Do not use a library at all. Just use the load step. That will also properly handle more subtle cases you have not considered, such as GitHub pull requests filed from forks, or several commits in rapid succession to the same branch.

            it is important to mention that we use this feature when working with global shared libraries, i.e. the ones that do not run sandboxed. Additionally, the `load` step does not work with IDEs. Which is something we like to use as much as possible.

            Anyhoo, I am closing this out.

            Show
            petr_motejlek Petr Motejlek added a comment - Hi, I think it's important to give some closure here. Especially to people coming, wanting to do this. I agree with Jesse Glick , that there is too much work being done. Ultimately, people should use Jenkins as a tool that can checkout their sources and trigger their process. Having Jenkins do too many things will become hard to maintain over time and will definitely be brittle. Not to mention that it's very easy to work with external script(s) that do(es) not depend on Jenkins at all, rather than only being to execute them through Jenkins. The next time I work on a project that requires extensive programming to orchestrate our infrastructure, I will advocate for using Jenkins as minimally as possible. To Jesse Glick 's point This is a bad idea and you are doing way too much work. Do not use a library at all. Just use the load step. That will also properly handle more subtle cases you have not considered, such as GitHub pull requests filed from forks, or several commits in rapid succession to the same branch. it is important to mention that we use this feature when working with global shared libraries, i.e. the ones that do not run sandboxed. Additionally, the `load` step does not work with IDEs. Which is something we like to use as much as possible. Anyhoo, I am closing this out.
            Hide
            jglick Jesse Glick added a comment -

             the multi-branch Pipeline executes a simple Jenkinsfile that only determines what branch (or ref) it is running, then uses the `library` step to load the shared library with the same branch (or ref) and calls it, delegating all work to the shared library.

            This is a bad idea and you are doing way too much work. Do not use a library at all. Just use the load step. That will also properly handle more subtle cases you have not considered, such as GitHub pull requests filed from forks, or several commits in rapid succession to the same branch.

            Show
            jglick Jesse Glick added a comment -  the multi-branch Pipeline executes a simple Jenkinsfile that only determines what branch (or ref) it is running, then uses the `library` step to load the shared library with the same branch (or ref) and calls it, delegating all work to the shared library. This is a bad idea and you are doing way too much work. Do not use a library at all. Just use the load step. That will also properly handle more subtle cases you have not considered, such as GitHub pull requests filed from forks, or several commits in rapid succession to the same branch.
            Hide
            eversc Chris Every added a comment -

            This would be really handy for me, as I want to be able to specify 'quite a few' shared pipelines, some of which are specific to lots of services within a single project, and I have multiple projects

            Being able to specify a dir structure within /vars would really help the management of shared libs

            Show
            eversc Chris Every added a comment - This would be really handy for me, as I want to be able to specify 'quite a few' shared pipelines, some of which are specific to lots of services within a single project, and I have multiple projects Being able to specify a dir structure within /vars would really help the management of shared libs
            Hide
            basil Basil Crow added a comment -

            Nice! My company has the same problem, and I'd really like for this functionality to be available.

            Show
            basil Basil Crow added a comment - Nice! My company has the same problem, and I'd really like for this functionality to be available.

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              petr_motejlek Petr Motejlek
              Votes:
              5 Vote for this issue
              Watchers:
              13 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: