Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-35415

Multiple branch projects per repository with different recognizers

    XMLWordPrintable

Details

    • New Feature
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • branch-api-plugin
    • None
    • Jenkins 2.3

    Description

      As a developer/QA... It could be necessary to have multiple build methods/options on a project. And it could be possible if we could have 2 or more multibranch project on the same git project, but with different jenkinsfile file.

      Attachments

        Issue Links

          Activity

            gloparm Gabriel Lopez added a comment - - edited

            Hi jglick

            I understand that cloudbees doesn't have a reason to do it, but are there any plans on releasing that functionality of CloudBees Pipeline: Template to non-enterprise users? 

            I'm referring to custom code as pipeline.

            gloparm Gabriel Lopez added a comment - - edited Hi jglick .  I understand that cloudbees doesn't have a reason to do it, but are there any plans on releasing that functionality of  CloudBees Pipeline: Template  to non-enterprise users?  I'm referring to custom code as pipeline .
            jglick Jesse Glick added a comment -

            I cannot comment except to say that I am unaware of any public commitment to do so.

            jglick Jesse Glick added a comment - I cannot comment except to say that I am unaware of any public commitment to do so.
            hrmpw Patrick Wolf added a comment -

            gloparm We have no plans to open source this functionality at this time. This feature is typically needed by larger organizations or enterprises with strict governance controls to enforce a single Pipeline.

            Using Shared Libraries in Pipeline allows users to create a common pipeline that can be reused across all repositories. Depending on the type of Pipeline you want to create the Jenkinsfile can be a single line or a simple conditional:

             

            if (BRANCH_NAME == 'master') {
              runDeploy()
            } else {
              runSimpleBuild()
            }

            The entire Pipeline for a deployment or a simple build is then defined in the Shared Library and the end user doesn't have to write anything.

             

            cc: cleclerc

            hrmpw Patrick Wolf added a comment - gloparm We have no plans to open source this functionality at this time. This feature is typically needed by larger organizations or enterprises with strict governance controls to enforce a single Pipeline. Using Shared Libraries in Pipeline allows users to create a common pipeline that can be reused across all repositories. Depending on the type of Pipeline you want to create the Jenkinsfile can be a single line or a simple conditional:   if (BRANCH_NAME == 'master' ) { runDeploy() } else { runSimpleBuild() } The entire Pipeline for a deployment or a simple build is then defined in the Shared Library and the end user doesn't have to write anything.   cc: cleclerc
            gloparm Gabriel Lopez added a comment -

            Thanks!. We are running into a similar use case and we are looking at possibilities to solve it. The aforementioned plugin was one, but maybe Shared libraries will do the trick.

            Thanks again for the info!.

            gloparm Gabriel Lopez added a comment - Thanks!. We are running into a similar use case and we are looking at possibilities to solve it. The aforementioned plugin was one, but maybe Shared libraries will do the trick. Thanks again for the info!.
            timblaktu Tim Black added a comment - - edited

            Just noticed [the issue I commented on](https://issues.jenkins.io/browse/JENKINS-43749?focusedCommentId=405518&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-405518) is marked a duplicate of this one, so cc'ing here:

            As noted above there are actually LOTS of reasons why someone would want multiple Jenkins sub-projects nested inside a repo. For example, this issue creates problems for folks that try to avoid submodules and tend to favor monorepos. In one such case, we have a "configuration-management" repo that we use to contain all of our configuration management tools and projects, e.g. packer, ansible, vagrant, etc.. Workflows are A LOT simpler having a monorepo for this, but when it comes to standing these processes up in Jenkins CI, we crash into this core problem.

            To support multiple Jenkinsfiles/multibranch projects in the same repo, we currently use jobdsl to explicitly and statically create a multibranch project for each known project location/subfolder. E.g. in this repo tree, we have 2 packer multibranch pipeline projects and 2 ansible multibranch pipeline projects:

            configuration-management
              - packer
                - subproject1
                  - Jenkinsfile
                  - src
                - subproject2
                  - Jenkinsfile
                  - src 
              - ansible
                - subproject1
                  - Jenkinsfile
                  - src
                - subproject2
                  - Jenkinsfile
                  - src 
            

            We use Folders to mirror the repo structure in Jenkins. The jobdsl to create the items in Jenkins looks like:

            folder("Packer") {
            // TODO: dynamically-create this list by searching repo on all branches
            //       for the superset of `packer/<packer_project_name>` subfolders that
            //       contain a Jenkinsfile: https://issues.jenkins.io/browse/JENKINS-43749
            packer_projects = ['subproject1', 'subproject2']
            for (packer_project in packer_projects)
            {
                multibranchPipelineJob("Packer/${packer_project}") {
                    .
                    .
                    factory {
                        workflowBranchProjectFactory {
                            scriptPath("packer/${packer_project}/Jenkinsfile")
                        }
                    }
                }
            }

            and

            folder("Ansible") {
            // TODO: dynamically-create this list by searching repo on all branches
            //       for the superset of `ansible/<ansible_project_name>` subfolders that
            //       contain a Jenkinsfile: https://issues.jenkins.io/browse/JENKINS-43749
            ansible_projects = ['subproject1', 'subproject2']
            for (ansible_project in ansible_projects)
            {
                multibranchPipelineJob("Ansible/${ansible_project}") {
                    .
                    .
                    factory {
                        workflowBranchProjectFactory {
                            scriptPath("ansible/${ansible_project}/Jenkinsfile")
                        }
                    }
                 }
            }
             

            This approach works well, except that every time we want to add a new sub-project (ansible or packer subfolder with Jenkinsfile, in this case), we have to modify and push the jobdsl. It'd be great if workflow-multibranch could recursively scan for Jenkinsfiles and basically do what the jobdsl above is doing dynamically for me, as noted by my comments.

            I was planning on making this dynamic by modifying my jobdsl to execute a script to perform this Jenkinsfile scanning for me. I assume this is possible - but it's not a complete solution since jobdsl is  only updated during Jenkins infrastructure provisioning - infrequently - so this wouldn't detect most cases where someone adds, commits and pushes a subproject with a Jenkinsfile. We really need this scanning to be done by workflow-multibranch.

            timblaktu Tim Black added a comment - - edited Just noticed [the issue I commented on] ( https://issues.jenkins.io/browse/JENKINS-43749?focusedCommentId=405518&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-405518 ) is marked a duplicate of this one, so cc'ing here: As noted above there are actually LOTS of reasons why someone would want multiple Jenkins sub-projects nested inside a repo. For example, this issue creates problems for folks that try to avoid submodules and tend to favor monorepos. In one such case, we have a "configuration-management" repo that we use to contain all of our configuration management tools and projects, e.g. packer, ansible, vagrant, etc.. Workflows are A LOT simpler having a monorepo for this, but when it comes to standing these processes up in Jenkins CI, we crash into this core problem. To support multiple Jenkinsfiles/multibranch projects in the same repo, we currently use jobdsl to explicitly and  statically  create a multibranch project for each  known  project location/subfolder. E.g. in this repo tree, we have 2  packer  multibranch pipeline projects and 2  ansible  multibranch pipeline projects: configuration-management - packer - subproject1 - Jenkinsfile - src - subproject2 - Jenkinsfile - src - ansible - subproject1 - Jenkinsfile - src - subproject2 - Jenkinsfile - src We use Folders to mirror the repo structure in Jenkins. The jobdsl to create the items in Jenkins looks like: folder( "Packer" ) { // TODO: dynamically-create this list by searching repo on all branches // for the superset of `packer/<packer_project_name>` subfolders that // contain a Jenkinsfile: https://issues.jenkins.io/browse/JENKINS-43749 packer_projects = [ 'subproject1' , 'subproject2' ] for (packer_project in packer_projects) { multibranchPipelineJob( "Packer/${packer_project}" ) { . . factory { workflowBranchProjectFactory { scriptPath( "packer/${packer_project}/Jenkinsfile" ) } } } } and folder( "Ansible" ) { // TODO: dynamically-create this list by searching repo on all branches // for the superset of `ansible/<ansible_project_name>` subfolders that // contain a Jenkinsfile: https://issues.jenkins.io/browse/JENKINS-43749 ansible_projects = [ 'subproject1' , 'subproject2' ] for (ansible_project in ansible_projects) { multibranchPipelineJob( "Ansible/${ansible_project}" ) { . . factory { workflowBranchProjectFactory { scriptPath( "ansible/${ansible_project}/Jenkinsfile" ) } } } }   This approach works well, except that  every time we want to add a new sub-project (ansible or packer subfolder with Jenkinsfile, in this case), we have to modify and push the jobdsl.   It'd be great if workflow-multibranch could recursively scan for Jenkinsfiles and basically do what the jobdsl above is doing dynamically for me , as noted by my comments. I was planning on making this dynamic by modifying my jobdsl to execute a script to perform this Jenkinsfile scanning for me. I assume this is possible - but it's not a complete solution since jobdsl is  only updated during Jenkins infrastructure provisioning - infrequently - so this wouldn't detect most cases where someone adds, commits and pushes a subproject with a Jenkinsfile. We really need this scanning to be done by workflow-multibranch.

            People

              Unassigned Unassigned
              elliotmoso Elliot Morales
              Votes:
              30 Vote for this issue
              Watchers:
              35 Start watching this issue

              Dates

                Created:
                Updated: