Details
-
New Feature
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Won't Fix
Description
I have two repos namely - commonlib.git, project1.git, and both of the have the root directories Source and CM. My Jenkins job is MultiGit_Job. I tried to configure a Jenkins job with two repos. During build, the source code for the first project is extracted (found when monitoring during the build execution) and the job fails during the second repo cloning. When I have analyzed the job's workspace, the commonlib.git source is extracted directly under workspace. There is identifier for commonlib repo. Because project1 repo also same root structure, the cloning is failing. It is a critical item for us as we are migrating our projects to GIT now. Thanks
Attachments
Activity
Ok - But I wonder if GIT plugin does not support checkout of distinct repositories under single workspace, why would it have the option to add multiple repositories in the configuration?
I even tried Multi-SCM plugin - which is also not differentiating the different repo code in workspace. Everything appears directly cluttering under the JOB's workspace
Ok - But I wonder if GIT plugin does not support checkout of distinct repositories under single workspace, why would it have the option to add multiple repositories in the configuration?
I even tried Multi-SCM plugin - which is also not differentiating the different repo code in workspace. Everything appears directly cluttering under the JOB's workspace
The git plugin allows multiple copies of the same repository and will poll for changes within those repositories. It assumes that the repositories are the same basic repository, potentially with newer commits in some of the repositories rather than others.
For example, I have a job which includes a reference to the jenkinsci/git-plugin.git repository and also includes a reference to MarkEWaite/git-plugin.git. That allows me to detect changes pushed by others, and changes which I push to my own repository.
Not sure if I understood correctly. I understand it as - Clone of same repository could be available to multiple remote users and the plugin can poll for changes of any number of such repositories and try to populate the local repo with pull from all of the configured remote repos. Is that correct?
If the answer is yes, there could be merge conflicts of common files from different remote repos - how does the plugin handles it?
Thanks for your support in tracking this.
I also think that identifying the repo name on local workspace is needed for every SCM plugin. For example, if we take Multi-SCM plugin which configures a TFS project ($/Project1/Source) and a Git repo http://gitstore-host/project2.git (sub directory starts with Source directory) - then the directories from both SCM tool will be merged and create junk workspace useful for nothing.
When configuring the distinct directory of SCM repo, have option for the user to choose the local directory name. For example, the local config will look something like: http://host1/project1.git (WORKSPACE\Source\h1_project1), http://host1/project2.git (WORKSPACE\Source\h1_project2) and http://host2/project1.git (WORKSPACE\Source\h2_project1)
I also think that identifying the repo name on local workspace is needed for every SCM plugin. For example, if we take Multi-SCM plugin which configures a TFS project ($/Project1/Source) and a Git repo http://gitstore-host/project2.git (sub directory starts with Source directory) - then the directories from both SCM tool will be merged and create junk workspace useful for nothing.
I don't know if the TFS plugin works with the multi-SCM plugin, but the git plugin does work with it, and allows the user to specify a subdirectory of the workspace for each of the repositories used in that job. With the git plugin, after I've added the git repository to the Multi-SCM configuration of the job, then I need to add an "Additional behaviour" to that git configuration to checkout to a specific subdirectory.
multi-scm plugin is nice but it's not good enough, it does not keep the git env vars for all repos but overrides, so they are available only for the last one cloned.
git publisher can not be used in the post build actions since it doesn't recognize any git settings
"Project not currently configured to use Git; cannot check remote repository"
This should definitely be part of Jenkin's GIT plugin itself. With CI gaining popularity, more and more companies use multiple repos. Unfortunately, with Jenkins not supporting it natively, the multi-scm plugin breaks other plugins' CI functionality.
We've seen problems with:
- github pull request builder
- github plugin's push functionality
This also caused us to checkout with the multi-scm plugin in a specific order to ensure Jenkins has the correct version in its variables.
When using Build Forge 10 years ago, a single repo was normal. Not so much any more.
I have the same problem. I need to use 3 Git repos and I don't want to configure git modules.
The multiSCM plugin as it says in it's own wiki page is more a proof-of-concept currently then a fully implemented plugin. It has it's shortcommings (although I think the idea behind it should be a core feature of Jenkins) which we did hit in the past even when using SVN.
Voted up.
Really annoying that Jenkins doesnt support multiple GIT repos
Voted up.
+1
Found this thread after experiencing the same issue.
My job is configured with two git repos. When the job runs it appears to check out both repos but the workspace only shows contents from the first.
I believe this is a bug because the configuration in the UI isn't being produced in the workspace. I'm sure I can work around the issue and am hoping maintainers can find a solution soon.
Adding multiple independent repository support to the existing git plugin looks extremely difficult to me.
You might investigate using a Jenkinsfile in the root directory of your starting repository. I believe that Jenkinsfile can then be used to define the set of repositories you would like to checkout. There is a stackoverflow article which provides an example.
You can use Multiple SCM plugin for this. Just be sure that you order the repositories so that the "most root" subdirectory is cloned first, otherwise Git plugin will erase the files due to how git-clone requires an empty directory.
jekeller: Read comments above and read the documentation of the multi-scm plugin. It was written as a POC; it was not written for everyone to use and accept as "the best Jenkins has to offer". It is not a production ready plugin.
We currently do use the muti-scm plugin and we have problems weekly due to the fact that "This plugin is more of a proof-of-concept than a robust and fully functional component. It does work in my particular build environment, and is meant to serve as a demonstration of what might be possible with more work."
This is a limitation in Jenkins and anyone who accepts a POC plugin as a real solution needs to back up and think a bit.
I'm not saying it's the best or that it couldn't be improved, but it's what we have today that doesn't require more work.
Yes, but Multiple SCM plugin does need more work to not cause trouble...
In some cases it doesn't work at all (generates exceptions in the defined child SCM entry plugins), in some cases it works incorrectly, in some scenarios it doesn't report changes correctly; see cases related to that plugin. I like that it exists though and I did use it for a while when I needed to combine multiple types of repos into one job (but I also did bump into these problems from time to time when using it).
I still think that what MultipleSCM plugin does should be a core Jenkins feature (complete/fully operational of course) - is there a place where I can suggest this in a more official way? And then this case is no longer relevant as you could do it that way. And any other SCM plugin would only need to be implemented to handle one repo.
This is the right place for feature requests. The Jenkins pipeline allows multiple repositories while defining a pipeline. You might try that for your use case
Thanks for the hint.
So the pipeline basically skips all the UI for building jobs and configuring them and replaces that by using groovy scripts right?
We will not move to that right now I but we might in the future.
Is this ability to support multiple repos. a pipeline plugin ability or is it just based on some underlying API that the UI for configuring jobs uses as well? Cause in that case it might not be hard to add that ability there as well.
Pipeline scripts are groovy. A "snippet generator" is included in the pipeline job type assists by generating snippets of groovy code that can be inserted into a pipeline script. There is a "checkout" snippet, and there is a "git" snippet. One of those two should be able to help with the transition from pure UI driven job definition to pipeline based job definition.
I suspect job definition would require re-work to support multiple SCMs. It's not un-doable but no one has put forth effort to work on it. I think it's best to be a core Jenkins feature, and get the review necessary to support the workflow. I don't think it's too far out there, and I think even changes could be handled properly that way.
I think that this extension of features is worth doing even though pipelines support multiple repositories, as pipelines are more complex and not for everyone.
Pipeline is Groovy, which is easier than many scripting languages. If it's too complex for users, they probably need to find another industry.
As Pipeline is the way of the future with Jenkins, it is an acceptable solution to this problem to recommend Pipeline. With 2.x, Pipeline becomes part of the product (a core plugin). It's available as a plugin now. My team accepted there is no reason to put an enormous effort into 1.x functionality and has moved on to using Pipeline.
Since pipeline supports multiple git repositories, that is the preferred solution for this.
Supporting multiple repositories from inside the git plugin will introduce too many compatibility problems for the git plugin.
Yes, the pipeline script supports multiple git repositories to be checked out (cloned) into the corresponding the sub-directories.
But how should one write this script so that the polling of these repositories would trigger the job in case of updates? Please, see my question in details here: https://stackoverflow.com/questions/47439718/trigger-a-job-by-polling-multiple-git-repos-inside-jenkinsfile.
Pipelines are not usable as for now, because any triggers and parameters defined in them are ignored until you manually start the job.
The git plugin does not directly support checkout of distinct repositories under a single workspace.
If you want to support distinct repositories under a single workspace, you'll need to use the multi-SCM's plugin to configure the separate repositories.