Details
-
Type:
New Feature
-
Status: Closed (View Workflow)
-
Priority:
Critical
-
Resolution: Fixed
-
Component/s: gitlab-hook-plugin
-
Labels:None
-
Similar Issues:
Description
Currently gitlab-hook-plugin searches all the jobs that match given repository url.
In fact, I use multijob configuration like the following:
MainJob <-- uses git configuration so gitlab-hook-plugin can find & clone & start it * Phase 1: compile job <-- uses git to fetch project sources * Phase 2: deploy * Phase 3: fast tests <-- uses git to fetch tests ...
Current gitlab-hook-plugin triggers all the jobs, not just the main one.
In my installation I've modified plugins\gitlab-hook\WEB-INF\classes\models\use_cases\process_commit.rb }} as follows (added {{ && (project.description.match /#{settings.description}/)) so it triggers only jobs with autogenerated description:
def get_projects_to_process(details)
projects = @get_jenkins_projects.matching_uri(details)
if projects.any?
if settings.automatic_project_creation?
projects.select! do |project|
project.matches?(details, details.branch, true) && (project.description.match /#{settings.description}/)
end
projects << @create_project_for_branch.with(details) if projects.empty?
However, it could make sense to use another approach: get the job by name (hook knows naming convention, so it can get the job) check if that matches. In case the job does not match fallback to the current implementation.
Attachments
Issue Links
- is related to
-
-
- Closed
-
-
-
- Open
-
Activity
Javier Palacios, my case is as follows:
1) I have a project at Gitlab. Suppose it is named "ProjectFoo".
2) I want each branch to spawn a dedicated Jenkins job.
The intention is as follows: each developer works in his/her own branch, thus having separate Jenkins jobs helps a lot.
However, the build process itself is not that simple.
Basically it includes several stages:
S1) Fetch & compile project
S2) Prepare database (each test is to be executed in a clean environment, thus each job execution gets its own database schema)
S3) Deploy the code (run installation SQL scripts, etc, etc)
S4) Execute fast tests (if they fail, no need to run full suite)
S5) Execute full suite of tests (here additional DB instances are provisioned to test against different DB versions, etc)
S6) Uninstall (to test uninstall)
I think it is somewhat typical scenario for integration tests.
In order to simplify management of those integration jobs, I use Job DSL plugin.
In other words, I have a "seed job" that generates all those S1, S2, ..., S3 given the input parameters like "branch name", "DB versions to test", etc.
The "seed job" has just a single step: "invoke groovy script".
Job management via groovy scripts turned out to be very efficient.
So my end-to-end is:
E1) A developer pushes commit to Gitlab
E2) Gitlab notifies Jenkins. Here only seed job should be triggered.
E3) Seed job fetches project sources, figures out the proper set of integration jobs, creates/updates as required, then triggers the actual integration testing.
The problem is at step E2 my Jenkins server has lots of jobs with given GIT project URL.
For each project I have exactly one seed job, so I do not want to trigger all of them.
Does that make sense?
Vladimir Sitnikov
added a comment - Javier Palacios , my case is as follows:
1) I have a project at Gitlab. Suppose it is named "ProjectFoo".
2) I want each branch to spawn a dedicated Jenkins job.
The intention is as follows: each developer works in his/her own branch, thus having separate Jenkins jobs helps a lot.
However, the build process itself is not that simple.
Basically it includes several stages:
S1) Fetch & compile project
S2) Prepare database (each test is to be executed in a clean environment, thus each job execution gets its own database schema)
S3) Deploy the code (run installation SQL scripts, etc, etc)
S4) Execute fast tests (if they fail, no need to run full suite)
S5) Execute full suite of tests (here additional DB instances are provisioned to test against different DB versions, etc)
S6) Uninstall (to test uninstall)
I think it is somewhat typical scenario for integration tests.
In order to simplify management of those integration jobs, I use Job DSL plugin.
In other words, I have a "seed job" that generates all those S1, S2, ..., S3 given the input parameters like "branch name", "DB versions to test", etc.
The "seed job" has just a single step: "invoke groovy script".
Job management via groovy scripts turned out to be very efficient.
So my end-to-end is:
E1) A developer pushes commit to Gitlab
E2) Gitlab notifies Jenkins. Here only seed job should be triggered.
E3) Seed job fetches project sources, figures out the proper set of integration jobs, creates/updates as required, then triggers the actual integration testing.
The problem is at step E2 my Jenkins server has lots of jobs with given GIT project URL.
For each project I have exactly one seed job, so I do not want to trigger all of them.
Does that make sense? Yes, it does make a lot of sense, but I'm not sure if it's doable in the context of this plugin. It basically depends on how non-build phases are done. I'll tell you the flow I used for this same scenario, although the config.xml of the jobs is still useful to fully understand your flow.
In my case, I have a single environment where to deploy artifacts and run the tests. Everything was built as debian packages, so I got an additional element, but I don't believe it is important. The build job was the only one that performed a git clone, and also to deploy the package to a repository which was the same for all the jobs. The deploy & test part was a bit complex, but there was a single set of them, and the connection was done with the standard jenkins triggering. This was a bit o a mesh, as concurrent builds are very dangerous, and in any case the flow graph was a bit ugly, but fortunately every repository used to have only one developer working on it (although there were plenty of repositories, libraries triggering other jobs and stuff like that). And tracking which builds introduced which error was far from being straightforward.
But I had separated (multiple) phase 1, and single unified later phases.
If your deploy & test phases must be multiple ones (maybe a full environment is deployed for them), I would suggest to write all those parts into the phase 1 job, to get a single job that could be easily cloned. But when I get a bit of time I will look to multijob, because there should be some link among them within the project configuration.
In the long term, I've made some attempts towards a real trigger, that should be configured not on global jenkins configuration, but enabled on every job. And at some point in that (not short) path there is probably a solution for your problem, as you could just select your seed project as the only one to be considered.
I have lots of jobs that access git for different reasons:
0) "seed job" to get "integration configuration"
1) compile job to compile the project
2) test job to get test sources
etc, etc.
In other words, what I want from Gitlab-Jenkins integration is that the plugin triggers ONLY the seed job.
I'm fine to set "trigger job in case of gitlab push" kind of flag on a seed job.
connection was done with the standard jenkins triggering
Multijob connection enables to have:
1) Single status view of each step
2) Single "stop all" button
3) Nice "cleanup if anything went wrong" steps
4) etc, etc.
However multijob stuff is completely irrelevant for the particular issue. The issue is to have an ability to limit the set of triggered jobs. Why trigger all the jobs with given url?
Vladimir Sitnikov
added a comment - - edited I have lots of jobs that access git for different reasons:
0) "seed job" to get "integration configuration"
1) compile job to compile the project
2) test job to get test sources
etc, etc.
In other words, what I want from Gitlab-Jenkins integration is that the plugin triggers ONLY the seed job.
I'm fine to set "trigger job in case of gitlab push" kind of flag on a seed job.
connection was done with the standard jenkins triggering
Multijob connection enables to have:
1) Single status view of each step
2) Single "stop all" button
3) Nice "cleanup if anything went wrong" steps
4) etc, etc.
However multijob stuff is completely irrelevant for the particular issue. The issue is to have an ability to limit the set of triggered jobs. Why trigger all the jobs with given url? However multijob stuff is completely irrelevant for the particular issue. The issue is to have an ability to limit the set of triggered jobs. Why trigger all the jobs with given url?
That's a matter of design. The plugin is not actually a Trigger object, but a RootAction one. It allows to use the same url on all gitlab repositories, but reduces the number of project specific things that can be done (in particular, most of advanced features of git plugin are not honoured). My plans for trigger alike behaviour will overcome this limitation, but it is not a matter of weeks.
It is acceptable to set the webhook url to something like http://jenkins.url/gitlab/build_now/seedproject? I've made a fast check, and apparently the plugin can handle such a url, so it should be hard to just trigger seedproject in that case.
It is acceptable to set the webhook url to something like http://jenkins.url/gitlab/build_now/seedproject?
I think so.
However, in the long run "dumb configuration of webhook" + "smart configuration at jenkins side" might be better.
so it should be hard to just trigger seedproject in that case.
You mean "it should not be hard to trigger in that case", don't you?
Vladimir Sitnikov
added a comment - It is acceptable to set the webhook url to something like http://jenkins.url/gitlab/build_now/seedproject?
I think so.
However, in the long run "dumb configuration of webhook" + "smart configuration at jenkins side" might be better.
so it should be hard to just trigger seedproject in that case.
You mean "it should not be hard to trigger in that case", don't you? Right. Was as not hard that you have a preliminar but working version on http://maven.jenkins-ci.org/content/repositories/snapshots/org/jenkins-ci/ruby-plugins/gitlab-hook/1.4.2-SNAPSHOT/
I've no experience with multijob projects, but the plugin does search projects that match the repository url, so if your phase 2 & 3 are triggered, it is because they also clone your git repository.
Your suggestion is not bad, but it breaks everything else, as every job created manually will stop building on pushes. A configuration item could be added to enable this "only autocreated" behaviour, but I would like to understand your use case before attempting it.
Regarding your second suggestion, if I understand it well, has the same problem of your current modification, and forces to name the project as the repository, and although it's probably happening with most cases, nothing guarantees that.