Status: Open (View Workflow)
Jenkins: 1.609/1.650 Windows
Dynamic Axis: 1.0.3
Matrix Project: 1.6
We create a matrix project which gets its dynamic axes from build parameter. The job is doeing nothing instead printing some messages and wait an amount of time. Now we create an upstream project. This project calls the matrix project with different axis as a build parameter. This means we call the project and this starts parallel the matrix project with diferent axis.
The projects are run with the correct matrix, but sometimes the job configuration contains the matrix values from another parallel build. This is visible if you compare the console output with the generated links in the build status UI.
We attached the job configurations and one build.xml where you can see that the builded matrix differs from the given build parameter.
I have a similar issue and tried to fix it.
I found that the method "MatrixBuild.MatrixBuildExecution.doRun" is called in multiple threads without locks. If there are parallel builds of the same projects they sometimes call the "MatrixProject.rebuildConfigurations" at the same time before they can get the axis -> the build that is a bit slower overwrites the axes of the first one.
I created a mutex for the MatrixProject instance and locked it before the "MatrixProject.rebuildConfigurations" is triggered and unlocked it after the axis is copied. Additional I used the fix from JENKINS-30437 for a deep copy of the axes.
The full fix can be found here (based on matrix-project 1.6).
something new about this issue?
Can i help by giving more analyze information?
Thanks in advance
bboehmke I couldn't find an PR for your mutex patch. Was it never submitted or is there a reason why it isn't mergable?
The patch I made was only a simple fix to get it working.
I think the part with the mutex is OK but the way the deep copy is done is not very well.
This should be done with correct copy constructors of the "AxisList" I think.
If you think the fix is enough I can create a pull request.
bboehmke I am completely unqualified to judge thread synchronization in the jenkins core. I would recommend opening a PR as some people (at least me) often check open PRs for unmerged fixes. I haven't tried the patch yet but I am contemplating trying to rebase it on the current 1.18 release.
I created a pull request here.
We are currently using this patch in our Jenkins instance at work.
There is only one visible down side: The mutex object is dumped to the job config.
This has no effect but looks strange.
Code changed in jenkins
User: Benjamin Böhmke
added unit test for JENKINS-34389
Code changed in jenkins
User: Oliver Gondža
Merge pull request #41 from bboehmke/dynamicAxis-fix
JENKINS-34389 Fixed handling of dynamic axis
addendum: i thin this issue is similar to JENKINS-30437