-
Bug
-
Resolution: Fixed
-
Major
-
Jenkins 1.565.1 LTS
Copy Artifact Plugin 1.31
Hi,
We have a problem when passing Build selector for Copy Artifact as a parameter filter for an older job/build.
Suppose 3 Jobs:
- Job 1: DepJob
- Job 2: SubJob
- Build selector for Copy Artifact Parameter: SELECT_ME
- Default selector : Specific build
- Build number : 123
- Build step: Copy artifacts from another project:
- Project Name : DepJob
- Which build : Specified by a build parameter
- Parameter Name : SELECT_ME
- Build selector for Copy Artifact Parameter: SELECT_ME
- Job 3: MainJob
- Build selector for Copy Artifact Parameter: SELECT_DEPJOB
- Default selector : Specific build
- Build number : 123
- Build step: Copy artifacts from another project:
- Project Name : SubJob
- Which build : Latest successful build
- Parameter filters : SELECT_ME=$SELECT_DEPJOB
- Build selector for Copy Artifact Parameter: SELECT_DEPJOB
Quick issue description
When last successful build of SubJob has been built with an older version of the Copy Artifact Plugin, then MainJob cannot find the artifacts although they exist!
Details
- DepJob has a build 123.
- SubJob performs some builds with Copy Artifact Plugin version 1.26:
- Build selector for Copy Artifact Parameter:
- SELECT_ME: Specific build -> 123
- Build selector for Copy Artifact Parameter:
- Update Copy Artifact Plugin
- MainJob performs build with Copy Artifact Plugin version 1.31:
- Build selector for Copy Artifact Parameter:
- SELECT_DEPJOB: Specific build -> 123
- Build selector for Copy Artifact Parameter:
- MainJob build fails since it cannot find atifacts with SELECT_ME: Specific build -> 123 in the old builds of SubJob!
This is a blocker for us since we have a very large project which uses this kind of constructions for packaging all sub-projects (aka sub jobs).
We recently updated the Copy Artifact Plugin, but we still need lots of artifacts the old(er) sub-projects (for which currently no rebuilt is needed).
[JENKINS-24626] Parameter values of build selector change when upgrading CopyArtifact
Hi Daniel,
Thank you for the quick reply.
Yes, we could do so, but meanwhile we also have
a lot of builds which used the newer Copy Artifact Plugin.
We also need some bug fixes and features from the new version.
Maybe it's quite easy to disable the version check
or to add some backward compatibility?
Thank you in advance!
ikedam
added a comment - Let me know:
- Does SubJob work fine after upgrading copyartifact to 1.31?
- Did MainJob work fine before upgrading copyartifact to 1.31?
And could you attach following files?
- configurations of those jobs. They are placed at ${JENKINS_HOME}/jobs/(jobname)/config.xml
- console outputs for a succeeded build and a failed build.
ikedam
added a comment - Let me know:
Does SubJob work fine after upgrading copyartifact to 1.31?
Did MainJob work fine before upgrading copyartifact to 1.31?
And could you attach following files?
configurations of those jobs. They are placed at ${JENKINS_HOME}/jobs/(jobname)/config.xml
console outputs for a succeeded build and a failed build.
ikedam
added a comment - I found the root cause.
It is caused for SELECT_ME=$SELECT_DEPJOB does not work.
A build of SubJob with copyartifact 1.26 is recorded like this (see recorded with plugin="copyartifact@1.26"):
<hudson.model.ParametersAction>
<parameters>
<hudson.model.StringParameterValue>
<name>SELECT_ME</name>
<description></description>
<value><SpecificBuildSelector plugin="copyartifact@1.26"> <buildNumber>1</buildNumber></SpecificBuildSelector></value>
</hudson.model.StringParameterValue>
</parameters>
</hudson.model.ParametersAction>
When running MainJob is started with copyartifact 1.31, the build is launched like this (see recorded with plugin="copyartifact@1.31"):
<hudson.model.ParametersAction>
<parameters>
<hudson.model.StringParameterValue>
<name>SELECT_DEPJOB</name>
<description></description>
<value><SpecificBuildSelector plugin="copyartifact@1.31"> <buildNumber>1</buildNumber></SpecificBuildSelector></value>
</hudson.model.StringParameterValue>
</parameters>
</hudson.model.ParametersAction>
And they doesn't satisfy SELECT_ME=$SELECT_DEPJOB.
As that serialization is the work of Jenkins core (or Jenkins framework), and I'm not sure I can fix that.
I think we should consider an alternate way to do what @tom_ghyselinck want to do.
ikedam
added a comment - I found the root cause.
It is caused for SELECT_ME=$SELECT_DEPJOB does not work.
A build of SubJob with copyartifact 1.26 is recorded like this (see recorded with plugin="copyartifact@1.26"):
<hudson.model.ParametersAction>
<parameters>
<hudson.model.StringParameterValue>
<name>SELECT_ME</name>
<description></description>
<value><SpecificBuildSelector plugin="copyartifact@1.26"> <buildNumber>1</buildNumber></SpecificBuildSelector></value>
</hudson.model.StringParameterValue>
</parameters>
</hudson.model.ParametersAction>
When running MainJob is started with copyartifact 1.31, the build is launched like this (see recorded with plugin="copyartifact@1.31"):
<hudson.model.ParametersAction>
<parameters>
<hudson.model.StringParameterValue>
<name>SELECT_DEPJOB</name>
<description></description>
<value><SpecificBuildSelector plugin="copyartifact@1.31"> <buildNumber>1</buildNumber></SpecificBuildSelector></value>
</hudson.model.StringParameterValue>
</parameters>
</hudson.model.ParametersAction>
And they doesn't satisfy SELECT_ME=$SELECT_DEPJOB .
As that serialization is the work of Jenkins core (or Jenkins framework), and I'm not sure I can fix that.
I think we should consider an alternate way to do what @tom_ghyselinck want to do. Hi ikedam,
Thank you for the update!
I would be great if you can find a workaround or an alternative way for it!
Is it an option to provide the values string-based instead of xml-based?
Using custom encoder / decoder? Or provide a custom dumper which does not add the version info?
This would allow for some backward compatibility.
ikedam
added a comment - Though I believe I can remove "plugin" attribute by overwiting XStream2#createDefaultConverter, that's not a good approach as it might cause copyartifact stop working in a future version of Jenkins.
https://github.com/jenkinsci/jenkins/blob/jenkins-1.565.1/core/src/main/java/hudson/util/XStream2.java#L122
Rather, I'm considering adding a new build selector "Downstream of". You can pick the build like this:
- Which build: Downstream of
- Upstream project: DepJob
- Upstream build #: 123
- Target Downstream Project: MainJob
ikedam
added a comment - Though I believe I can remove "plugin" attribute by overwiting XStream2#createDefaultConverter , that's not a good approach as it might cause copyartifact stop working in a future version of Jenkins.
https://github.com/jenkinsci/jenkins/blob/jenkins-1.565.1/core/src/main/java/hudson/util/XStream2.java#L122
Rather, I'm considering adding a new build selector "Downstream of". You can pick the build like this:
Which build: Downstream of
Upstream project: DepJob
Upstream build #: 123
Target Downstream Project: MainJob
Hi ikedam,
The Downstream of seems a feasible solution for our problem.
Am I correct: This selector is not subject to the "plugin" property problem because:
- The iteration on the builds of SubJob will be handled internally and
- the "plugin" property will not be verified?
(i.e. it is not a bare string comparison)
So, if I see it correct, the configuration would be as follows:
- Job 1: DepJob
- Job 2: SubJob
- Build selector for Copy Artifact Parameter: SELECT_ME
- Default selector : Specific build
- Build number : 123
- Build step: Copy artifacts from another project:
- Project Name : DepJob
- Which build : Specified by a build parameter
- Parameter Name : SELECT_ME
- Build selector for Copy Artifact Parameter: SELECT_ME
- Job 3: MainJob
- Build selector for Copy Artifact Parameter: SELECT_DEPJOB
- Default selector : Downstream of
- Upstream project : DepJob
- Build number : 123
- Build step: Copy artifacts from another project:
- Project Name : SubJob
- Which build : Specified by a build parameter
- Parameter Name : SELECT_DEPJOB
- Parameter filters : none
- Build selector for Copy Artifact Parameter: SELECT_DEPJOB
Just one final remark
Shouldn't it be called Upstream of instead of Downstream of?
Rationale: In the MainJob configuration: You choose the upstream (i.e. DepJob) project selection of the SubJob.
P.S. Thank you very much for the quick responses! It is really appreciated!
ikedam
added a comment - Yes, it no longer tests builds by string comparison, and we will be no longer bothered by "plugin=".
I also have to add texts not to use build selecter variables as parameter filters.
I don't agree "Upstream of" as it's confusing especially when used in a project configuration page.
ikedam
added a comment - Yes, it no longer tests builds by string comparison, and we will be no longer bothered by "plugin=".
I also have to add texts not to use build selecter variables as parameter filters.
I don't agree "Upstream of" as it's confusing especially when used in a project configuration page. OK, I'll let you decide how to name it.
The important thing for us is that it works 
Thank you for the update!
ikedam
added a comment - https://github.com/jenkinsci/copyartifact-plugin/pull/44 
Ray Sennewald
added a comment - How difficult would it be to add support for MultiJob's here? It would be great if I could use this with MultiJobs, but it appears that a job that is triggered as part of a Phase of the MultiJob isn't considered downstream. Would it be possible to add support for this?
Ray Sennewald
added a comment - How difficult would it be to add support for MultiJob's here? It would be great if I could use this with MultiJobs, but it appears that a job that is triggered as part of a Phase of the MultiJob isn't considered downstream. Would it be possible to add support for this? ikedam
added a comment - > MultiJob support
I don't plan that at least for the first release of this feature.
Please create another ticket.
Anyway, I don't know much about MultiJob (I haven't ever used that)
and I cannot contribute for that.
ikedam
added a comment - > MultiJob support
I don't plan that at least for the first release of this feature.
Please create another ticket.
Anyway, I don't know much about MultiJob (I haven't ever used that)
and I cannot contribute for that. Code changed in jenkins
User: ikedam
Path:
src/main/java/hudson/plugins/copyartifact/DownstreamBuildSelector.java
src/main/resources/hudson/plugins/copyartifact/DownstreamBuildSelector/config.jelly
src/main/resources/hudson/plugins/copyartifact/Messages.properties
src/main/resources/hudson/plugins/copyartifact/Messages_ja.properties
http://jenkins-ci.org/commit/copyartifact-plugin/1515de100de0d52575f9c13cdccbf9c20a5813c4
Log:
[FIXED JENKINS-24626] Added a new build selector "Downstream of"
Code changed in jenkins
User: ikedam
Path:
src/main/java/hudson/plugins/copyartifact/DownstreamBuildSelector.java
src/main/resources/hudson/plugins/copyartifact/DownstreamBuildSelector/config.jelly
src/main/resources/hudson/plugins/copyartifact/DownstreamBuildSelector/config_ja.properties
src/main/resources/hudson/plugins/copyartifact/DownstreamBuildSelector/help-upstreamBuildNumber.html
src/main/resources/hudson/plugins/copyartifact/DownstreamBuildSelector/help-upstreamBuildNumber_ja.html
src/main/resources/hudson/plugins/copyartifact/DownstreamBuildSelector/help-upstreamProjectName.html
src/main/resources/hudson/plugins/copyartifact/DownstreamBuildSelector/help-upstreamProjectName_ja.html
src/main/resources/hudson/plugins/copyartifact/Messages.properties
src/main/resources/hudson/plugins/copyartifact/Messages_ja.properties
http://jenkins-ci.org/commit/copyartifact-plugin/7277e1d46b9f630b0caec62b2a6791f471bb203a
Log:
JENKINS-24626 Added permission check, help texts for DownstreamBuildSelector.
Code changed in jenkins
User: ikedam
Path:
src/main/java/hudson/plugins/copyartifact/DownstreamBuildSelector.java
src/test/java/hudson/plugins/copyartifact/DownstreamBuildSelectorTest.java
http://jenkins-ci.org/commit/copyartifact-plugin/83219279e24f36c14cd8d049a7a2d9c316d0e275
Log:
JENKINS-24626 Added tests for DownstreamBuildSelector.
Code changed in jenkins
User: ikedam
Path:
src/main/resources/hudson/plugins/copyartifact/BuildSelectorParameter/help.html
src/main/resources/hudson/plugins/copyartifact/BuildSelectorParameter/help_de.html
src/main/resources/hudson/plugins/copyartifact/BuildSelectorParameter/help_ja.html
src/main/resources/hudson/plugins/copyartifact/CopyArtifact/help-parameters.html
http://jenkins-ci.org/commit/copyartifact-plugin/0d970525e59d0e3ab6fd4a477b9c224fa6a8ece0
Log:
JENKINS-24626 Add notices for compatibility of BuildSelectorParameter.
Code changed in jenkins
User: ikedam
Path:
src/main/java/hudson/plugins/copyartifact/DownstreamBuildSelector.java
src/main/resources/hudson/plugins/copyartifact/BuildSelectorParameter/help.html
src/main/resources/hudson/plugins/copyartifact/BuildSelectorParameter/help_de.html
src/main/resources/hudson/plugins/copyartifact/BuildSelectorParameter/help_ja.html
src/main/resources/hudson/plugins/copyartifact/CopyArtifact/help-parameters.html
src/main/resources/hudson/plugins/copyartifact/DownstreamBuildSelector/config.jelly
src/main/resources/hudson/plugins/copyartifact/DownstreamBuildSelector/config_ja.properties
src/main/resources/hudson/plugins/copyartifact/DownstreamBuildSelector/help-upstreamBuildNumber.html
src/main/resources/hudson/plugins/copyartifact/DownstreamBuildSelector/help-upstreamBuildNumber_ja.html
src/main/resources/hudson/plugins/copyartifact/DownstreamBuildSelector/help-upstreamProjectName.html
src/main/resources/hudson/plugins/copyartifact/DownstreamBuildSelector/help-upstreamProjectName_ja.html
src/main/resources/hudson/plugins/copyartifact/Messages.properties
src/main/resources/hudson/plugins/copyartifact/Messages_ja.properties
src/test/java/hudson/plugins/copyartifact/DownstreamBuildSelectorTest.java
http://jenkins-ci.org/commit/copyartifact-plugin/e80d72c7cde37bc53699acfaddbcd95b21f0522a
Log:
Merge pull request #44 from ikedam/feature/JENKINS-24626_DownstreamOf
JENKINS-24626 Added a selector "Downstream of"
Compare: https://github.com/jenkinsci/copyartifact-plugin/compare/a7ce03cff19f...e80d72c7cde3
ikedam
added a comment - Let me have one more week to make a new release (1.32).
I'm working on fixes for other bugs.
I can provide a snapshot version that includes this feature if you want.
Let me know if you need this feature soon.
ikedam
added a comment - Let me have one more week to make a new release (1.32).
I'm working on fixes for other bugs.
I can provide a snapshot version that includes this feature if you want.
Let me know if you need this feature soon. Hi ikedam,
Thank you for the update.
Next week would be fine for us!
This week we can still use other builds for testing.
Thank you in advance!
Ray Sennewald
added a comment - Hi ikedam. I found an existing ticket for what I was wanting to implement for MultiJob: https://issues.jenkins-ci.org/browse/JENKINS-16847. Any chance you could review my work there?
Ray Sennewald
added a comment - Hi ikedam. I found an existing ticket for what I was wanting to implement for MultiJob: https://issues.jenkins-ci.org/browse/JENKINS-16847 . Any chance you could review my work there? ikedam
added a comment - You can use "Downstream build of" to pick a build in a same multijob.
- Pass the build number of the multijob build to downstream builds.
- Use as a parameter for "Downstream build of").
- Fingerprint files with a same content in the parent multijob build and the child build to copy artifacts from.
- Fingerprinting is required for "Downstream build of" to find downstream builds.
- You need Fingerprint plugin to fingerprint in multijob.
- ${BUILD_ID} of the parent multijob build is good for the content of the file.
I know this is just a workaround as it's not easy to use.
The best way is to introduce a new buld selector to multijob-plugin.
ikedam
added a comment - You can use "Downstream build of" to pick a build in a same multijob.
Pass the build number of the multijob build to downstream builds.
Use as a parameter for "Downstream build of").
Fingerprint files with a same content in the parent multijob build and the child build to copy artifacts from.
Fingerprinting is required for "Downstream build of" to find downstream builds.
You need Fingerprint plugin to fingerprint in multijob.
${BUILD_ID} of the parent multijob build is good for the content of the file.
I know this is just a workaround as it's not easy to use.
The best way is to introduce a new buld selector to multijob-plugin. Code changed in jenkins
User: ikedam
Path:
src/main/java/hudson/plugins/copyartifact/DownstreamBuildSelector.java
http://jenkins-ci.org/commit/copyartifact-plugin/bdbfc6f1b7bc86d6184547c23c7e1d59ceb3feb3
Log:
JENKINS-24626 Added auto-completion for project names in "Downstream build of".
Code changed in jenkins
User: ikedam
Path:
src/main/java/hudson/plugins/copyartifact/DownstreamBuildSelector.java
http://jenkins-ci.org/commit/copyartifact-plugin/06322b202497b3468a85c97c2033d05c771060b4
Log:
Merge pull request #50 from ikedam/feature/JENKINS-24626_AutoCompleteForDownstreamOf
JENKINS-24626 Added auto-completion for project names in "Downstream build of".
Compare: https://github.com/jenkinsci/copyartifact-plugin/compare/c9809c881745...06322b202497
Regarding the blocker priority, can't you just downgrade the plugin?