• Icon: New Feature New Feature
    • Resolution: Postponed
    • Icon: Major Major
    • multijob-plugin
    • None

      We have MultiJob Projects setup in which we do a build in the first phase and test in the second phase. We want to grab the artifact from the first phase as part of the current MultiJob Build in the second phase as well as at the end of the MultiJob itself. It would be great if MultiJob Plugin could add support for this as Copy Artifact Plugin prefers this code to live in MultiJob Plugin as its specific for it.

          [JENKINS-25111] Ability to Copy Artifacts from a Phase Job

          Code changed in jenkins
          User: itaior
          Path:
          pom.xml
          src/main/java/com/tikal/jenkins/plugins/multijob/MultiJobBuildSelector.java
          src/main/resources/com/tikal/jenkins/plugins/multijob/Messages.properties
          src/main/resources/com/tikal/jenkins/plugins/multijob/MultiJobBuildSelector/config.jelly
          src/main/webapp/help-copyArtifact.html
          src/test/java/com/tikal/jenkins/plugins/multijob/test/testutils/FileWriteBuilder.java
          http://jenkins-ci.org/commit/tikal-multijob-plugin/9061bb42b52507d3d3624afb6890a3171315c4bc
          Log:
          Merge pull request #54 from rsennewald/JENKINS-25111

          JENKINS-25111 Copy Artifacts from Phase Job

          Compare: https://github.com/jenkinsci/tikal-multijob-plugin/compare/0cd86c543fa3...9061bb42b525

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: itaior Path: pom.xml src/main/java/com/tikal/jenkins/plugins/multijob/MultiJobBuildSelector.java src/main/resources/com/tikal/jenkins/plugins/multijob/Messages.properties src/main/resources/com/tikal/jenkins/plugins/multijob/MultiJobBuildSelector/config.jelly src/main/webapp/help-copyArtifact.html src/test/java/com/tikal/jenkins/plugins/multijob/test/testutils/FileWriteBuilder.java http://jenkins-ci.org/commit/tikal-multijob-plugin/9061bb42b52507d3d3624afb6890a3171315c4bc Log: Merge pull request #54 from rsennewald/ JENKINS-25111 JENKINS-25111 Copy Artifacts from Phase Job Compare: https://github.com/jenkinsci/tikal-multijob-plugin/compare/0cd86c543fa3...9061bb42b525

          Paul Holmberg added a comment -

          We pulled the change above and rebuilt the plugin but still got 'unable to find job to copy artifact from' errors. We are trying to copy junit test result files from child phases up to the parent job so as to collate build results in one place.

          In case someone else is currently blocked by this there is a workaround using the EnvInject plugin. After your build phases have run execute a python build step with the following code. It uses the Jenkins JSON API to create a new file called propsfile containing the build numbers and success/failures of all the child phases that have executed so far as JOB_NAME_BUILD_NUMBER and JOB_NAME_BUILD_SUCCESS (note it replaces hyphens with underscores to avoid invalid environment variables).

          #!/usr/bin/env python
          import os
          import json
          import urllib2
          
          url = os.environ['BUILD_URL']
          
          req = urllib2.Request('%s/api/json' % url, headers = {'Accept' : 'application/json'})
          res = urllib2.urlopen(req)
          res = json.loads(res.read())
          
          f = open('propsfile', 'w')
          try:
              buildNumbers = {}
              for subbuild in res['subBuilds']:
                  name = subbuild['jobName'].upper()
                  name = name.replace('-', '_')
                  f.write('%s_BUILD_NUMBER=%s\n' % (name, subbuild['buildNumber']))
                  f.write('%s_BUILD_SUCCESS=%d\n' % (name, subbuild['result'].lower() == 'success'))
          finally:
              f.close()
          

          Then add a subsequent envinject step to pull the contents of that propsfile in and you can use the resulting environment variables to copy artifacts from the correct triggered child builds. You can also use the build success environment variables and the ConditionalJob plugin to optionally run steps (in the case where you have an early smoke test phase that prevents the lengthy subsequent phases from executing if it fails).

          Paul Holmberg added a comment - We pulled the change above and rebuilt the plugin but still got 'unable to find job to copy artifact from' errors. We are trying to copy junit test result files from child phases up to the parent job so as to collate build results in one place. In case someone else is currently blocked by this there is a workaround using the EnvInject plugin. After your build phases have run execute a python build step with the following code. It uses the Jenkins JSON API to create a new file called propsfile containing the build numbers and success/failures of all the child phases that have executed so far as JOB_NAME_BUILD_NUMBER and JOB_NAME_BUILD_SUCCESS (note it replaces hyphens with underscores to avoid invalid environment variables). #!/usr/bin/env python import os import json import urllib2 url = os.environ[ 'BUILD_URL' ] req = urllib2.Request( '%s/api/json' % url, headers = { 'Accept' : 'application/json' }) res = urllib2.urlopen(req) res = json.loads(res.read()) f = open( 'propsfile' , 'w' ) try : buildNumbers = {} for subbuild in res[ 'subBuilds' ]: name = subbuild[ 'jobName' ].upper() name = name.replace( '-' , '_' ) f.write( '%s_BUILD_NUMBER=%s\n' % (name, subbuild[ 'buildNumber' ])) f.write( '%s_BUILD_SUCCESS=%d\n' % (name, subbuild[ 'result' ].lower() == 'success' )) finally : f.close() Then add a subsequent envinject step to pull the contents of that propsfile in and you can use the resulting environment variables to copy artifacts from the correct triggered child builds. You can also use the build success environment variables and the ConditionalJob plugin to optionally run steps (in the case where you have an early smoke test phase that prevents the lengthy subsequent phases from executing if it fails).

          FYI, for me the patch is working (except that it did not support build filters until [1] was merged just today). You can look at the job config at [2] to see how we're using it.

          [1] https://github.com/jenkinsci/tikal-multijob-plugin/pull/64
          [2] https://dscho.cloudapp.net/job/gfw-msys1-test-git-multi-node/configure

          Sebastian Schuberth added a comment - FYI, for me the patch is working (except that it did not support build filters until [1] was merged just today). You can look at the job config at [2] to see how we're using it. [1] https://github.com/jenkinsci/tikal-multijob-plugin/pull/64 [2] https://dscho.cloudapp.net/job/gfw-msys1-test-git-multi-node/configure

          This patch also works for me for a simple 1 phase multijob (tried revision 17c94362f0111b12b7dea7d1a9728666f9949a39) and provides a very neat solution to the problem of triggering multiple jobs and gathering results. Is there a plan to release a new version of the plugin with this functionality soon?

          Russell Gallop added a comment - This patch also works for me for a simple 1 phase multijob (tried revision 17c94362f0111b12b7dea7d1a9728666f9949a39) and provides a very neat solution to the problem of triggering multiple jobs and gathering results. Is there a plan to release a new version of the plugin with this functionality soon?

          It appears that the patch does not work with folders (Using revision 17c94362 of multi-job-plugin). I have configured something like:
          Copy artifacts from another project
          Project name: folder/job
          Which build: Build triggered by current MultiJob build

          And in the console log I get:
          ERROR: Unable to find a build for artifact copy from: folder/job

          It works if I use "Last successful build" and it works if I use a project name without the folder, just not folders in combination with "Build triggered by current MultiJob build".

          Russell Gallop added a comment - It appears that the patch does not work with folders (Using revision 17c94362 of multi-job-plugin). I have configured something like: Copy artifacts from another project Project name: folder/job Which build: Build triggered by current MultiJob build And in the console log I get: ERROR: Unable to find a build for artifact copy from: folder/job It works if I use "Last successful build" and it works if I use a project name without the folder, just not folders in combination with "Build triggered by current MultiJob build".

          I created a Pull Request to fix the problem with folders: https://github.com/jenkinsci/tikal-multijob-plugin/pull/74

          Olivier Sechet added a comment - I created a Pull Request to fix the problem with folders: https://github.com/jenkinsci/tikal-multijob-plugin/pull/74

          Code changed in jenkins
          User: osechet
          Path:
          src/main/java/com/tikal/jenkins/plugins/multijob/MultiJobBuildSelector.java
          http://jenkins-ci.org/commit/tikal-multijob-plugin/f326c2f74b232e07c891dc2a9e5382c6c0befebb
          Log:
          Fix problem when using folders

          As mentioned in a comment of the JENKINS-25111 issue, the Multijob build selector does not work when using folders. The problem is due to the use of getFullDisplayName() instead of getDisplayName() when looking for a sub build. getFullDisplayName() includes the parent names whereas sub build's job name contains only the job name. It's the reason the check couldn't find a match.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: osechet Path: src/main/java/com/tikal/jenkins/plugins/multijob/MultiJobBuildSelector.java http://jenkins-ci.org/commit/tikal-multijob-plugin/f326c2f74b232e07c891dc2a9e5382c6c0befebb Log: Fix problem when using folders As mentioned in a comment of the JENKINS-25111 issue, the Multijob build selector does not work when using folders. The problem is due to the use of getFullDisplayName() instead of getDisplayName() when looking for a sub build. getFullDisplayName() includes the parent names whereas sub build's job name contains only the job name. It's the reason the check couldn't find a match.

          Code changed in jenkins
          User: Shachar Ben-Zeev
          Path:
          src/main/java/com/tikal/jenkins/plugins/multijob/MultiJobBuildSelector.java
          http://jenkins-ci.org/commit/tikal-multijob-plugin/93a4453f1c253ddf602a695b992a2a3886e5a0ff
          Log:
          Merge pull request #74 from osechet/JENKINS-25111-folders

          JENKINS-25111 Fix problem in MultijobBuildSelector when using folders

          Compare: https://github.com/jenkinsci/tikal-multijob-plugin/compare/bf52cc2b1f62...93a4453f1c25

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Shachar Ben-Zeev Path: src/main/java/com/tikal/jenkins/plugins/multijob/MultiJobBuildSelector.java http://jenkins-ci.org/commit/tikal-multijob-plugin/93a4453f1c253ddf602a695b992a2a3886e5a0ff Log: Merge pull request #74 from osechet/ JENKINS-25111 -folders JENKINS-25111 Fix problem in MultijobBuildSelector when using folders Compare: https://github.com/jenkinsci/tikal-multijob-plugin/compare/bf52cc2b1f62...93a4453f1c25

          Geoffroy Jabouley added a comment - - edited

          Hi

          will this feature support multijob of multijob?

          Lets say i have the following job organization:

          multiJobA
          |--> PhaseA1
          |------> jobA1
          |--> PhaseA2 
          |------> multijobB
                   |--> PhaseB1
                   |--------> jobB1
          

          I need to retrieve an artefact from jobA1 in jobB1.
          We currently pass the JOBA1_BUILD_NUMBER variable as parameter of MultijobB.

          Any change to use this new functionality with our use case?

          Geoffroy Jabouley added a comment - - edited Hi will this feature support multijob of multijob? Lets say i have the following job organization: multiJobA |--> PhaseA1 |------> jobA1 |--> PhaseA2 |------> multijobB |--> PhaseB1 |--------> jobB1 I need to retrieve an artefact from jobA1 in jobB1. We currently pass the JOBA1_BUILD_NUMBER variable as parameter of MultijobB. Any change to use this new functionality with our use case?

          Closing issue as part of tikal-multijob-plugin issues cleanup.
          If still relevant, please open a matching issue in https://github.com/jenkinsci/tikal-multijob-plugin/issues (you can refer to this issue in its description)

          Yoram Michaeli added a comment - Closing issue as part of tikal-multijob-plugin issues cleanup. If still relevant, please open a matching issue in https://github.com/jenkinsci/tikal-multijob-plugin/issues (you can refer to this issue in its description)

            rsennewald Ray Sennewald
            rsennewald Ray Sennewald
            Votes:
            5 Vote for this issue
            Watchers:
            11 Start watching this issue

              Created:
              Updated:
              Resolved: