• Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Critical Critical
    • other
    • None
    • Platform: All, OS: All

      Seems that the usual way how jobs work on hudson is that there is one job for
      building a project and then if I want to have a subsequent job to run all unit
      tests, that job needs to download artifacts produced by that build job from
      Hudson. As a consequence, people usually add separate targets to their build
      scripts for running unit tests from hudson which download the build artifacts
      from the network.
      I think it would be nice if things could work the way how users would do it when
      running tests locally - i.e. use usual targets for building and then simply run
      a target for unit tests in the same workspace. This could be achieved by
      providing the subsequent jobs an exact snapshot of the workspace at the time
      when the parent job finished running.

          [JENKINS-682] Clone workspace between jobs

          Andrew Bayer added a comment -

          The challenge with that model is the slave situation - since each slave has their own workspace area, how do we have several jobs using the same workspace across multiple slaves? I'm by no means averse to the kind of thing you're talking about, I'm just not sure at all how to do it within Hudson.

          Andrew Bayer added a comment - The challenge with that model is the slave situation - since each slave has their own workspace area, how do we have several jobs using the same workspace across multiple slaves? I'm by no means averse to the kind of thing you're talking about, I'm just not sure at all how to do it within Hudson.

          mdonohue added a comment -

          @aheritier

          I believe what you describe is a distinct enhancement request from JENKINS-682 (this issue)
          Careful reading of the original description and the early comments indicates a desire to automatically clone the entire workspace to downstream jobs. Sharing a single workspace doesn't yet have an issue filed, that I'm aware of.

          mdonohue added a comment - @aheritier I believe what you describe is a distinct enhancement request from JENKINS-682 (this issue) Careful reading of the original description and the early comments indicates a desire to automatically clone the entire workspace to downstream jobs. Sharing a single workspace doesn't yet have an issue filed, that I'm aware of.

          Romain Seguy added a comment -

          In my company we split jobs in several task-oriented jobs such as 01-SCM, 02-Build, 03-UnitTesting, 04-Deploy, 05-RegressionTesting, etc.
          Each of these jobs highly depends on the result of the previous one (+ generally the SCM one), that is, on the content of the workspace of other jobs. We currently achieve this by doing some shell scripting, taking a look at the workspace path to see if we're running on a slave or the master, to copy files from one job to another. This means all the jobs must be assigned to the same node (or otherwise you have to use the Copy To Slave plugin)

          My idea some months ago was to create some kind of $WORKSPACE[<job name>] variable which would ease the pointing to upstream/downstream job workspaces. This does not address the issue regarding "one commonly shared" workspace, nor the node assignement one, but I think it can be a first easy & useful solution to set-up.

          We can also think about a new out-of-the-box build step (or a wrapper, etc.) "Grab from other job", with job name and includes/excludes (Ant style).

          Romain Seguy added a comment - In my company we split jobs in several task-oriented jobs such as 01-SCM, 02-Build, 03-UnitTesting, 04-Deploy, 05-RegressionTesting, etc. Each of these jobs highly depends on the result of the previous one (+ generally the SCM one), that is, on the content of the workspace of other jobs. We currently achieve this by doing some shell scripting, taking a look at the workspace path to see if we're running on a slave or the master, to copy files from one job to another. This means all the jobs must be assigned to the same node (or otherwise you have to use the Copy To Slave plugin) My idea some months ago was to create some kind of $WORKSPACE [<job name>] variable which would ease the pointing to upstream/downstream job workspaces. This does not address the issue regarding "one commonly shared" workspace, nor the node assignement one, but I think it can be a first easy & useful solution to set-up. We can also think about a new out-of-the-box build step (or a wrapper, etc.) "Grab from other job", with job name and includes/excludes (Ant style).

          mmatula added a comment - - edited

          @mdonohue

          I did mean sharing the workspace when I filed this issue. Whether it is done by ensuring these related jobs run on the same slave or by cloning the exact state of the workspace to other slave that runs the subsequent job is an implementation detail.

          Or maybe not - I see the difference is that in case of cloning the workspace for the previous job would not be affected by the subsequent one - which may be desirable - but not for the use-case I had in mind. When filing this issue I was mostly concerned about not being able to easily share the workspace of a job with subsequent jobs so they could use the results of the previous job without doing additional hudson-specific magic in my build scripts.

          mmatula added a comment - - edited @mdonohue I did mean sharing the workspace when I filed this issue. Whether it is done by ensuring these related jobs run on the same slave or by cloning the exact state of the workspace to other slave that runs the subsequent job is an implementation detail. Or maybe not - I see the difference is that in case of cloning the workspace for the previous job would not be affected by the subsequent one - which may be desirable - but not for the use-case I had in mind. When filing this issue I was mostly concerned about not being able to easily share the workspace of a job with subsequent jobs so they could use the results of the previous job without doing additional hudson-specific magic in my build scripts.

          Andrew Bayer added a comment -

          @mmatula - does the approach I proposed (archive the workspace - or a selected subset thereof - from one job's build and then one or more other jobs taking that archive and exploding it as the basis of their workspace) fit your request? I want to be sure before I really dive into coding this up. =)

          Andrew Bayer added a comment - @mmatula - does the approach I proposed (archive the workspace - or a selected subset thereof - from one job's build and then one or more other jobs taking that archive and exploding it as the basis of their workspace) fit your request? I want to be sure before I really dive into coding this up. =)

          mmatula added a comment -

          @abayer - Yes, that looks good . Btw., re changelog - I guess it should just inherit the parent job's changelog.

          mmatula added a comment - @abayer - Yes, that looks good . Btw., re changelog - I guess it should just inherit the parent job's changelog.

          Andrew Bayer added a comment -

          FYI, I've got this working (more or less - still a couple things to test and docs/tests to write) - it's up at http://github.com/abayer/hudson-clone-workspace-scm-plugin, though it'll need 1.350 to be released before I can release it.

          Andrew Bayer added a comment - FYI, I've got this working (more or less - still a couple things to test and docs/tests to write) - it's up at http://github.com/abayer/hudson-clone-workspace-scm-plugin , though it'll need 1.350 to be released before I can release it.

          Andrew Bayer added a comment -

          I've released the clone-workspace-scm plugin, which contains both a publisher for archiving workspaces and an SCM for, well, using those archived workspaces as SCM sources. It does need Hudson 1.350 or later - it'll blow up with earlier versions. I've got a wiki page up at http://wiki.jenkins-ci.org/display/JENKINS/Clone+Workspace+SCM+Plugin, but I need to do more work on the documentation. And as I mentioned before, the actual source is at GitHub (http://github.com/abayer/hudson-clone-workspace-scm-plugin) rather than the Hudson SVN repo.

          The plugin should show up in the Update Center within the next 6-12 hours.

          Andrew Bayer added a comment - I've released the clone-workspace-scm plugin, which contains both a publisher for archiving workspaces and an SCM for, well, using those archived workspaces as SCM sources. It does need Hudson 1.350 or later - it'll blow up with earlier versions. I've got a wiki page up at http://wiki.jenkins-ci.org/display/JENKINS/Clone+Workspace+SCM+Plugin , but I need to do more work on the documentation. And as I mentioned before, the actual source is at GitHub ( http://github.com/abayer/hudson-clone-workspace-scm-plugin ) rather than the Hudson SVN repo. The plugin should show up in the Update Center within the next 6-12 hours.

          erwan_q added a comment -

          This plugin is compatible with the matrix projects? If not it is possible?

          erwan_q added a comment - This plugin is compatible with the matrix projects? If not it is possible?

          Andrew Bayer added a comment -

          Not sure what you mean - matrix projects should be able to use the SCM part of it, at the very least. I'm not 100% sure how the publisher/archiver would work with a matrix project, though.

          Andrew Bayer added a comment - Not sure what you mean - matrix projects should be able to use the SCM part of it, at the very least. I'm not 100% sure how the publisher/archiver would work with a matrix project, though.

            abayer Andrew Bayer
            mmatula mmatula
            Votes:
            42 Vote for this issue
            Watchers:
            30 Start watching this issue

              Created:
              Updated:
              Resolved: