-
New Feature
-
Resolution: Unresolved
-
Minor
-
None
Before I explain my use case (which does not work today), I will explain a related use case (which does work today) to help illustrate my point.
Working Use Case
The working use case is as follows. We start with the naïve version:
- The user starts a Pipeline job against a branch of a fork in GitHub/GitLab.
- The job runs many tests in parallel using a parallel step: (unit tests, integration tests, and functional tests, for example). Each branch of the parallel step does a Git checkout of the user's code and then runs the relevant tests.
The above tests take a while to run, during which the user might push again to their branch on GitHub/GitLab. Our users expect that once they start a Jenkins job, the job will test their code at a specific revision. Since the implementation described above checks out the Git repository in each test branch, this constraint could be violated, as evidenced by the following sequence of events:
- The user starts a Pipeline job against a branch of a fork in GitHub/GitLab.
- The job clones their Git repository and starts the unit tests. But there is no executor available for the functional tests, so the job blocks in that branch.
- The user pushes again to the same branch of their fork in GitHub/GitLab.
- The job now enters the branch for the functional tests and clones the Git repository.
At this point, the job is testing an inconsistent set of code: the parallel branch from step 2 is testing the code before the user pushed the second commit, but the parallel branch from step 4 is testing the code after the user pushed the second commit.
Since the Git repository is a moving target that is out of our control, we clearly need to "snapshot" the state of the Git repository at the beginning of the job to avoid this scenario. So one might consider using stash at the beginning of the job and then unstash in each branch of the job to unstash the Git repository at the same exact commit before starting each set of tests. We can correct the naïve implementation with this in mind:
- The user starts a Pipeline job against a branch of a fork in GitHub/GitLab.
- The job checks out the Git repository and uses the stash step to "snapshot" it.
- The job runs many tests in parallel using a parallel step: (unit tests, integration tests, and functional tests, for example). Each branch of the parallel step runs unstash to retrieve the snapshot and then runs the relevant tests.
This solves the problem.
Now let me describe my use case, where this problem cannot be solved.
Failing Use Case
The problem comes when the tests being run in parallel are not run from the same pipeline, but rather from different pipelines altogether (invoked with the build step). There could be a variety of legitimate reasons for this: for example, to do "parallel within parallel" testing (which is not possible within a single pipeline), or to visually separate functional testing or deployment processes that are sufficiently complex to warrant their own full-fledged job. The naïve implementation of this is as follows:
- The user starts a Pipeline job against a branch of a fork in GitHub/GitLab.
- The parent job starts many child Jenkins jobs (unit tests, integration tests, and functional tests, for example), using the build step. Each child job does a Git checkout of the user's code and then runs the relevant tests.
This has the same problem as the naïve implementation of the first use case; namely, a user can push to their fork on GitHub/GitLab while the job is running and affect the consistency of the job. However, this problem cannot be solved with stash and unstash. Suppose one tries the following:
- The user starts a Pipeline job against a branch of a fork in GitHub/GitLab.
- The job checks out the Git repository and uses the stash step to "snapshot" it.
- The parent job starts many child Jenkins jobs (unit tests, integration tests, and functional tests, for example), using the build step. Each child job runs unstash to retrieve the snapshot and then runs the relevant tests.
This does not work, because the stash operation was done in the parent job. The child job does not have access to the stash and fails with ERROR: No such saved stash 'stash'.