Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-26052

Fork without join

    XMLWordPrintable

Details

    Description

      Some flows have a need to start some long-running branches where we do not really care about waiting for termination. They might be running informational tests that would not fail the main job, or sending notifications, etc. Using parallel for these is awkward because you have to put the entire remainder of the flow in another branch, which does not nest nicely, etc.

      Suggest an alternate fork {...} which runs some stuff in the background. Could return a Future-like object you could decide to wait on later if you wished. Possibly needed parameters:

      • A label, to be used as the thread name; perhaps optional, with some generated default.
      • A daemon flag. If off, there would be an implicit join at the end of the flow. If on, if the flow reaches the end and the thread is still running, it would be stop-ped (but we probably still need to wait for it to exit cleanly).

      There should be a bounding box for the fork that runs inside so that the exit from the block is contingent on every fork inside to terminate.

      Potentially this could allow parallel to be implemented as a library.

      Attachments

        Issue Links

          is duplicated by

          New Feature - A new feature of the product, which has yet to be developed. JENKINS-36098 General purpose fire-and-forget command "wrapper" for Pipeline

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          New Feature - A new feature of the product, which has yet to be developed. JENKINS-27127 wait/notify steps

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. JENKINS-26135 User-defined functions in git-global-lib

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. JENKINS-26034 Option to interrupt remaining branches when one branch fails

          • Major - Major loss of function.
          • Closed

          Activity

            Descending order - Click to sort in ascending order
            jglick Jesse Glick added a comment -

            Potentially this could allow parallel to be implemented as a library.

            See this discussion of the Trio library for Python.

            jglick Jesse Glick added a comment - Potentially this could allow parallel to be implemented as a library. See this discussion of the Trio library for Python.
            dannychoi94 Danny Choi added a comment -

            svanoort This could still be improved on as invoking an external job with a "wait: false" flag will not stream any logs/status back to the pipeline. With my limited knowledge on CPS, I don't think streaming information to a stage in the pipeline post-completion is an easy task by any means. However, a feature like this would definitely help for my projects.

            dannychoi94 Danny Choi added a comment - svanoort This could still be improved on as invoking an external job with a "wait: false" flag will not stream any logs/status back to the pipeline. With my limited knowledge on CPS, I don't think streaming information to a stage in the pipeline post-completion is an easy task by any means. However, a feature like this would definitely help for my projects.
            svanoort Sam Van Oort added a comment -

            jglick Can we close this one or is there still a tangible need for it?  AFAICT it can be done more or less via invoking external jobs?

            svanoort Sam Van Oort added a comment - jglick Can we close this one or is there still a tangible need for it?  AFAICT it can be done more or less via invoking external jobs?
            jglick Jesse Glick added a comment -

            Not really. There would be a bounding box, so all “threads” would still be joined at a predictable time. You would just get some more flexibility in expressing when to start each thread. Nothing you cannot do already, just a more convenient form for certain use cases.

            jglick Jesse Glick added a comment - Not really. There would be a bounding box, so all “threads” would still be joined at a predictable time. You would just get some more flexibility in expressing when to start each thread. Nothing you cannot do already, just a more convenient form for certain use cases.
            deepchip Martin d'Anjou added a comment - - edited

            Are you asking for a fork-join_none construct similar to the one in SystemVerilog?

            deepchip Martin d'Anjou added a comment - - edited Are you asking for a fork-join_none construct similar to the one in SystemVerilog ?
            jglick Jesse Glick added a comment -

            I do not think kohsuke is working on this currently. Pull requests are welcome in general but I would expect this to be one of the hardest things to implement in all of Workflow. You would need to understand in depth how the CPS engine works.

            jglick Jesse Glick added a comment - I do not think kohsuke is working on this currently. Pull requests are welcome in general but I would expect this to be one of the hardest things to implement in all of Workflow. You would need to understand in depth how the CPS engine works.
            akbertram Alexander Bertram added a comment -

            Is this something you're working on currently or could I submit a pull request for this?

            Is this something that could be added to the 'support' module?

            akbertram Alexander Bertram added a comment - Is this something you're working on currently or could I submit a pull request for this? Is this something that could be added to the 'support' module?

            People

              kohsuke Kohsuke Kawaguchi
              jglick Jesse Glick
              Votes:
              7 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

                Created:
                Updated: