• Pipeline - July/August

      This ticket is closed – please read carefully

      Hi everyone,

      I am aware that this ticket is valued by many people. However, on review there are too many disparate use cases and problems attached to this issue that are already described elsewhere.

      This issue means too many things for too many people and makes it unsatisfiable with a single change. The appropriate action here is to close this ticket and ask watchers to watch and vote for a specific ticket that will solve their problem.

      If you are interested in these problems, please watch and vote for the following:

      • JENKINS-39203 - The entire pipeline gets marked as unstable rather than the stage or parallel where the unstable status was set.
      • JENKINS-45579 - Capability to mark a stage as "failed" with message and have the pipeline continue executing.
      • JENKINS-43995 - Infrastructure to allow any step to have its own distinct status. This enables the tickets listed above.
      • JENKINS-46170 - Add notes to stages and parallels

      I may have missed a few use cases. If that is the case:

      • Search JIRA for an issue matching your use case. If you find one, link it in the comments and I will update the ticket. Be sure to leave a comment on the linked ticket describing the problem you are trying to solve and then your desired solution to the problem.
      • If you cannot find an existing issue, raise a new one and comment here. Leave a comment on your new ticket describing problem you are trying to solve and then your desired solution to the problem.

      Thanks,
      James Dumay

      Original request
      It is often useful to provide more information up-front than a simple red or green "this step passed" or "this step failed". Our current in-house CI software has a "summary" field in its "Running Steps" equivalent, which can be filled with other info when a step finishes. For example, we have a step "upload-to-storage" which can optionally replace a "/latest/" symlink to the output it just uploaded. Either replacing or not replacing the symlink is correct, so we populate our "summary" field with the decision taken.

      I've attached an example from our current CI system - note that by allowing a step to populate its own summary, it means all pertinent information is available immediately to developers, on one page, rather than needing to always click through to log output for high-level information.

          [JENKINS-26522] Annotated block/stage status

          Jo Shields created issue -
          Jesse Glick made changes -
          Link New: This issue is related to JENKINS-26107 [ JENKINS-26107 ]
          Jesse Glick made changes -
          Link New: This issue is related to JENKINS-26523 [ JENKINS-26523 ]

          Jesse Glick added a comment -

          JENKINS-26107 suggests a label for a block, but a (generally static) label set at the start, whereas this is mainly about adding summary information at the end. So that would probably have to be a different step.

          JENKINS-26523 notes that the ability to set Run.result may be useful.

          Jesse Glick added a comment - JENKINS-26107 suggests a label for a block, but a (generally static) label set at the start , whereas this is mainly about adding summary information at the end . So that would probably have to be a different step. JENKINS-26523 notes that the ability to set Run.result may be useful.
          Jesse Glick made changes -
          Issue Type Original: Bug [ 1 ] New: New Feature [ 2 ]
          Jesse Glick made changes -
          Summary Original: Annotated block results New: Annotated block/stage status
          Jesse Glick made changes -
          Link New: This issue is related to JENKINS-27395 [ JENKINS-27395 ]
          Jesse Glick made changes -
          Link New: This issue is related to JENKINS-29188 [ JENKINS-29188 ]

          Jesse Glick added a comment -

          A related request was for a stage to be marked with a custom icon, to indicate what kind of stage this way, or to add some key/value pairs such as name of environment we deployed to.

          Jesse Glick added a comment - A related request was for a stage to be marked with a custom icon, to indicate what kind of stage this way, or to add some key/value pairs such as name of environment we deployed to.

          Jo Shields added a comment - - edited

          Okay, to try & better explain what we're trying to achieve, here's our current implementation, which is split into three parts:

          Custom logic in the job shell script to annotate results

          roughly speaking, this looks like:

          STEP="System.Security-4.5"
          STARTTIME=`date +%s`
          echo "*** start: ${STEP}"
          ${TIMEOUTCMD} --signal=ABRT --kill-after=60s 5m make -w -C mcs/class/System.Security run-test TEST_HARNESS_FLAGS=-labels PROFILE=net_4_x && echo -e "*** end($(echo $(date +%s) - ${STARTTIME} | bc)): ${STEP}: \\e[42mPassed\e[0m" || echo -e "*** end($(echo $(date +%s) - ${STARTTIME} | bc)): ${STEP}: \\e[43mUnstable\e[0m"

          This assigns blocks of tests (roughly equivalent to Pipeline stages) to run together, with a common timeout, and either a "pass" status or an "unstable" status (unstable indicating this block had a non-zero exit status, but that execution of the script should continue). For steps where failure should be deemed critical, the colour code is red instead of yellow, status text says "failed", and there's some extra exit 1's thrown in.

          Groovy script to parse the build log and generate a job status table

          Pretty much just builds an HTML <table>, pairing "*** start" and "*** end" lines to build the table structure and write the execution time, and adding the most recent test runner output as an at-a-glance summary

          Aggregate the data from multiple jobs in a giant table, with a custom plugin

          this gives the most dense data view we can manage - 76 steps, 5 architectures, with an at-a-glance look for developers of "did I break tests grouped into X on architecture Y", or "is builder Z misbehaving and in need of attention"

          I would love to move as much of this crap into a Jenkinsfile as possible. Certainly parts 1 and 2 of the above seem theoretically feasible to me, but there are gaps that means I'd just end up emulating bash really badly in Groovy, rather than taking advantage of Pipeline features (because the Pipeline versions are missing things)

          Jo Shields added a comment - - edited Okay, to try & better explain what we're trying to achieve, here's our current implementation, which is split into three parts: Custom logic in the job shell script to annotate results roughly speaking, this looks like: STEP="System.Security-4.5" STARTTIME=`date +%s` echo "*** start: ${STEP}" ${TIMEOUTCMD} --signal=ABRT --kill-after=60s 5m make -w -C mcs/class/System.Security run-test TEST_HARNESS_FLAGS=-labels PROFILE=net_4_x && echo -e "*** end($(echo $(date +%s) - ${STARTTIME} | bc)): ${STEP}: \\e[42mPassed\e[0m" || echo -e "*** end($(echo $(date +%s) - ${STARTTIME} | bc)): ${STEP}: \\e[43mUnstable\e[0m" This assigns blocks of tests (roughly equivalent to Pipeline stages) to run together, with a common timeout, and either a "pass" status or an "unstable" status (unstable indicating this block had a non-zero exit status, but that execution of the script should continue). For steps where failure should be deemed critical, the colour code is red instead of yellow, status text says "failed", and there's some extra exit 1 's thrown in. Groovy script to parse the build log and generate a job status table Pretty much just builds an HTML <table>, pairing "*** start" and "*** end" lines to build the table structure and write the execution time, and adding the most recent test runner output as an at-a-glance summary Aggregate the data from multiple jobs in a giant table, with a custom plugin this gives the most dense data view we can manage - 76 steps, 5 architectures, with an at-a-glance look for developers of "did I break tests grouped into X on architecture Y", or "is builder Z misbehaving and in need of attention" I would love to move as much of this crap into a Jenkinsfile as possible. Certainly parts 1 and 2 of the above seem theoretically feasible to me, but there are gaps that means I'd just end up emulating bash really badly in Groovy, rather than taking advantage of Pipeline features (because the Pipeline versions are missing things)
          Jo Shields made changes -
          Attachment New: Screenshot from 2016-04-05 14-30-24.png [ 32360 ]
          Attachment New: Screenshot from 2016-04-05 14-28-42.png [ 32361 ]

            Unassigned Unassigned
            directhex Jo Shields
            Votes:
            115 Vote for this issue
            Watchers:
            167 Start watching this issue

              Created:
              Updated:
              Resolved: