Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-35806

Make it clear when current run broke the build

    • 1.0-japan-m9

      A common question asked when using CI is "did something change to break this build", not just who broke it (blame dark pattern) or where it was broken (which is already clear with stage graph).

      1. Calling out that this specific run is where the build broke (ie previously it was ok) may invite users to look at the change (it is something I commonly do, after looking at the failure message)
      2. For any failed view, it may be wise to have a quick link back to the last successful run.

          [JENKINS-35806] Make it clear when current run broke the build

          Michael Neale added a comment -

          greiber just to make things more fun - what about unstable build states...

          Michael Neale added a comment - greiber just to make things more fun - what about unstable build states...

          Michael Neale added a comment -

          greiber yes, hover could provide a lot of info, eg if the comments are JIRA tickets, it could have info on them. Lots of possibilities.

          I like this, but needs more thinking.

          Michael Neale added a comment - greiber yes, hover could provide a lot of info, eg if the comments are JIRA tickets, it could have info on them. Lots of possibilities. I like this, but needs more thinking.

          gus reiber added a comment -

          Unstable states are tougher, but could be handled the same way. Would just be orange or yellow. We would need to decide the replacement rules for unstable, as in, when does an unstable stage get repainted with a new run number. Would it be every time the stage is activated, every time the failing tests are different, or not until the stage is completely passed?

          ....for the more thinking, what does that mean? Do you want more drawings or just time to sit with it?

          gus reiber added a comment - Unstable states are tougher, but could be handled the same way. Would just be orange or yellow. We would need to decide the replacement rules for unstable, as in, when does an unstable stage get repainted with a new run number. Would it be every time the stage is activated, every time the failing tests are different, or not until the stage is completely passed? ....for the more thinking, what does that mean? Do you want more drawings or just time to sit with it?

          Michael Neale added a comment -

          greiber I need some time for me to sit with it, but if you have any other options for visualising when things broke/passed, that would be good to see. Is this something one would only see when things break (I think so) - or perhaps something people turn on to diagnose?

          For unstable - to me that is the same as failure, but it can be up to pipeline authors. By default unstable doesn't quit the pipeline the way a failure does, so it is possible that unstable could kind of be ignored and a deployment happen anyway (I think this is bad, but it is up to authors to opt out of that). Not sure if that makes things clearer...

          Michael Neale added a comment - greiber I need some time for me to sit with it, but if you have any other options for visualising when things broke/passed, that would be good to see. Is this something one would only see when things break (I think so) - or perhaps something people turn on to diagnose? For unstable - to me that is the same as failure, but it can be up to pipeline authors. By default unstable doesn't quit the pipeline the way a failure does, so it is possible that unstable could kind of be ignored and a deployment happen anyway (I think this is bad, but it is up to authors to opt out of that). Not sure if that makes things clearer...

          James Dumay added a comment -

          Sorry I haven't gotten to this one just yet Gus - snowed under with a few things.

          James Dumay added a comment - Sorry I haven't gotten to this one just yet Gus - snowed under with a few things.

          Michael Neale added a comment - - edited

          So my suggestions: use hover states where possible (means more room to "explain" as numbers alone take some training):

          Also, showing any upcoming ones is very nice (who wants to debug something when there is a fix in the pipe). I do like the idea of showing the build numbers of last attempted ones on each stage (but not clear if it should be success or not).

          greiber I like the idea of showing more info in the little ball things, possibly interesting when it is executing (not this ticket) so you can trace what it running where, but I think the hover things may give a lot more space to ease people into it.

          Michael Neale added a comment - - edited So my suggestions: use hover states where possible (means more room to "explain" as numbers alone take some training): Also, showing any upcoming ones is very nice (who wants to debug something when there is a fix in the pipe). I do like the idea of showing the build numbers of last attempted ones on each stage (but not clear if it should be success or not). greiber I like the idea of showing more info in the little ball things, possibly interesting when it is executing (not this ticket) so you can trace what it running where, but I think the hover things may give a lot more space to ease people into it.

          gus reiber added a comment -

          mneale I think hovers is tough here. I would expect this to be a screen that folks would want to project on a big screen somewhere in their team area. "Broken since" which is what the red stage dots would show would be super helpful information.

          Here I have added another state indication to show the stage didn't run this time (it is semi-opaque). To me, adding a gesture adds at least as much complexity and user burden as showing the combination of a number and a color and maintains the 'hands-free' use cases.

          The Nokia guys also include time to complete stage within each stage circle, as one of their uses cases was pipeline CD optimization. They wanted to see at a glance how long the total run took and which stages were slow. They were expert users and designing specifically for themselves, but I found the few things that they showed immediately on the screen to be fairly compelling, generally. They wanted 'at a glance' diagnosis so they could project the results to a group and flip quickly between pipelines.

          ...anyway, that is my caution.
          I think past stage failures is a really important bit of health information worth getting 'at a glance'. Stuffing important information into hovers, I doesn't always reduce user complexity, in fact, often it increases it. ...thus I would look for other places to trim clutter, if this feels too cluttered.

          gus reiber added a comment - mneale I think hovers is tough here. I would expect this to be a screen that folks would want to project on a big screen somewhere in their team area. "Broken since" which is what the red stage dots would show would be super helpful information. Here I have added another state indication to show the stage didn't run this time (it is semi-opaque). To me, adding a gesture adds at least as much complexity and user burden as showing the combination of a number and a color and maintains the 'hands-free' use cases. The Nokia guys also include time to complete stage within each stage circle, as one of their uses cases was pipeline CD optimization. They wanted to see at a glance how long the total run took and which stages were slow. They were expert users and designing specifically for themselves, but I found the few things that they showed immediately on the screen to be fairly compelling, generally. They wanted 'at a glance' diagnosis so they could project the results to a group and flip quickly between pipelines. ...anyway, that is my caution. I think past stage failures is a really important bit of health information worth getting 'at a glance'. Stuffing important information into hovers, I doesn't always reduce user complexity, in fact, often it increases it. ...thus I would look for other places to trim clutter, if this feels too cluttered.

          gus reiber added a comment -

          Another thing the Nokia guy talked about was wanting to see quickly when and which a CD run had been kicked off from some point other than the beginning. Other than resume, I am not sure if we allow that, but that also seems to me to be a common enough diagnostic issue.

          The Nokia guy was describing that situation as an instance of team or developer bad behavior, where to speed things along they were skipping blocks of tests. Naughty or not, his teams were using that feature pretty often. My friend here in MS games as a matter of requirement needs to jump into midpoints of his CD do to the time consuming nature of his full pipeline. Most of the time, he cannot spare the 2 or 3 days that would be required to run a full regression.

          ....this isn't directly related to showing skipped build steps in a hover or not, but this was another piece of information the Nokia guy scanned for and their stage numbering system allowed him to see quickly without touching the screen. ...so I toss that in here as well.

          gus reiber added a comment - Another thing the Nokia guy talked about was wanting to see quickly when and which a CD run had been kicked off from some point other than the beginning. Other than resume, I am not sure if we allow that, but that also seems to me to be a common enough diagnostic issue. The Nokia guy was describing that situation as an instance of team or developer bad behavior, where to speed things along they were skipping blocks of tests. Naughty or not, his teams were using that feature pretty often. My friend here in MS games as a matter of requirement needs to jump into midpoints of his CD do to the time consuming nature of his full pipeline. Most of the time, he cannot spare the 2 or 3 days that would be required to run a full regression. ....this isn't directly related to showing skipped build steps in a hover or not, but this was another piece of information the Nokia guy scanned for and their stage numbering system allowed him to see quickly without touching the screen. ...so I toss that in here as well.

          Michael Neale added a comment -

          greiber I think we could re-use this component for dashboard "large screen" like scenarious, but probably not as part of the default. As long as there is a visual indicator that things are "past" vs "now" that is interesting to me.

          There are some other screens about diagnostics (trends? jdumay) that don't belong here as directly (they can show a lot more) - this is more a dev centric screen to start with, but the components can evolve to include these other methods.

          As for skipping things... I am not sure how they would be doing that with pipeline.

          Michael Neale added a comment - greiber I think we could re-use this component for dashboard "large screen" like scenarious, but probably not as part of the default. As long as there is a visual indicator that things are "past" vs "now" that is interesting to me. There are some other screens about diagnostics (trends? jdumay ) that don't belong here as directly (they can show a lot more) - this is more a dev centric screen to start with, but the components can evolve to include these other methods. As for skipping things... I am not sure how they would be doing that with pipeline.

          Michael Neale added a comment -

          greiber I really like the idea of showing past things, but I can't see how to do it without explaining it to people, which thus far hasn't been needed for this important screen. If we know everyone uses build numbers, I think that could be good (we can't show anything else though, as it would get too crowded).

          Another option - below the stage graph there is the step listing, there is also space there to show more detailed information if we want.

          The main thing is that it isn't visually obvious what is "now" vs past, if we can make that clear (I can't quite see it in the colours on my monitor) this could be really good.

          I would also like (separate ticket) to look at this from the "hands off" perspective - in that case it won't be run specific - but that is not this feature (scope has increase enough for now).

          I think there is also a ticket to show calculated average times per stage etc (there was going to be some backend maths to do it in a more robust way than it does now) - but also, not this ticket.

          Michael Neale added a comment - greiber I really like the idea of showing past things, but I can't see how to do it without explaining it to people, which thus far hasn't been needed for this important screen. If we know everyone uses build numbers, I think that could be good (we can't show anything else though, as it would get too crowded). Another option - below the stage graph there is the step listing, there is also space there to show more detailed information if we want. The main thing is that it isn't visually obvious what is "now" vs past, if we can make that clear (I can't quite see it in the colours on my monitor) this could be really good. I would also like (separate ticket) to look at this from the "hands off" perspective - in that case it won't be run specific - but that is not this feature (scope has increase enough for now). I think there is also a ticket to show calculated average times per stage etc (there was going to be some backend maths to do it in a more robust way than it does now) - but also, not this ticket.

            Unassigned Unassigned
            jamesdumay James Dumay
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: