Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-42809

Activity/Branch dashboard show pipeline library commit instead of actual built commit

    XMLWordPrintable

    Details

    • Similar Issues:
    • Epic Link:
    • Sprint:
      Blue Ocean - 1.1-beta-1, Blue Ocean - 1.1-beta2

      Description

      Notes
      We don't want to be showing the commits from a shared library in this case. We need to do some testing here and find out why shared library changes are coming through and not the application code. We suspect that we pick the first grouping of change sets and this happens to be the changeset of the shared library in this scenario.

       

      Expected behavior: 

      Show only the latest commit from the repository that the pipeline originates from (ie where the Jenkinsfile is changed) that was triggering the pipeline. 

      Any other commits are to be ignored as part of this. They will show up in the design shown in: https://issues.jenkins-ci.org/browse/JENKINS-39860

       

      Original request
      Heyo!

      I'm using pipelines for a while now and have most of the steps used abstracted away into a pipeline library in a git repository. I now installed blue ocean today and was scrolling through my builds which had all suprisingly the same commit shown:

       

      Turns out this is actually the latest commit of the pipeline library I'm using as seen in the console log:

       

      Obtained Jenkinsfile from ccf54369437ff7dcd66b888fde50b19bad7ecf23
      
      Loading library mylib@master
      > git rev-parse --is-inside-work-tree # timeout=10
      Setting origin to /mnt/devops
      > git config remote.origin.url /mnt/devops # timeout=10
      Fetching origin...
      Fetching upstream changes from origin
      > git --version # timeout=10
      > git fetch --tags --progress origin +refs/heads/:refs/remotes/origin/
      > git rev-parse master^{commit} # timeout=10
      > git rev-parse origin/master^{commit} # timeout=10
      > git rev-parse --is-inside-work-tree # timeout=10
      Fetching changes from the remote Git repository
      > git config remote.origin.url /mnt/devops # timeout=10
      Fetching upstream changes from /mnt/devops
      > git --version # timeout=10 > git fetch --tags --progress /mnt/devops +refs/heads/:refs/remotes/origin/
      Checking out Revision 76cca6a9021830b3850eab338050c1d839d7b318 (master) > git config core.sparsecheckout # timeout=10
      > git checkout -f 76cca6a9021830b3850eab338050c1d839d7b318
      > git rev-list 76cca6a9021830b3850eab338050c1d839d7b318 # timeout=10
      
      [Pipeline] node Running on master in /var/jenkins_home/workspace/PR-63-WGSBDQPE6N2TEWS2C25CRCFXVLMNQURTXNY3KELAD4YZHGLFQIPA
      [Pipeline] {
      [Pipeline] checkout
      > git rev-parse --is-inside-work-tree # timeout=10
      Fetching changes from 2 remote Git repositories
      > git config remote.origin.url https://github.com/sneils/some-repo.git # timeout=10
      Fetching upstream changes from https://github.com/sneils/some-repo.git
      > git --version # timeout=10 using GIT_ASKPASS to set credentials checkout
      > git fetch --tags --progress https://github.com/sneils/some-repo.git +refs/heads/:refs/remotes/origin/
      > git config remote.origin1.url https://github.com/sneils/some-repo.git # timeout=10
      Fetching upstream changes from https://github.com/sneils/some-repo.git using GIT_ASKPASS to set credentials checkout
      > git fetch --tags --progress https://github.com/sneils/some-repo.git +refs/pull//head:refs/remotes/origin/pr/
      Checking out Revision ccf54369437ff7dcd66b888fde50b19bad7ecf23 (PR-63)
      > git config core.sparsecheckout # timeout=10
      > git checkout -f ccf54369437ff7dcd66b888fde50b19bad7ecf23
      ...
      

       

      I would expect this to be the commit of the actually checked out repository.

       

        Attachments

          Issue Links

            Activity

            Hide
            michaelneale Michael Neale added a comment -

            Thanks Franz B. - if you click back to the "classic" view of the run - does it show the same thing, or 2 distinct commits? (ie depends if we can fix it in blue ocean or if we need to direct efforts somewhere else).

            Show
            michaelneale Michael Neale added a comment - Thanks Franz B. - if you click back to the "classic" view of the run - does it show the same thing, or 2 distinct commits? (ie depends if we can fix it in blue ocean or if we need to direct efforts somewhere else).
            Hide
            bwalding Ben Walding added a comment -

            Classic view lists all the Git sources for the project.

            Unfortunately the first git source is also the library repo, which is not generally as interesting as the code repo.

            Show
            bwalding Ben Walding added a comment - Classic view lists all the Git sources for the project. Unfortunately the first git source is also the library repo, which is not generally as interesting as the code repo.
            Hide
            michaelneale Michael Neale added a comment -

            So James Dumay is there a design ticket to link to this? or shoudl this be first fixed to show the "relevant" commit (ie we never really want to show the library version). However in the case of 2 source repos, we probably by rights want to show both? 

             

            cc Brody Maclean

            Show
            michaelneale Michael Neale added a comment - So James Dumay is there a design ticket to link to this? or shoudl this be first fixed to show the "relevant" commit (ie we never really want to show the library version). However in the case of 2 source repos, we probably by rights want to show both?    cc Brody Maclean
            Hide
            jamesdumay James Dumay added a comment - - edited

            Vivek Pandey can you do some snooping and see how we can differentiate checkouts of shared libraries vs code that we explicitly checked out during the pipeline?

            Some thoughts here:

            • Ideally we still want to show changes from shared libraries but they shouldn't be reported as the main revision for the pipeline
            • Need a way of differentiating for each commit what repository it came from in our REST API
            Show
            jamesdumay James Dumay added a comment - - edited Vivek Pandey can you do some snooping and see how we can differentiate checkouts of shared libraries vs code that we explicitly checked out during the pipeline? Some thoughts here: Ideally we still want to show changes from shared libraries but they shouldn't be reported as the main revision for the pipeline Need a way of differentiating for each commit what repository it came from in our REST API
            Hide
            scatt Stephen Catt added a comment -

            I am also having this issue running Jenkins 2.55, Blue Ocean 1.0.1, with Java 8 OpenJDK.

            This seems related: when I call currentBuild.changeSets.last() to get the change set for the current build, the hash for the global library is included in the change set along with the build hashes.  Since the change sets don't include repo or branch info, it's a little hard to tell what is from the build and what is from the global lib – I'm trying to use this info to get a total file change set for the build from git for folder change detection.

            However, I've determined that if the library change shows up in the changeSet, it reliably shows up as the last one in the list. Please lmk if I should file a separate bug report or if this is just expected behavior.

            Show
            scatt Stephen Catt added a comment - I am also having this issue running Jenkins 2.55, Blue Ocean 1.0.1, with Java 8 OpenJDK. This seems related: when I call currentBuild.changeSets.last() to get the change set for the current build, the hash for the global library is included in the change set along with the build hashes.  Since the change sets don't include repo or branch info, it's a little hard to tell what is from the build and what is from the global lib – I'm trying to use this info to get a total file change set for the build from git for folder change detection. However, I've determined that if the library change shows up in the changeSet, it reliably shows up as the last one in the list. Please lmk if I should file a separate bug report or if this is just expected behavior.
            Hide
            jamesdumay James Dumay added a comment -

            Stephen Catt don't bother with another ticket - we are going to do some snooping here and will update this ticket

            Jesse Glick any guidance you could provide us on this one?

            Show
            jamesdumay James Dumay added a comment - Stephen Catt don't bother with another ticket - we are going to do some snooping here and will update this ticket Jesse Glick any guidance you could provide us on this one?
            Hide
            jglick Jesse Glick added a comment -

            Well if JENKINS-41497 is implemented and used by a given job, then there is no issue. Otherwise the problem is that BO is failing to differentiate between multiple ChangeLogSet associated with the build, or I guess even display them all.

            Show
            jglick Jesse Glick added a comment - Well if JENKINS-41497 is implemented and used by a given job, then there is no issue. Otherwise the problem is that BO is failing to differentiate between multiple ChangeLogSet associated with the build, or I guess even display them all.
            Hide
            michaelneale Michael Neale added a comment -

            I am a little conflicted by this one. Simplistically speaking, we can be opinionated and say that the commit should be from the repo which the MBP was originally created from, but I know from experience that isn't always the one that I care about (in some CD scenarios) - or I might want to show all of them (but I am not sure the design with the table can cope with showing more than one)

            Show
            michaelneale Michael Neale added a comment - I am a little conflicted by this one. Simplistically speaking, we can be opinionated and say that the commit should be from the repo which the MBP was originally created from, but I know from experience that isn't always the one that I care about (in some CD scenarios) - or I might want to show all of them (but I am not sure the design with the table can cope with showing more than one)
            Hide
            bwalding Ben Walding added a comment -

            Michael Neale - perfect is the enemy of the good

            Simplistic system that will cover 90% of the cases

            I think we can "definitively" say that if there are multiple changesets, then a change in the pipeline library is not what we are primarily interested in. Even in the case where I manually trigger a build based on changing my library - I generally wouldn't really care if BO displayed the code as "not changing".

            • Use any changeset that isn't a library changeset

            Overly complex system

            If you were going to go for something a bit more generally applicable, then each changeset would be ranked.

            (Imagine I don't care about implementation details - because for the most part I don't)

            • does this change-set have changes - "+3"
            • is this changeset for the MBP Jenkinsfile - "+2"
            • is this changeset for an SCM repo referenced in the Pipeline - "+1"
            • is this changeset for a library - "+0"

            Sort descending, display the first

            This then introduces issues where the referenced commit for display may change repos for each build - which may be more confusing than going with the simplistic approach.

            Show
            bwalding Ben Walding added a comment - Michael Neale - perfect is the enemy of the good Simplistic system that will cover 90% of the cases I think we can "definitively" say that if there are multiple changesets, then a change in the pipeline library is not what we are primarily interested in. Even in the case where I manually trigger a build based on changing my library - I generally wouldn't really care if BO displayed the code as "not changing". Use any changeset that isn't a library changeset Overly complex system If you were going to go for something a bit more generally applicable, then each changeset would be ranked. (Imagine I don't care about implementation details - because for the most part I don't) does this change-set have changes - "+3" is this changeset for the MBP Jenkinsfile - "+2" is this changeset for an SCM repo referenced in the Pipeline - "+1" is this changeset for a library - "+0" Sort descending, display the first This then introduces issues where the referenced commit for display may change repos for each build - which may be more confusing than going with the simplistic approach.
            Hide
            michaelneale Michael Neale added a comment -

            Ben Walding well the problem is there isn't space to show more than one. 

            I think yeah, people rarely care about library version - I was talking about additional SCM checkouts coming in the pipeline at different points (something I have done a fair bit of - although not without regret)

            Show
            michaelneale Michael Neale added a comment - Ben Walding well the problem is there isn't space to show more than one.  I think yeah, people rarely care about library version - I was talking about additional SCM checkouts coming in the pipeline at different points (something I have done a fair bit of - although not without regret)
            Hide
            jamesdumay James Dumay added a comment -

            Michael Neale Ben Walding the work around for multiple commits is JENKINS-39860

            Show
            jamesdumay James Dumay added a comment - Michael Neale Ben Walding the work around for multiple commits is JENKINS-39860
            Hide
            michaelneale Michael Neale added a comment -

            oh I like that James Dumay

            Show
            michaelneale Michael Neale added a comment - oh I like that James Dumay
            Hide
            jamesdumay James Dumay added a comment -
            Show
            jamesdumay James Dumay added a comment - Michael Neale nice - all Brody Maclean
            Hide
            vivek Vivek Pandey added a comment -

            In blueocean we were computing commit id of current scm checkout by using BuildData action. BuildData action always points to shared library commit id. For this reason all commit ids always pointed to shared library. Hard to tell if thats right behavior reading it's Javadoc.

            However using SCMRevisionAction, we get checked out repo's commit id correctly. I tested it with multiple branches and it seems to behave well.

            I have PR opened with this change, https://github.com/jenkinsci/blueocean-plugin/pull/1010. Jesse Glick James Dumay PTAL.

            Show
            vivek Vivek Pandey added a comment - In blueocean we were computing commit id of current scm checkout by using BuildData action. BuildData action always points to shared library commit id. For this reason all commit ids always pointed to shared library. Hard to tell if thats right behavior reading it's Javadoc. However using SCMRevisionAction, we get checked out repo's commit id correctly. I tested it with multiple branches and it seems to behave well. I have PR opened with this change, https://github.com/jenkinsci/blueocean-plugin/pull/1010 . Jesse Glick James Dumay PTAL.
            Hide
            jamesdumay James Dumay added a comment -

            Will be released in Blue Ocean 1.1

            Show
            jamesdumay James Dumay added a comment - Will be released in Blue Ocean 1.1
            Hide
            lalmeras Laurent Almeras added a comment -

            Hi,

            Sorry to comment again on a closed issue, but I am fighting with a similar issue with other plugins, and I am a bit lost on how to fix it.

            I encounter this issue (SHA retrieved from pipeline's lib instead of project) on github-plugin and gitlab-plugin (last version for each one). I open the following issues :

            For each plugin, revision information lookup is done with BuildData, and is buggy ; using SCMRevisionAction fixes the bug for github-plugin, but :

            • gitlab-plugin also uses BuildData to lookup remote url, this information is not accessible in SCMRevisionAction;
            • it seems to me that « BuildData containing pipeline's lib information » 's behavior is a regression, as it used to work;
            • I also think, but I'm not familiar with Jenkins API, that it is not a good choice to push pipeline's library information into BuildData.

            Maybe previous BuildData's behavior may be restored (and custom actions should be used to store pipeline's library information). If this is not the case, is there anyone willing to point out to Jenkins API allowing to mimic old behaviors ?

            Thanks.

            Show
            lalmeras Laurent Almeras added a comment - Hi, Sorry to comment again on a closed issue, but I am fighting with a similar issue with other plugins, and I am a bit lost on how to fix it. I encounter this issue (SHA retrieved from pipeline's lib instead of project) on github-plugin and gitlab-plugin (last version for each one). I open the following issues : gitlab : https://github.com/jenkinsci/gitlab-plugin/issues/553 github : https://issues.jenkins-ci.org/browse/JENKINS-40150 For each plugin, revision information lookup is done with BuildData, and is buggy ; using SCMRevisionAction fixes the bug for github-plugin, but : gitlab-plugin also uses BuildData to lookup remote url, this information is not accessible in SCMRevisionAction; it seems to me that « BuildData containing pipeline's lib information » 's behavior is a regression, as it used to work; I also think, but I'm not familiar with Jenkins API, that it is not a good choice to push pipeline's library information into BuildData. Maybe previous BuildData's behavior may be restored (and custom actions should be used to store pipeline's library information). If this is not the case, is there anyone willing to point out to Jenkins API allowing to mimic old behaviors ? Thanks.
            Hide
            jglick Jesse Glick added a comment -

            Please keep the conversation in open issues, not here.

            Show
            jglick Jesse Glick added a comment - Please keep the conversation in open issues, not here.

              People

              Assignee:
              vivek Vivek Pandey
              Reporter:
              sneils Franz B.
              Votes:
              2 Vote for this issue
              Watchers:
              11 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: