Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-23606

Git Polling Jobs with Included Regions being triggered unnecessarily with merge commits

      We have a single Git repository with several projects in a directory structure. I have created build jobs with polling enabled for each project and have included the appropriate directory structure in the "Included Regions" option in the git plugin setup. I have also turned on "Force Polling using Workspace" as indicated in https://issues.jenkins-ci.org/browse/JENKINS-20569.

      We are using a git-flow type development process, and the jobs are all configured to watch the '*/dev' branch. All commits made to this branch are merges. It appears that whenever a merge commit is made, it will trigger all of the jobs, regardless of if the merge only contained changes to one of the projects' directories.

      I looked into the polling log and it looks like the way that the plugin is getting the commit log and checking the paths may be causing the issue. Here is the command that the job runs in the polling log:

      > /usr/bin/git log --full-history --no-abbrev --format=raw -M -m --raw ff0ec94a066cee23c13faa16b38b247911cd5ee7..f47993565f701a4f28174c4ea81f35d0e1103590

      If I run this command manually, I can see that the returned paths for each commit also includes paths from other commits. I'm not sure why this is, but from the 'git' documentation, it looks like the '-m' parameter will do a full diff on both parents of a merge commit. If I leave off the '-m' parameter, or if I include '--first-parent', the list of paths in the commits only shows the actual affected files made by the commit which triggered the job. Perhaps this option needs to be included in the job config?

          [JENKINS-23606] Git Polling Jobs with Included Regions being triggered unnecessarily with merge commits

          lledhub dell added a comment -

          Same issue here, in a setting in which GitFlow is used. The Jenkins job is configured to poll the developer branch: Polling ignores commits in certain paths - 'Included Regions' - /sources/.*
          Two feature branches are made:
          1. commit to a file in /sources/
          2. commit to a file in /documentation/
          For both feature branches, when being merged back to developer, the job is triggered. This should only be one build trigger, i.e. only the commit+merge that affects files in the sources directory.

          Git client plugin
          1.21.0

          Git plugin
          2.5.3

          Jenkins
          1.651.1

          lledhub dell added a comment - Same issue here, in a setting in which GitFlow is used. The Jenkins job is configured to poll the developer branch: Polling ignores commits in certain paths - 'Included Regions' - /sources/.* Two feature branches are made: 1. commit to a file in /sources/ 2. commit to a file in /documentation/ For both feature branches, when being merged back to developer, the job is triggered. This should only be one build trigger, i.e. only the commit+merge that affects files in the sources directory. Git client plugin 1.21.0 Git plugin 2.5.3 Jenkins 1.651.1

          Kevin Reed added a comment -

          Same issue, this is a big deal for large multi app repos

          Kevin Reed added a comment - Same issue, this is a big deal for large multi app repos

          Mark Waite added a comment - - edited

          I've placed a series of verification jobs into my lts-with-plugins docker image that duplicates the problem. I've resolved the PR162 conflicts in and pushed it as a branch in my development repository.

          The changes in PR162 are the git client plugin changes. The necessary changes in the git plugin have not been made. Changes in both will be required. The new API in git client will need to be called from git plugin (just as elsabbahyahmed said in his earlier comment).

          Mark Waite added a comment - - edited I've placed a series of verification jobs into my lts-with-plugins docker image that duplicates the problem. I've resolved the PR162 conflicts in and pushed it as a branch in my development repository . The changes in PR162 are the git client plugin changes. The necessary changes in the git plugin have not been made. Changes in both will be required. The new API in git client will need to be called from git plugin (just as elsabbahyahmed said in his earlier comment).

          Phil Johnson added a comment -

           

          Phil Johnson added a comment -  

          Tomasz M added a comment -

          We have the same problem. Our temporary workaround is to set "Merge strategies" to "Rebase and merge" on Bitbucket Server. After rebase current git log command seems to work properly.

           

          Maybe, as a simple fix, you could allow to customize git log command in configuration?

           

          Tomasz M added a comment - We have the same problem. Our temporary workaround is to set "Merge strategies" to "Rebase and merge" on Bitbucket Server. After rebase current git log command seems to work properly.   Maybe, as a simple fix, you could allow to customize git log command in configuration?  

          Karl Shultz added a comment -

          tmar (and others) - here's a question about the issues you're experiencing with included and excluded regions. Are you, by chance, starting your included regions with a leading /, as lledhub alluded to in this comment from September 9, 2016?

          I've written eight new tests that look at included/excluded areas specifically with merge commits. It took me a really, really long time to see it, but eventually I realized that the last failing test was failing because it had committed directory/filename to the git repository, but the included/excluded region pattern was looking for /directory/filename. The difference was that leading slash.

          Karl Shultz added a comment - tmar (and others) - here's a question about the issues you're experiencing with included and excluded regions. Are you, by chance, starting your included regions with a leading / , as lledhub alluded to in this comment from September 9, 2016? I've written eight new tests that look at included/excluded areas specifically with merge commits. It took me a really, really long time to see it, but eventually I realized that the last failing test was failing because it had committed directory/filename to the git repository, but the included/excluded region pattern was looking for /directory/filename . The difference was that leading slash.

          Karl Shultz added a comment - - edited

          The eight new tests have been merged to master. They include, in the form of "top level directory of the repo" and "inside a subdirectory" tests, the following:

          • Merge commits inside of excluded regions get ignored (file, directory)
          • Merge commits inside of included regions get processed as new changes (file, directory)
          • Merge commits outside of included regions get ignored (file, directory)
          • Merge commits outside of excluded regions get processed (file, directory)

          Karl Shultz added a comment - - edited The eight new tests have been merged to master . They include, in the form of "top level directory of the repo" and "inside a subdirectory" tests, the following: Merge commits inside of excluded regions get ignored ( file , directory ) Merge commits inside of included regions get processed as new changes ( file , directory ) Merge commits outside of included regions get ignored ( file , directory ) Merge commits outside of excluded regions get processed ( file , directory )

          Tomasz M added a comment -

          kshultz - here are the settings we use:

          Branch Specifier (blank for 'any') :origin/develop
          Included Regions ^report-service/.*
          Excluded Regions [empty]  

          Tomasz M added a comment - kshultz - here are the settings we use: Branch Specifier (blank for 'any') :origin/develop Included Regions ^report-service/.* Excluded Regions [empty]  

          We have the same issue: https://issues.jenkins.io/browse/JENKINS-65932
          Indeed the command with '-m'
          ```
          git log --full-history --no-abbrev --format=raw --raw -M -m 9aa631d36264e7d21e60881265572ec099ebc2b1..4fa8bc728c0a83d8defd2fb8a786505c1fbca7be
          ```
          contains invalid commit history included foreign changes from merge request

          How to disable '-m' flag in polling?

          Anatoly Shirokov added a comment - We have the same issue: https://issues.jenkins.io/browse/JENKINS-65932 Indeed the command with '-m' ``` git log --full-history --no-abbrev --format=raw --raw -M -m 9aa631d36264e7d21e60881265572ec099ebc2b1..4fa8bc728c0a83d8defd2fb8a786505c1fbca7be ``` contains invalid commit history included foreign changes from merge request How to disable '-m' flag in polling?

          Anatoly Shirokov added a comment - It seems to me here is fix https://github.com/jenkinsci/git-client-plugin/pull/750

            Unassigned Unassigned
            tduskin Trey Duskin
            Votes:
            29 Vote for this issue
            Watchers:
            32 Start watching this issue

              Created:
              Updated: