Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-28032

Included Regions do not work when monitoring all branches

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • git-plugin
    • Jenkins 1.608, Git plugin 2.3.5

      Set up a job to monitor all branches ('**' in 'Branches to build') but restrict it to one folder ('Project1/.*' in 'Included Regions').

      As long as you make submissions to one branch, the included regions works fine.
      However, when you make a change on a different branch, the first job that triggers will ignore included regions and pick up changes from any location. After that the next jobs on the same branch will follow the restrictions correctly until a change on another branch is made.

      Here's part of my config.xml:

      <scm class="hudson.plugins.git.GitSCM" plugin="git@2.3.5">
      <configVersion>2</configVersion>
      <userRemoteConfigs>
      <hudson.plugins.git.UserRemoteConfig>
      <url>git@github.com:SomeAccount/1.git</url>
      <credentialsId>ff33d8db-ec3a-48d2-83cd-6d5cbd8f51a4</credentialsId>
      </hudson.plugins.git.UserRemoteConfig>
      </userRemoteConfigs>
      <branches>
      <hudson.plugins.git.BranchSpec>
      <name>**</name>
      </hudson.plugins.git.BranchSpec>
      </branches>
      <doGenerateSubmoduleConfigurations>false</doGenerateSubmoduleConfigurations>
      <submoduleCfg class="list"/>
      <extensions>
      <hudson.plugins.git.extensions.impl.PathRestriction>
      <includedRegions>Project1/.*</includedRegions>
      <excludedRegions></excludedRegions>
      </hudson.plugins.git.extensions.impl.PathRestriction>
      </extensions>
      </scm>

      Here's a part of the 'GitHub Hook log':
      Seen 51 remote branches
      > /usr/bin/git log --full-history --no-abbrev --format=raw -M -m --raw [edited change A] ..[edited change B] # timeout=10
      Ignored commit [edited change B]: Found only excluded paths:
      Done. Took 1.2 sec
      Changes found

          [JENKINS-28032] Included Regions do not work when monitoring all branches

          The issue appears to boil down to the way changelist is being gathered via the git log command. That is happening in 'showRevision' method (CliGitAPIImpl.java):

          git log --full-history --no-abbrev --format=raw -M -m --raw <oldSHA>..<newSha>

          When the SHAs come from different branches, the result has more entries than expected, so 'isRevExcluded' method in GiSCM.java will check changes that were already built before.

          Jacek Sniecikowski added a comment - The issue appears to boil down to the way changelist is being gathered via the git log command. That is happening in 'showRevision' method (CliGitAPIImpl.java): git log --full-history --no-abbrev --format=raw -M -m --raw <oldSHA>..<newSha> When the SHAs come from different branches, the result has more entries than expected, so 'isRevExcluded' method in GiSCM.java will check changes that were already built before.

          Mark Waite added a comment -

          I think that hints at a fundamental difference between your use model and the way the plugin is coded in that area. I think you want to compare against some prior point that is not the preceding build, while the plugin wants to compare against the results of the preceding build.

          Can you explain further how you would envision the plugin should decide the base SHA1 for the comparison?

          If the job changes freely from one unrelated branch to another, how would it assemble the list of files which changed?

          Mark Waite added a comment - I think that hints at a fundamental difference between your use model and the way the plugin is coded in that area. I think you want to compare against some prior point that is not the preceding build, while the plugin wants to compare against the results of the preceding build. Can you explain further how you would envision the plugin should decide the base SHA1 for the comparison? If the job changes freely from one unrelated branch to another, how would it assemble the list of files which changed?

          Thanks for looking into this, Mark.

          My use case seems simple on the surface. These are the basic requirements:

          • One single repo with all projects in different folders (ProjectA, ProjectB, etc.)
          • Jenkins has a universal jobs called 'ProjectA', 'ProjectB' and so on, that monitor all branches. This way I can have devs validate their changes and validate anything going into a release branch using one job.
          • At the same time, when polling Git, check for changes only under 'ProjectA' folder, so I don't trigger 'ProjectA' job for changes under 'ProjectB' folder.

          Let me know if you have an idea on how to deal with this setup.

          Jacek Sniecikowski added a comment - Thanks for looking into this, Mark. My use case seems simple on the surface. These are the basic requirements: One single repo with all projects in different folders (ProjectA, ProjectB, etc.) Jenkins has a universal jobs called 'ProjectA', 'ProjectB' and so on, that monitor all branches. This way I can have devs validate their changes and validate anything going into a release branch using one job. At the same time, when polling Git, check for changes only under 'ProjectA' folder, so I don't trigger 'ProjectA' job for changes under 'ProjectB' folder. Let me know if you have an idea on how to deal with this setup.

          Mark Waite added a comment -

          When you say "monitor all branches", the plugin thinks that means "if a previously unbuilt SHA1 is detected on any branch, then compute the differences between the most recent build and that unbuilt SHA1 and pass those differences through the inclusion and exclusion filters".

          There is an option in the plugin to make comparisons to a specific named branch, rather than to the last build, but I'm not sure if even that would meet your needs.

          You might try a different experiment, using the multi-branch plugin to automatically create and delete jobs for branches as they are created and deleted. I'm not sure that plugin allows inclusion / exclusion, but it is probably worth an experiment.

          Mark Waite added a comment - When you say "monitor all branches", the plugin thinks that means "if a previously unbuilt SHA1 is detected on any branch, then compute the differences between the most recent build and that unbuilt SHA1 and pass those differences through the inclusion and exclusion filters". There is an option in the plugin to make comparisons to a specific named branch, rather than to the last build, but I'm not sure if even that would meet your needs. You might try a different experiment, using the multi-branch plugin to automatically create and delete jobs for branches as they are created and deleted. I'm not sure that plugin allows inclusion / exclusion, but it is probably worth an experiment.

          Thanks for the suggestions.
          I ended up setting up a job that runs before compile and is in charge of monitoring Git on all branches.
          Then I use shell script to check if any of the committed files match the job's folder and if it does, I trigger the compile job.

          A little complicated and probably not easy to maintain but it's the closest I've gotten to solving the problem.

          Jacek Sniecikowski added a comment - Thanks for the suggestions. I ended up setting up a job that runs before compile and is in charge of monitoring Git on all branches. Then I use shell script to check if any of the committed files match the job's folder and if it does, I trigger the compile job. A little complicated and probably not easy to maintain but it's the closest I've gotten to solving the problem.

          There's one more thing I noticed...
          Even though include paths and poll all branches don't seem to work as expected together, Jenkins' change log is still correctly populated.

          For example.
          1. Make change in a valid path on branch A
          2. Make change in invalid path on branch B
          3. Git polling will trigger the build even though the change is outside of the include path
          4. The job's changelog will correctly show only one change.

          Given this, is there a way we could change the polling to work like change log gathering?

          Jacek Sniecikowski added a comment - There's one more thing I noticed... Even though include paths and poll all branches don't seem to work as expected together, Jenkins' change log is still correctly populated. For example. 1. Make change in a valid path on branch A 2. Make change in invalid path on branch B 3. Git polling will trigger the build even though the change is outside of the include path 4. The job's changelog will correctly show only one change. Given this, is there a way we could change the polling to work like change log gathering?

          Mark Waite added a comment -

          jsniecikowski I think that when you ask "is there a way we could change the polling to work like change log gathering?", you're really asking "can JENKINS-28032 be fixed?" Yes, it is a matter of someone writing the tests which show the failure, then writing the code which fixes the failure, and confirming that the fixes for the failure did not cause any unexpected side effects or harm to existing use models.

          Mark Waite added a comment - jsniecikowski I think that when you ask "is there a way we could change the polling to work like change log gathering?", you're really asking "can JENKINS-28032 be fixed?" Yes, it is a matter of someone writing the tests which show the failure, then writing the code which fixes the failure, and confirming that the fixes for the failure did not cause any unexpected side effects or harm to existing use models.

            Unassigned Unassigned
            jsniecikowski Jacek Sniecikowski
            Votes:
            3 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated: