Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-20569

Polling with excluded regions don't prevent job triggered

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • git-plugin
    • None
    • Git plugin 2.0

      I've three jobs that trigger git with the same repository and branch.
      In then Excluded Regions I add specific folders for each job.

      Example:
      project/common
      project/projectA
      project/projectB
      project/projectC

      If I made change on projectB folder, all the three job are build.

          [JENKINS-20569] Polling with excluded regions don't prevent job triggered

          Mark Waite added a comment -

          tduskin I think you're having some other issue, not what is described in this bug report. When I enable "Force polling from workspace", I see the expected exclusion behavior from the jobs I've created.

          Here are the steps I took, and the results I observed:

          1. Create a git repository with project/common/common-file, project/projectA/A-file, project/projectB/B-file, project/projectC/C-file
          2. For testing speed, I added a post-receive "hook" to the repository (see Kohsuke's "polling must die" blog post for the technique)
          3. Define a Jenkins job JENKINS-20569-projectA using that repository
          4. Add "Force polling using workspace" behavior to that project
          5. Add "Polling ignores commits in certain paths" to that project
          6. Define the exclusion as
            project/projectB/.*
            project/projectC/.*
            
          7. Commit a change to the repository outside the exclusion (like project/common/common-file), confirm projectA builds on next poll
          8. Commit a change to the repository inside the exclusions (like project/projectB/B-file), confirm projectA does not build on next poll
          9. Copy the JENKINS-20569-projectA job to JENKINS-20569-projectB, define the exclusion as
            project/projectA/.*
            project/projectC/.*
            
          10. Commit a change to the repository outside the exclusion (like project/common/common-file), confirm projectA and projectB build on next poll
          11. Commit a change to the repository inside the exclusions (like project/projectB/B-file), confirm projectB builds, and projectA does not build on next poll
          12. Commit a change to the repository inside the exclusions (like project/projectB/A-file), confirm projectB does not build, and projectA builds on next poll

          So long as I use "Force polling using workspace", and use a regular expression to match the exclusion, then the exclusion works the way I expect.

          Can you provide more details on the failure case you found?

          Mark Waite added a comment - tduskin I think you're having some other issue, not what is described in this bug report. When I enable "Force polling from workspace", I see the expected exclusion behavior from the jobs I've created. Here are the steps I took, and the results I observed: Create a git repository with project/common/common-file, project/projectA/A-file, project/projectB/B-file, project/projectC/C-file For testing speed, I added a post-receive "hook" to the repository (see Kohsuke's "polling must die" blog post for the technique) Define a Jenkins job JENKINS-20569 -projectA using that repository Add "Force polling using workspace" behavior to that project Add "Polling ignores commits in certain paths" to that project Define the exclusion as project/projectB/.* project/projectC/.* Commit a change to the repository outside the exclusion (like project/common/common-file), confirm projectA builds on next poll Commit a change to the repository inside the exclusions (like project/projectB/B-file), confirm projectA does not build on next poll Copy the JENKINS-20569 -projectA job to JENKINS-20569 -projectB, define the exclusion as project/projectA/.* project/projectC/.* Commit a change to the repository outside the exclusion (like project/common/common-file), confirm projectA and projectB build on next poll Commit a change to the repository inside the exclusions (like project/projectB/B-file), confirm projectB builds, and projectA does not build on next poll Commit a change to the repository inside the exclusions (like project/projectB/A-file), confirm projectB does not build, and projectA builds on next poll So long as I use "Force polling using workspace", and use a regular expression to match the exclusion, then the exclusion works the way I expect. Can you provide more details on the failure case you found?

          Trey Duskin added a comment -

          My case differs from your example in a few ways.

          First, my jobs are set up to only check out a specific branch (e.g. '*/dev' instead of 'master'). There could be other commits in the history which are going into other branches, but I don't care about those and I only want to watch for commits in a specific branch which have changes in specific paths.

          Second, we are using git-flow style development, and so most of the commits are merges of feature branches into the the branch I am watching.

          If I look at the git polling log on a job which should not have been triggered, I can see it trying to get a history between the last build revision and the commit which triggered the poll (we are using web-hooks from our git provider, Assembla). However, if I repeat the command it uses to get the commit history (e.g. /usr/bin/git log --full-history --no-abbrev --format=raw -M -m --raw ff0ec94a066cee23c13faa16b38b247911cd5ee7..f47993565f701a4f28174c4ea81f35d0e1103590), I see the commits i expect, but weirdly, I also see a commit listed twice - once containing the paths which the commit affected, and another (same hash) which contain paths from some other commit in some other branch! Is this a problem with my git provider?

          Trey Duskin added a comment - My case differs from your example in a few ways. First, my jobs are set up to only check out a specific branch (e.g. '*/dev' instead of 'master'). There could be other commits in the history which are going into other branches, but I don't care about those and I only want to watch for commits in a specific branch which have changes in specific paths. Second, we are using git-flow style development, and so most of the commits are merges of feature branches into the the branch I am watching. If I look at the git polling log on a job which should not have been triggered, I can see it trying to get a history between the last build revision and the commit which triggered the poll (we are using web-hooks from our git provider, Assembla). However, if I repeat the command it uses to get the commit history (e.g. /usr/bin/git log --full-history --no-abbrev --format=raw -M -m --raw ff0ec94a066cee23c13faa16b38b247911cd5ee7..f47993565f701a4f28174c4ea81f35d0e1103590), I see the commits i expect, but weirdly, I also see a commit listed twice - once containing the paths which the commit affected, and another (same hash) which contain paths from some other commit in some other branch! Is this a problem with my git provider?

          Trey Duskin added a comment -

          Oh, and to be clear, I am only using Included Paths. I am not specifying any Excluded Paths.

          Trey Duskin added a comment - Oh, and to be clear, I am only using Included Paths. I am not specifying any Excluded Paths.

          Mark Waite added a comment -

          tduskin could you open a separate bug report for your use case?

          This bug report is focused on excluded regions and on the implicit requirement that include and exclude regions implicitly require "Force polling using workspace". As far as I can tell, the exclude regions are working as expected (at least as described by this bug report) so long as "Force polling using workspace" is enabled.

          Mark Waite added a comment - tduskin could you open a separate bug report for your use case? This bug report is focused on excluded regions and on the implicit requirement that include and exclude regions implicitly require "Force polling using workspace". As far as I can tell, the exclude regions are working as expected (at least as described by this bug report) so long as "Force polling using workspace" is enabled.

          Trey Duskin added a comment -

          Thanks. I did some more experimentation and I think my issue is specific to merge commits. My new bug is here: https://issues.jenkins-ci.org/browse/JENKINS-23606

          Trey Duskin added a comment - Thanks. I did some more experimentation and I think my issue is specific to merge commits. My new bug is here: https://issues.jenkins-ci.org/browse/JENKINS-23606

          Erik Shreve added a comment - - edited

          Excluded regions are also not working for me.

          I have a test repo with the following structure
          /src/main.c
          /docs/otherdocs/somefile.txt

          I've added "Force polling using workspace" as an Additional behavior.
          The excluded regions are set to:

          /docs/otherdocs/.*
          /docs/otherdocs/somefile.txt
          docs/otherdocs/somefile.txt
          docs/otherdocs/.*\.txt

          Included regions is empty.

          When the project polls the repo, it still builds when the only changed file is:
          /docs/otherdocs/somefile.txt

          Change to file is simply
          >echo hello >> somefile.txt
          >git commit -a -m "changed somefile"
          >git push origin master

          I'm on Jenkins 1.597, with git plugin 2.3.5

          Polling Log shows:
          Started on Jun 26, 2015 10:02:01 AM
          Polling SCM changes on mybuildmachine
          Using strategy: Default
          [poll] Last Built Revision: Revision c4ab323cf0ac778ed3b0e5fd13467be5587d3fe7 (refs/remotes/ErikTest/master)
          Fetching changes from the remote Git repositories
          Polling for changes in
          Done. Took 1.1 sec
          Changes found

          Erik Shreve added a comment - - edited Excluded regions are also not working for me. I have a test repo with the following structure /src/main.c /docs/otherdocs/somefile.txt I've added "Force polling using workspace" as an Additional behavior. The excluded regions are set to: /docs/otherdocs/.* /docs/otherdocs/somefile.txt docs/otherdocs/somefile.txt docs/otherdocs/.*\.txt Included regions is empty. When the project polls the repo, it still builds when the only changed file is: /docs/otherdocs/somefile.txt Change to file is simply >echo hello >> somefile.txt >git commit -a -m "changed somefile" >git push origin master I'm on Jenkins 1.597, with git plugin 2.3.5 Polling Log shows: Started on Jun 26, 2015 10:02:01 AM Polling SCM changes on mybuildmachine Using strategy: Default [poll] Last Built Revision: Revision c4ab323cf0ac778ed3b0e5fd13467be5587d3fe7 (refs/remotes/ErikTest/master) Fetching changes from the remote Git repositories Polling for changes in Done. Took 1.1 sec Changes found

          Erik Shreve added a comment -

          For me something is fundamentally broken. Exclusion by commit message also does not work (I haven't tried the other exclusion features).

          Also, I tried setting the excluded regions to just:
          .*

          If I read the PathRestriction class correctly, this should prevent all builds from occurring. However, my build still happens.

          My branch specifier is set to */master
          Build triggers section just has "Poll SCM" checked and it is set to poll once an hour. I'm visiting the git/notifyCommit?url= address of the jenkins server to start the polling.

          Erik Shreve added a comment - For me something is fundamentally broken. Exclusion by commit message also does not work (I haven't tried the other exclusion features). Also, I tried setting the excluded regions to just: .* If I read the PathRestriction class correctly, this should prevent all builds from occurring. However, my build still happens. My branch specifier is set to */master Build triggers section just has "Poll SCM" checked and it is set to poll once an hour. I'm visiting the git/notifyCommit?url= address of the jenkins server to start the polling.

          Erik Shreve added a comment -

          I noticed the following in Jenkins' log file:
          Jun 26, 2015 4:11:02 PM WARNING hudson.plugins.git.GitTool$DescriptorImpl getInstallation
          invalid gitTool selection Default

          The Git Client plugin was configured with JGit. I deleted the JGit entry on the configuration page and after reloading the page it had put a Git entry in as default.

          The exclude regions are now working.

          Erik Shreve added a comment - I noticed the following in Jenkins' log file: Jun 26, 2015 4:11:02 PM WARNING hudson.plugins.git.GitTool$DescriptorImpl getInstallation invalid gitTool selection Default The Git Client plugin was configured with JGit. I deleted the JGit entry on the configuration page and after reloading the page it had put a Git entry in as default. The exclude regions are now working.

          Mark Waite added a comment - - edited

          eshreveti I don't understand the conflict between your quote from the log file and your statement about the job using JGit. The message "invalid gitTool selection Default" is the message I would expect to see when using command line git, yet you indicate that you switched from JGit to command line git. Could you double check that to confirm?

          If you find that is the case, please open another bug report, since this bug report is focused on the requirement that if an include region is defined, or an exclude region is defined, then the job should implicitly require a workspace for polling.

          I double checked with my 3 jobs that I use to experiment with this bug report. I switched all three of them to use JGit, then committed first to only projectA, confirmed projectA was the only one built. Then I committed to projectB and confirmed projectB was the only one built. The same for projectC, then for the combination of A and B. All behaved as expected.

          I had enabled "Polling uses workspace" so that they exclusions and inclusions could be resolved.

          Mark Waite added a comment - - edited eshreveti I don't understand the conflict between your quote from the log file and your statement about the job using JGit. The message "invalid gitTool selection Default" is the message I would expect to see when using command line git, yet you indicate that you switched from JGit to command line git. Could you double check that to confirm? If you find that is the case, please open another bug report, since this bug report is focused on the requirement that if an include region is defined, or an exclude region is defined, then the job should implicitly require a workspace for polling. I double checked with my 3 jobs that I use to experiment with this bug report. I switched all three of them to use JGit, then committed first to only projectA, confirmed projectA was the only one built. Then I committed to projectB and confirmed projectB was the only one built. The same for projectC, then for the combination of A and B. All behaved as expected. I had enabled "Polling uses workspace" so that they exclusions and inclusions could be resolved.

          citizenkahn added a comment - - edited

          I'm seeing this as well and it seems to only happen when the job clears its workspace
          We have two jobs:

          Base line when workspace exists

          • change to master triggers only triggermaster job
          • change to myfeature only triggermyfeature job

          Modify jobs adding exclusion when workspace exists

          • Additional Behaviors : Polling ignores commts in certain paths
            • excluded regions:
              • master: multMergeCommits/.*
              • feature: myfeature/.*
          Tests when workspace exists:
          • master commit to root - trigger only master
          • master commit to file beneath excluded - no trigger
          • feature commit to root - trigger only feature
          • feature commit to file beneath excluded - no trigger
          After purge workspace

          Commit to master branch causes both jobs to trigger. The github payload contains commit data so the plugin goin analyze the payload and determine if the commit path matched the regex for exclusion but it looks like it wants a workspace in which to perform the checks.

          Recommend bump to at least major. This problem with the feature makes it unusable in our environment and I expect many cannot use this feature.

          citizenkahn added a comment - - edited I'm seeing this as well and it seems to only happen when the job clears its workspace We have two jobs: trigger-feature.xml feature branch, excludes region: myfeature/.* trigger-master.xml master, excludes region: multiMergeCommits/.* Base line when workspace exists change to master triggers only triggermaster job change to myfeature only triggermyfeature job Modify jobs adding exclusion when workspace exists Additional Behaviors : Polling ignores commts in certain paths excluded regions: master: multMergeCommits/.* feature: myfeature/.* Tests when workspace exists: master commit to root - trigger only master master commit to file beneath excluded - no trigger feature commit to root - trigger only feature feature commit to file beneath excluded - no trigger After purge workspace Commit to master branch causes both jobs to trigger. The github payload contains commit data so the plugin goin analyze the payload and determine if the commit path matched the regex for exclusion but it looks like it wants a workspace in which to perform the checks. Recommend bump to at least major . This problem with the feature makes it unusable in our environment and I expect many cannot use this feature.

            Unassigned Unassigned
            anthony Anthony FLORANT
            Votes:
            13 Vote for this issue
            Watchers:
            15 Start watching this issue

              Created:
              Updated: