Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-29631

Non-workspace polling using git ls-remote doesn't work for several edge cases

      Right now, there are several cases where 'git ls-remote' is used by default to check for changes that result in incorrect behavior.

      1. Currently, '-h' is passed to ls-remote, which automatically ignores any refs not under refs/heads or refs/tags. If the refspec is set to fetch refs/pull-requests/*/from for example, passing -h will cause that to be ignored (this example comes from Atlassian Stash)

      2. The plugin only uses the first result of ls-remote. In the case of branch strings with wildcards, there could be multiple results.

      3. If the refspec remaps between the remote and the local, ls-remote isn't suitable to check for changes because the branch might have a different name. For example, if the refspec '+refs/pull-requests/*/from:refs/heads/origin/pr/*' is used with a branch string of 'origin/pr/**', checking the remote for changes to origin/pr/** won't work.

      It looks like the plugin is already smart enough to switch over to workspace-based polling in some of these cases if the branch string is specified directly, but we've run into cases where it reverts to using 'git ls-remote' when branch comes from a job parameter.

      All of our projects have a BRANCH parameter with a default value that's used to fill in the git plugin's branch field.

      In addition, we have two refspecs for normal jobs - the regular all branches one and one to match tags. Pull request jobs point at Atlassian Stash pull requests (+refs/pull-requests/*/from:refs/heads/origin/pr/*).

      When Jenkins gets notified to poll for changes, it appears to outright ignore anything that wouldn't be matched by a default refspec - in this case, it won't match tags or pull requests since they aren't present in a standard fetch, despite explicitly setting the refspec. Worse, if both the default refspec and the tags refspec are listed, it won't trigger on any changes, reporting that nothing matches the configured refspec, not even master.

      This worked just fine in the prior version of the git plugin (2.2.7). I've reproduced the problem on both Jenkins 1.584 and 1.609.1. I've tried forcing re-clone and forcing polling with a workspace to no effect.

      It seems to have something to do with the parameterized branch, as using a concrete branch parameter seemed to make the issue go away.

          [JENKINS-29631] Non-workspace polling using git ls-remote doesn't work for several edge cases

          Jason Miller created issue -
          Jason Miller made changes -
          Description Original: All of our projects have a BRANCH parameter with a default value that's used to fill in the git plugin's branch field.

          In addition, we have two refspecs for normal jobs - the regular all branches one and one to match tags. Pull request jobs point at Atlassian Stash pull requests ({code}+refs/pull-requests/*/from:refs/heads/origin/pr/*{code}).

          When Jenkins gets notified to poll for changes, it appears to outright ignore anything that wouldn't be matched by a default refspec - in this case, it won't match tags or pull requests since they aren't present in a standard fetch, despite explicitly setting the refspec. Worse, if both the default refspec and the tags refspec are listed, it won't trigger on *any* changes, reporting that nothing matches the configured refspec, not even master.

          This worked just fine in the prior version of the git plugin (2.3.5). I've reproduced the problem on both Jenkins 1.584 and 1.609.1. I've tried forcing re-clone and forcing polling with a workspace to no effect.

          Example polling log for pull request:
          {code}
          [poll] Last Built Revision: Revision aba91cae473c7318a85b31f5271c95f4a8f7c0cc (origin/*******)
           > git rev-parse --is-inside-work-tree # timeout=10
          Fetching changes from the remote Git repositories
           > git config remote.origin.url ssh://git@stash.*******.com/GROUP/PROJECT.git # timeout=10
          Fetching upstream changes from ssh://git@stash.*******.com/GROUP/PROJECT.git
           > git --version # timeout=10
           > git -c core.askpass=true fetch --tags --progress ssh://git@stash.*******.com/GROUP/PROJECT.git +refs/pull-requests/*/from:refs/heads/origin/pr/*
          Polling for changes in
          Seen branch in repository origin/**********
          Seen branch in repository origin/**********
          Seen branch in repository origin/**********
          Seen branch in repository origin/master
          Seen branch in repository origin/**********
          Seen branch in repository origin/TAG
          Seen 6 remote branches
          Done. Took 0.52 sec
          No changes
          {code}
          New: All of our projects have a BRANCH parameter with a default value that's used to fill in the git plugin's branch field.

          In addition, we have two refspecs for normal jobs - the regular all branches one and one to match tags. Pull request jobs point at Atlassian Stash pull requests ({code}+refs/pull-requests/*/from:refs/heads/origin/pr/*{code}).

          When Jenkins gets notified to poll for changes, it appears to outright ignore anything that wouldn't be matched by a default refspec - in this case, it won't match tags or pull requests since they aren't present in a standard fetch, despite explicitly setting the refspec. Worse, if both the default refspec and the tags refspec are listed, it won't trigger on *any* changes, reporting that nothing matches the configured refspec, not even master.

          This worked just fine in the prior version of the git plugin (2.3.5). I've reproduced the problem on both Jenkins 1.584 and 1.609.1. I've tried forcing re-clone and forcing polling with a workspace to no effect.

          It seems to have something to do with the parameterized branch, as using a concrete branch parameter seemed to make the issue go away.

          Example polling log for pull request:
          {code}
          [poll] Last Built Revision: Revision aba91cae473c7318a85b31f5271c95f4a8f7c0cc (origin/*******)
           > git rev-parse --is-inside-work-tree # timeout=10
          Fetching changes from the remote Git repositories
           > git config remote.origin.url ssh://git@stash.*******.com/GROUP/PROJECT.git # timeout=10
          Fetching upstream changes from ssh://git@stash.*******.com/GROUP/PROJECT.git
           > git --version # timeout=10
           > git -c core.askpass=true fetch --tags --progress ssh://git@stash.*******.com/GROUP/PROJECT.git +refs/pull-requests/*/from:refs/heads/origin/pr/*
          Polling for changes in
          Seen branch in repository origin/**********
          Seen branch in repository origin/**********
          Seen branch in repository origin/**********
          Seen branch in repository origin/master
          Seen branch in repository origin/**********
          Seen branch in repository origin/TAG
          Seen 6 remote branches
          Done. Took 0.52 sec
          No changes
          {code}
          Jason Miller made changes -
          Description Original: All of our projects have a BRANCH parameter with a default value that's used to fill in the git plugin's branch field.

          In addition, we have two refspecs for normal jobs - the regular all branches one and one to match tags. Pull request jobs point at Atlassian Stash pull requests ({code}+refs/pull-requests/*/from:refs/heads/origin/pr/*{code}).

          When Jenkins gets notified to poll for changes, it appears to outright ignore anything that wouldn't be matched by a default refspec - in this case, it won't match tags or pull requests since they aren't present in a standard fetch, despite explicitly setting the refspec. Worse, if both the default refspec and the tags refspec are listed, it won't trigger on *any* changes, reporting that nothing matches the configured refspec, not even master.

          This worked just fine in the prior version of the git plugin (2.3.5). I've reproduced the problem on both Jenkins 1.584 and 1.609.1. I've tried forcing re-clone and forcing polling with a workspace to no effect.

          It seems to have something to do with the parameterized branch, as using a concrete branch parameter seemed to make the issue go away.

          Example polling log for pull request:
          {code}
          [poll] Last Built Revision: Revision aba91cae473c7318a85b31f5271c95f4a8f7c0cc (origin/*******)
           > git rev-parse --is-inside-work-tree # timeout=10
          Fetching changes from the remote Git repositories
           > git config remote.origin.url ssh://git@stash.*******.com/GROUP/PROJECT.git # timeout=10
          Fetching upstream changes from ssh://git@stash.*******.com/GROUP/PROJECT.git
           > git --version # timeout=10
           > git -c core.askpass=true fetch --tags --progress ssh://git@stash.*******.com/GROUP/PROJECT.git +refs/pull-requests/*/from:refs/heads/origin/pr/*
          Polling for changes in
          Seen branch in repository origin/**********
          Seen branch in repository origin/**********
          Seen branch in repository origin/**********
          Seen branch in repository origin/master
          Seen branch in repository origin/**********
          Seen branch in repository origin/TAG
          Seen 6 remote branches
          Done. Took 0.52 sec
          No changes
          {code}
          New: All of our projects have a BRANCH parameter with a default value that's used to fill in the git plugin's branch field.

          In addition, we have two refspecs for normal jobs - the regular all branches one and one to match tags. Pull request jobs point at Atlassian Stash pull requests ({code}+refs/pull-requests/*/from:refs/heads/origin/pr/*{code}).

          When Jenkins gets notified to poll for changes, it appears to outright ignore anything that wouldn't be matched by a default refspec - in this case, it won't match tags or pull requests since they aren't present in a standard fetch, despite explicitly setting the refspec. Worse, if both the default refspec and the tags refspec are listed, it won't trigger on *any* changes, reporting that nothing matches the configured refspec, not even master.

          This worked just fine in the prior version of the git plugin (2.3.5). I've reproduced the problem on both Jenkins 1.584 and 1.609.1. I've tried forcing re-clone and forcing polling with a workspace to no effect.

          It seems to have something to do with the parameterized branch, as using a concrete branch parameter seemed to make the issue go away.

          Example polling log for pull request:
          {noformat}
          [poll] Last Built Revision: Revision aba91cae473c7318a85b31f5271c95f4a8f7c0cc (origin/*******)
           > git rev-parse --is-inside-work-tree # timeout=10
          Fetching changes from the remote Git repositories
           > git config remote.origin.url ssh://git@stash.*******.com/GROUP/PROJECT.git # timeout=10
          Fetching upstream changes from ssh://git@stash.*******.com/GROUP/PROJECT.git
           > git --version # timeout=10
           > git -c core.askpass=true fetch --tags --progress ssh://git@stash.*******.com/GROUP/PROJECT.git +refs/pull-requests/*/from:refs/heads/origin/pr/*
          Polling for changes in
          Seen branch in repository origin/**********
          Seen branch in repository origin/**********
          Seen branch in repository origin/**********
          Seen branch in repository origin/master
          Seen branch in repository origin/**********
          Seen branch in repository origin/TAG
          Seen 6 remote branches
          Done. Took 0.52 sec
          No changes
          {noformat}
          Jason Miller made changes -
          Description Original: All of our projects have a BRANCH parameter with a default value that's used to fill in the git plugin's branch field.

          In addition, we have two refspecs for normal jobs - the regular all branches one and one to match tags. Pull request jobs point at Atlassian Stash pull requests ({code}+refs/pull-requests/*/from:refs/heads/origin/pr/*{code}).

          When Jenkins gets notified to poll for changes, it appears to outright ignore anything that wouldn't be matched by a default refspec - in this case, it won't match tags or pull requests since they aren't present in a standard fetch, despite explicitly setting the refspec. Worse, if both the default refspec and the tags refspec are listed, it won't trigger on *any* changes, reporting that nothing matches the configured refspec, not even master.

          This worked just fine in the prior version of the git plugin (2.3.5). I've reproduced the problem on both Jenkins 1.584 and 1.609.1. I've tried forcing re-clone and forcing polling with a workspace to no effect.

          It seems to have something to do with the parameterized branch, as using a concrete branch parameter seemed to make the issue go away.

          Example polling log for pull request:
          {noformat}
          [poll] Last Built Revision: Revision aba91cae473c7318a85b31f5271c95f4a8f7c0cc (origin/*******)
           > git rev-parse --is-inside-work-tree # timeout=10
          Fetching changes from the remote Git repositories
           > git config remote.origin.url ssh://git@stash.*******.com/GROUP/PROJECT.git # timeout=10
          Fetching upstream changes from ssh://git@stash.*******.com/GROUP/PROJECT.git
           > git --version # timeout=10
           > git -c core.askpass=true fetch --tags --progress ssh://git@stash.*******.com/GROUP/PROJECT.git +refs/pull-requests/*/from:refs/heads/origin/pr/*
          Polling for changes in
          Seen branch in repository origin/**********
          Seen branch in repository origin/**********
          Seen branch in repository origin/**********
          Seen branch in repository origin/master
          Seen branch in repository origin/**********
          Seen branch in repository origin/TAG
          Seen 6 remote branches
          Done. Took 0.52 sec
          No changes
          {noformat}
          New: All of our projects have a BRANCH parameter with a default value that's used to fill in the git plugin's branch field.

          In addition, we have two refspecs for normal jobs - the regular all branches one and one to match tags. Pull request jobs point at Atlassian Stash pull requests ({code}+refs/pull-requests/*/from:refs/heads/origin/pr/*{code}).

          When Jenkins gets notified to poll for changes, it appears to outright ignore anything that wouldn't be matched by a default refspec - in this case, it won't match tags or pull requests since they aren't present in a standard fetch, despite explicitly setting the refspec. Worse, if both the default refspec and the tags refspec are listed, it won't trigger on *any* changes, reporting that nothing matches the configured refspec, not even master.

          This worked just fine in the prior version of the git plugin (-2.3.5- 2.2.7). I've reproduced the problem on both Jenkins 1.584 and 1.609.1. I've tried forcing re-clone and forcing polling with a workspace to no effect.

          It seems to have something to do with the parameterized branch, as using a concrete branch parameter seemed to make the issue go away.

          Example polling log for pull request:
          {noformat}
          [poll] Last Built Revision: Revision aba91cae473c7318a85b31f5271c95f4a8f7c0cc (origin/*******)
           > git rev-parse --is-inside-work-tree # timeout=10
          Fetching changes from the remote Git repositories
           > git config remote.origin.url ssh://git@stash.*******.com/GROUP/PROJECT.git # timeout=10
          Fetching upstream changes from ssh://git@stash.*******.com/GROUP/PROJECT.git
           > git --version # timeout=10
           > git -c core.askpass=true fetch --tags --progress ssh://git@stash.*******.com/GROUP/PROJECT.git +refs/pull-requests/*/from:refs/heads/origin/pr/*
          Polling for changes in
          Seen branch in repository origin/**********
          Seen branch in repository origin/**********
          Seen branch in repository origin/**********
          Seen branch in repository origin/master
          Seen branch in repository origin/**********
          Seen branch in repository origin/TAG
          Seen 6 remote branches
          Done. Took 0.52 sec
          No changes
          {noformat}
          Jason Miller made changes -
          Description Original: All of our projects have a BRANCH parameter with a default value that's used to fill in the git plugin's branch field.

          In addition, we have two refspecs for normal jobs - the regular all branches one and one to match tags. Pull request jobs point at Atlassian Stash pull requests ({code}+refs/pull-requests/*/from:refs/heads/origin/pr/*{code}).

          When Jenkins gets notified to poll for changes, it appears to outright ignore anything that wouldn't be matched by a default refspec - in this case, it won't match tags or pull requests since they aren't present in a standard fetch, despite explicitly setting the refspec. Worse, if both the default refspec and the tags refspec are listed, it won't trigger on *any* changes, reporting that nothing matches the configured refspec, not even master.

          This worked just fine in the prior version of the git plugin (-2.3.5- 2.2.7). I've reproduced the problem on both Jenkins 1.584 and 1.609.1. I've tried forcing re-clone and forcing polling with a workspace to no effect.

          It seems to have something to do with the parameterized branch, as using a concrete branch parameter seemed to make the issue go away.

          Example polling log for pull request:
          {noformat}
          [poll] Last Built Revision: Revision aba91cae473c7318a85b31f5271c95f4a8f7c0cc (origin/*******)
           > git rev-parse --is-inside-work-tree # timeout=10
          Fetching changes from the remote Git repositories
           > git config remote.origin.url ssh://git@stash.*******.com/GROUP/PROJECT.git # timeout=10
          Fetching upstream changes from ssh://git@stash.*******.com/GROUP/PROJECT.git
           > git --version # timeout=10
           > git -c core.askpass=true fetch --tags --progress ssh://git@stash.*******.com/GROUP/PROJECT.git +refs/pull-requests/*/from:refs/heads/origin/pr/*
          Polling for changes in
          Seen branch in repository origin/**********
          Seen branch in repository origin/**********
          Seen branch in repository origin/**********
          Seen branch in repository origin/master
          Seen branch in repository origin/**********
          Seen branch in repository origin/TAG
          Seen 6 remote branches
          Done. Took 0.52 sec
          No changes
          {noformat}
          New: Non-workspace polling using git ls-remote doesn't work for several edge cases

          Right now, there are several cases where 'git ls-remote' is used by default to check for changes that result in incorrect behavior.

          1. Currently, '-h' is passed to ls-remote, which automatically ignores any refs not under refs/heads or refs/tags. If the refspec is set to fetch refs/pull-requests/*/from for example, passing -h will cause that to be ignored (this example comes from Atlassian Stash)

          2. The plugin only uses the first result of ls-remote. In the case of branch strings with wildcards, there could be multiple results.

          3. If the refspec remaps between the remote and the local, ls-remote isn't suitable to check for changes because the branch might have a different name. For example, if the refspec '+refs/pull-requests/\*/from:refs/heads/origin/pr/\*' is used with a branch string of 'origin/pr/\*\*', checking the remote for changes to origin/pr/\*\* won't work.

          It looks like the plugin is already smart enough to switch over to workspace-based polling in some of these cases if the branch string is specified directly, but we've run into cases where it reverts to using 'git ls-remote' when branch comes from a job parameter.

          -All of our projects have a BRANCH parameter with a default value that's used to fill in the git plugin's branch field.-

          -In addition, we have two refspecs for normal jobs - the regular all branches one and one to match tags. Pull request jobs point at Atlassian Stash pull requests (+refs/pull-requests/\*/from:refs/heads/origin/pr/\*).-

          -When Jenkins gets notified to poll for changes, it appears to outright ignore anything that wouldn't be matched by a default refspec - in this case, it won't match tags or pull requests since they aren't present in a standard fetch, despite explicitly setting the refspec. Worse, if both the default refspec and the tags refspec are listed, it won't trigger on *any* changes, reporting that nothing matches the configured refspec, not even master.-

          -This worked just fine in the prior version of the git plugin (2.2.7). I've reproduced the problem on both Jenkins 1.584 and 1.609.1. I've tried forcing re-clone and forcing polling with a workspace to no effect.-

          -It seems to have something to do with the parameterized branch, as using a concrete branch parameter seemed to make the issue go away.-
          Jason Miller made changes -
          Summary Original: Trigger notification broken for non-default refspecs when using parameterized branch New: Non-workspace polling using git ls-remote doesn't work for several edge cases
          Jason Miller made changes -
          Description Original: Non-workspace polling using git ls-remote doesn't work for several edge cases

          Right now, there are several cases where 'git ls-remote' is used by default to check for changes that result in incorrect behavior.

          1. Currently, '-h' is passed to ls-remote, which automatically ignores any refs not under refs/heads or refs/tags. If the refspec is set to fetch refs/pull-requests/*/from for example, passing -h will cause that to be ignored (this example comes from Atlassian Stash)

          2. The plugin only uses the first result of ls-remote. In the case of branch strings with wildcards, there could be multiple results.

          3. If the refspec remaps between the remote and the local, ls-remote isn't suitable to check for changes because the branch might have a different name. For example, if the refspec '+refs/pull-requests/\*/from:refs/heads/origin/pr/\*' is used with a branch string of 'origin/pr/\*\*', checking the remote for changes to origin/pr/\*\* won't work.

          It looks like the plugin is already smart enough to switch over to workspace-based polling in some of these cases if the branch string is specified directly, but we've run into cases where it reverts to using 'git ls-remote' when branch comes from a job parameter.

          -All of our projects have a BRANCH parameter with a default value that's used to fill in the git plugin's branch field.-

          -In addition, we have two refspecs for normal jobs - the regular all branches one and one to match tags. Pull request jobs point at Atlassian Stash pull requests (+refs/pull-requests/\*/from:refs/heads/origin/pr/\*).-

          -When Jenkins gets notified to poll for changes, it appears to outright ignore anything that wouldn't be matched by a default refspec - in this case, it won't match tags or pull requests since they aren't present in a standard fetch, despite explicitly setting the refspec. Worse, if both the default refspec and the tags refspec are listed, it won't trigger on *any* changes, reporting that nothing matches the configured refspec, not even master.-

          -This worked just fine in the prior version of the git plugin (2.2.7). I've reproduced the problem on both Jenkins 1.584 and 1.609.1. I've tried forcing re-clone and forcing polling with a workspace to no effect.-

          -It seems to have something to do with the parameterized branch, as using a concrete branch parameter seemed to make the issue go away.-
          New: Right now, there are several cases where 'git ls-remote' is used by default to check for changes that result in incorrect behavior.

          1. Currently, '-h' is passed to ls-remote, which automatically ignores any refs not under refs/heads or refs/tags. If the refspec is set to fetch refs/pull-requests/*/from for example, passing -h will cause that to be ignored (this example comes from Atlassian Stash)

          2. The plugin only uses the first result of ls-remote. In the case of branch strings with wildcards, there could be multiple results.

          3. If the refspec remaps between the remote and the local, ls-remote isn't suitable to check for changes because the branch might have a different name. For example, if the refspec '+refs/pull-requests/\*/from:refs/heads/origin/pr/\*' is used with a branch string of 'origin/pr/\*\*', checking the remote for changes to origin/pr/\*\* won't work.

          It looks like the plugin is already smart enough to switch over to workspace-based polling in some of these cases if the branch string is specified directly, but we've run into cases where it reverts to using 'git ls-remote' when branch comes from a job parameter.

          -All of our projects have a BRANCH parameter with a default value that's used to fill in the git plugin's branch field.-

          -In addition, we have two refspecs for normal jobs - the regular all branches one and one to match tags. Pull request jobs point at Atlassian Stash pull requests (+refs/pull-requests/\*/from:refs/heads/origin/pr/\*).-

          -When Jenkins gets notified to poll for changes, it appears to outright ignore anything that wouldn't be matched by a default refspec - in this case, it won't match tags or pull requests since they aren't present in a standard fetch, despite explicitly setting the refspec. Worse, if both the default refspec and the tags refspec are listed, it won't trigger on *any* changes, reporting that nothing matches the configured refspec, not even master.-

          -This worked just fine in the prior version of the git plugin (2.2.7). I've reproduced the problem on both Jenkins 1.584 and 1.609.1. I've tried forcing re-clone and forcing polling with a workspace to no effect.-

          -It seems to have something to do with the parameterized branch, as using a concrete branch parameter seemed to make the issue go away.-
          R. Tyler Croy made changes -
          Workflow Original: JNJira [ 164513 ] New: JNJira + In-Review [ 181648 ]
          CloudBees Inc. made changes -
          Remote Link New: This issue links to "CloudBees Internal OSS-244 (Web Link)" [ 18936 ]
          Nicolas De Loof made changes -
          Assignee Original: Nicolas De Loof [ ndeloof ]

            Unassigned Unassigned
            stormtau Jason Miller
            Votes:
            9 Vote for this issue
            Watchers:
            14 Start watching this issue

              Created:
              Updated: