Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-34603

git scm poll not working with global environment variable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • git-plugin
    • None

      Steps to reproduce -

      Manage Jenkins => Configure System => Global properties => Environment variables
      Add
      Name = GITHUB_HOST
      Value = <github IP address>

      Name = GITHUB_REPO
      Value = <github repo path>

      Create a job
      Under job configuration define git repository as below
      https://$

      {GITHUB_HOST}/${GITHUB_REPO}
      provide user/pass for credential
      complete the job configuration

      Trigger SCM poll by using curl (Note - provide your Jenkins host in following command)

      curl http://${JENKINS_SERVER}/jenkins/git/notifyCommit?url=[https://\$|https://%24/]"{GITHUB_HOST}"/\$"{GITHUB_REPO}"

      It'll poll the corresponding job but if you see Git SCM poll log, it's dumping the following error
      In short - It's not able to expand the global variable

      > /usr/bin/git --version # timeout=10
      > /usr/bin/git -c core.askpass=true fetch --tags --progress https://${GITHUB_HOST}

      /${GITHUB_REPO} +refs/heads/:refs/remotes/origin/
      FATAL: hudson.plugins.git.GitException: Failed to fetch from https://$

      {GITHUB_HOST}/${GITHUB_REPO}
      hudson.util.IOException2: hudson.plugins.git.GitException: Failed to fetch from https://${GITHUB_HOST}

      /${GITHUB_REPO}
      at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:573)
      at hudson.scm.SCM.compareRemoteRevisionWith(SCM.java:381)
      at hudson.scm.SCM.poll(SCM.java:398)
      at hudson.model.AbstractProject.pollWithWorkspace(AbstractProject.java:1468)
      at hudson.model.AbstractProject._poll(AbstractProject.java:1438)
      at hudson.model.AbstractProject.poll(AbstractProject.java:1349)
      at jenkins.triggers.SCMTriggerItem$SCMTriggerItems$Bridge.poll(SCMTriggerItem.java:119)
      at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:526)
      at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:555)
      at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:119)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:745)
      Caused by: hudson.plugins.git.GitException: Failed to fetch from https://$

      {GITHUB_HOST}/${GITHUB_REPO}
      at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:810)
      at hudson.plugins.git.GitSCM.compareRemoteRevisionWithImpl(GitSCM.java:701)
      at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:571)
      ... 14 more
      Caused by: hudson.plugins.git.GitException: Command "/usr/bin/git -c core.askpass=true fetch --tags --progress https://${GITHUB_HOST}

      /${GITHUB_REPO} +refs/heads/:refs/remotes/origin/" returned status code 128:
      stdout:
      stderr: remote: Invalid username or password.
       

          [JENKINS-34603] git scm poll not working with global environment variable

          smruti sahoo created issue -

          smruti sahoo added a comment - - edited

          If I'm triggering the job manually either individually or build pipeline - It's working as expected
          It's failing only for "Poll SCM"

          But because of this the notification mails are not getting triggered upon git commit which is causing big issue

          smruti sahoo added a comment - - edited If I'm triggering the job manually either individually or build pipeline - It's working as expected It's failing only for "Poll SCM" But because of this the notification mails are not getting triggered upon git commit which is causing big issue
          smruti sahoo made changes -
          Description Original: Steps to reproduce -

          Manage Jenkins => Configure System => Global properties => Environment variables
          Add
          Name = GITHUB_HOST
          Value = <github IP address>

          Name = GITHUB_REPO
          Value = <github repo path>

          Create a job
          Under job configuration define git repository as below
          https://${GITHUB_HOST}/${GITHUB_REPO}
          provide user/pass for credential
          complete the job configuration

          Trigger SCM poll by using curl (Note - provide your Jenkins host in following command)

          curl http://${JENKINS_SERVER}/jenkins/git/notifyCommit?url=https://\$"\{GITHUB_HOST\}"/\$"\{GITHUB_REPO\}"

          It'll poll the corresponding job but if you see Git SCM poll log, it's dumping the following error
          {color:red}It's not able to expand the global variable {color}

           > /usr/bin/git --version # timeout=10
           > /usr/bin/git -c core.askpass=true fetch --tags --progress https://${GITHUB_HOST}/${GITHUB_REPO} +refs/heads/*:refs/remotes/origin/*
          FATAL: hudson.plugins.git.GitException: Failed to fetch from https://${GITHUB_HOST}/${GITHUB_REPO}
          hudson.util.IOException2: hudson.plugins.git.GitException: Failed to fetch from https://${GITHUB_HOST}/${GITHUB_REPO}
          at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:573)
          at hudson.scm.SCM.compareRemoteRevisionWith(SCM.java:381)
          at hudson.scm.SCM.poll(SCM.java:398)
          at hudson.model.AbstractProject.pollWithWorkspace(AbstractProject.java:1468)
          at hudson.model.AbstractProject._poll(AbstractProject.java:1438)
          at hudson.model.AbstractProject.poll(AbstractProject.java:1349)
          at jenkins.triggers.SCMTriggerItem$SCMTriggerItems$Bridge.poll(SCMTriggerItem.java:119)
          at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:526)
          at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:555)
          at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:119)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
          at java.util.concurrent.FutureTask.run(FutureTask.java:262)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
          at java.lang.Thread.run(Thread.java:745)
          Caused by: hudson.plugins.git.GitException: Failed to fetch from https://${GITHUB_HOST}/${GITHUB_REPO}
          at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:810)
          at hudson.plugins.git.GitSCM.compareRemoteRevisionWithImpl(GitSCM.java:701)
          at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:571)
          ... 14 more
          Caused by: hudson.plugins.git.GitException: Command "/usr/bin/git -c core.askpass=true fetch --tags --progress https://${GITHUB_HOST}/${GITHUB_REPO} +refs/heads/*:refs/remotes/origin/*" returned status code 128:
          stdout:
          stderr: remote: Invalid username or password.

          New: Steps to reproduce -

          Manage Jenkins => Configure System => Global properties => Environment variables
          Add
          Name = GITHUB_HOST
          Value = <github IP address>

          Name = GITHUB_REPO
          Value = <github repo path>

          Create a job
          Under job configuration define git repository as below
          https://${GITHUB_HOST}/${GITHUB_REPO}
          provide user/pass for credential
          complete the job configuration

          Trigger SCM poll by using curl (Note - provide your Jenkins host in following command)

          curl http://${JENKINS_SERVER}/jenkins/git/notifyCommit?url=https://\$"\{GITHUB_HOST\}"/\$"\{GITHUB_REPO\}"

          It'll poll the corresponding job but if you see Git SCM poll log, it's dumping the following error
          {color:red}In short - It's not able to expand the global variable {color}

           > /usr/bin/git --version # timeout=10
           > /usr/bin/git -c core.askpass=true fetch --tags --progress https://${GITHUB_HOST}/${GITHUB_REPO} +refs/heads/*:refs/remotes/origin/*
          FATAL: hudson.plugins.git.GitException: Failed to fetch from https://${GITHUB_HOST}/${GITHUB_REPO}
          hudson.util.IOException2: hudson.plugins.git.GitException: Failed to fetch from https://${GITHUB_HOST}/${GITHUB_REPO}
          at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:573)
          at hudson.scm.SCM.compareRemoteRevisionWith(SCM.java:381)
          at hudson.scm.SCM.poll(SCM.java:398)
          at hudson.model.AbstractProject.pollWithWorkspace(AbstractProject.java:1468)
          at hudson.model.AbstractProject._poll(AbstractProject.java:1438)
          at hudson.model.AbstractProject.poll(AbstractProject.java:1349)
          at jenkins.triggers.SCMTriggerItem$SCMTriggerItems$Bridge.poll(SCMTriggerItem.java:119)
          at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:526)
          at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:555)
          at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:119)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
          at java.util.concurrent.FutureTask.run(FutureTask.java:262)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
          at java.lang.Thread.run(Thread.java:745)
          Caused by: hudson.plugins.git.GitException: Failed to fetch from https://${GITHUB_HOST}/${GITHUB_REPO}
          at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:810)
          at hudson.plugins.git.GitSCM.compareRemoteRevisionWithImpl(GitSCM.java:701)
          at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:571)
          ... 14 more
          Caused by: hudson.plugins.git.GitException: Command "/usr/bin/git -c core.askpass=true fetch --tags --progress https://${GITHUB_HOST}/${GITHUB_REPO} +refs/heads/*:refs/remotes/origin/*" returned status code 128:
          stdout:
          stderr: remote: Invalid username or password.

          smruti sahoo added a comment -

          smruti sahoo added a comment - A similar issue raised earlier https://issues.jenkins-ci.org/browse/JENKINS-26315

          Brian Barker added a comment -

          I also would like this addressed. When you have hundreds of engineers browsing build jobs, it is nice to set a var like $BRANCH=develop so they can just change that and not miss the multiple places in our config we use that value.

          Brian Barker added a comment - I also would like this addressed. When you have hundreds of engineers browsing build jobs, it is nice to set a var like $BRANCH=develop so they can just change that and not miss the multiple places in our config we use that value.
          Mark Waite made changes -
          Assignee Original: Mark Waite [ markewaite ]
          R. Tyler Croy made changes -
          Workflow Original: JNJira [ 170764 ] New: JNJira + In-Review [ 184029 ]

          I am also seeing this issue. Environment variables are not being substituted when polling runs. I am using an environment variable for my Branch Specifier in the SCM configuration.

          I have tried this in many versions of the Git Plugin. Version 3.0 is the latest, but version 2.3.4 is the last known working version that I have found.

          Jeffrey Mendez added a comment - I am also seeing this issue. Environment variables are not being substituted when polling runs. I am using an environment variable for my Branch Specifier in the SCM configuration. I have tried this in many versions of the Git Plugin. Version 3.0 is the latest, but version 2.3.4 is the last known working version that I have found.

          I've just updated a bunch of other plugins that affect Jenkins's security, but I left the Git plugin at 2.3.4. However, this is now broken for me again, so it would seem the Git plugin is not at fault here.

          Is anyone else still affected by this problem?

          Jeffrey Mendez added a comment - I've just updated a bunch of other plugins that affect Jenkins's security, but I left the Git plugin at 2.3.4. However, this is now broken for me again, so it would seem the Git plugin is not at fault here. Is anyone else still affected by this problem?

          Brian Ray added a comment -

          jmendez, I performed a regular upgrade of our LTS core and plugins a few days ago and hit the current issue again despite it having worked with Freestyle jobs for quite some time. Some of the upgraded components:

          • LTS Core: 2.32.3 > 2.46.3
          • Git Client: 2.4.1 > 2.4.6
          • Git: 3.2.0 > 3.3.0

          And some plugins that were already at the latest versions:

          • SCM API: 2.1.1
          • Subversion: 2.7.2

          Job parameters in the Repository URL string expand fine, but not global parameters.

          Interestingly enough our polling Subversion jobs also started manifesting an analagous error in the Polling Log:

          Started on Jun 7, 2017 7:33:00 PM
          ERROR: Failed to check repository revision for ${SVN_URL}
          org.tmatesoft.svn.core.SVNException: svn: E125002: Malformed URL '${DEV_SRC_REPO_PB}/trunk/********'
          	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)
          	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
          	at org.tmatesoft.svn.core.SVNURL.<init>(SVNURL.java:227)
          	at org.tmatesoft.svn.core.SVNURL.parseURIEncoded(SVNURL.java:121)
          	at hudson.scm.SubversionSCM$ModuleLocation.getSVNURL(SubversionSCM.java:2769)
          	at hudson.scm.SubversionSCM.compareRemoteRevisionWith(SubversionSCM.java:1415)
          	at hudson.scm.SCM.compareRemoteRevisionWith(SCM.java:392)
          	at hudson.scm.SCM.poll(SCM.java:409)
          	at hudson.model.AbstractProject._poll(AbstractProject.java:1463)
          	at hudson.model.AbstractProject.poll(AbstractProject.java:1366)
          	at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:596)
          	at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:642)
          	at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:119)
          	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
          	at java.util.concurrent.FutureTask.run(Unknown Source)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
          	at java.lang.Thread.run(Unknown Source)
          Done. Took 34 ms
          Changes found
          

          Note how the error is nonfatal for SVN polling; changes are detected despite the failure to expand the global property. As hinted at in the Jenkins error log, I think SVN polling falls back on the last successful URL polled if available.

          WARNING: no revision found corresponding to ${DEV_SRC_REPO_PB}/trunk/*********; known: [https://*********:18080/svn/*********/trunk/*********]
          

          As we are migrating to Pipeline I have simply hardcoded those values in the jobs for now.

          Brian Ray added a comment - jmendez , I performed a regular upgrade of our LTS core and plugins a few days ago and hit the current issue again despite it having worked with Freestyle jobs for quite some time. Some of the upgraded components: LTS Core: 2.32.3 > 2.46.3 Git Client: 2.4.1 > 2.4.6 Git: 3.2.0 > 3.3.0 And some plugins that were already at the latest versions: SCM API: 2.1.1 Subversion: 2.7.2 Job parameters in the Repository URL string expand fine, but not global parameters. Interestingly enough our polling Subversion jobs also started manifesting an analagous error in the Polling Log: Started on Jun 7, 2017 7:33:00 PM ERROR: Failed to check repository revision for ${SVN_URL} org.tmatesoft.svn.core.SVNException: svn: E125002: Malformed URL '${DEV_SRC_REPO_PB}/trunk/********' at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70) at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57) at org.tmatesoft.svn.core.SVNURL.<init>(SVNURL.java:227) at org.tmatesoft.svn.core.SVNURL.parseURIEncoded(SVNURL.java:121) at hudson.scm.SubversionSCM$ModuleLocation.getSVNURL(SubversionSCM.java:2769) at hudson.scm.SubversionSCM.compareRemoteRevisionWith(SubversionSCM.java:1415) at hudson.scm.SCM.compareRemoteRevisionWith(SCM.java:392) at hudson.scm.SCM.poll(SCM.java:409) at hudson.model.AbstractProject._poll(AbstractProject.java:1463) at hudson.model.AbstractProject.poll(AbstractProject.java:1366) at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:596) at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:642) at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:119) at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Done. Took 34 ms Changes found Note how the error is nonfatal for SVN polling; changes are detected despite the failure to expand the global property. As hinted at in the Jenkins error log, I think SVN polling falls back on the last successful URL polled if available. WARNING: no revision found corresponding to ${DEV_SRC_REPO_PB}/trunk/*********; known: [https://*********:18080/svn/*********/trunk/*********] As we are migrating to Pipeline I have simply hardcoded those values in the jobs for now.

            Unassigned Unassigned
            smruti_sahoo smruti sahoo
            Votes:
            4 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated: