Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-31464

git-client uses proxy from plugins section of jenkins

    • Icon: New Feature New Feature
    • Resolution: Unresolved
    • Icon: Major Major
    • git-client-plugin
    • None

      The git client appears to be using the http proxy settings from the "Manage plugins" section of Jenkins. That proxy is for the master to download plugins, not for SCM clients to access repositories. Slave machines shouldn't require the same access to the internet as the master.

      Here you can see that localhost:3128 is being used as a proxy when that slave does not have a proxy listening on port 31218
      > /usr/bin/git config --local credential.username frank # timeout=10
      > /usr/bin/git config --local credential.helper store --file=/tmp/git6157196107535640953.credentials # timeout=10
      Setting http proxy: localhost:3128
      > /usr/bin/git -c core.askpass=true fetch --tags --progress https://github.com/frank/fright.git +refs/heads/:refs/remotes/origin/
      > /usr/bin/git config --local --remove-section credential # timeout=10

      Using environmental variables should be the approach for accessing http/https on the web from a git client.

          [JENKINS-31464] git-client uses proxy from plugins section of jenkins

          Matt Clark created issue -
          Mark Waite made changes -
          Assignee Original: Nicolas De Loof [ ndeloof ]

          Mark Waite added a comment -

          I disagree. I think the most common proxy use case for the many users of the plugin is when the master and slave are both behind the same http proxy which is blocking direct access to the outside. I can imagine scenarios where the slave is not behind the same proxy as the master (slave in the public cloud managed by a master inside a corporate firewall), but those seem much less likely than master and slaves both behind an http proxy.

          Mark Waite added a comment - I disagree. I think the most common proxy use case for the many users of the plugin is when the master and slave are both behind the same http proxy which is blocking direct access to the outside. I can imagine scenarios where the slave is not behind the same proxy as the master (slave in the public cloud managed by a master inside a corporate firewall), but those seem much less likely than master and slaves both behind an http proxy.

          While it might not be the most common use case, it's certainly a valid one, which jenkins should support.

          If the slave has the usual http_proxy environment variables defined, they should be used.

          Matthias Pretzer added a comment - While it might not be the most common use case, it's certainly a valid one, which jenkins should support. If the slave has the usual http_proxy environment variables defined, they should be used.

          Even worse, jenkins seems to be resetting the http_proxy environment variable. So i have to do

          export http_proxy=$HTTP_PROXY
          git clone http://...
          

          instead of using jenkins built in git.

          Matthias Pretzer added a comment - Even worse, jenkins seems to be resetting the http_proxy environment variable. So i have to do export http_proxy=$HTTP_PROXY git clone http: //... instead of using jenkins built in git.

          Arran added a comment -

          Mark, this isn't always true. The basis of what you say, yes, but if my master is set up to use a proxy to reach the outside world, but my git repo is inside, then the Git client is picking up the proxy from the master and using that.

          Arran added a comment - Mark, this isn't always true. The basis of what you say, yes, but if my master is set up to use a proxy to reach the outside world, but my git repo is inside , then the Git client is picking up the proxy from the master and using that .

          Mark Waite added a comment -

          ahuxtable and matthiasjenkins I agree that a Jenkins system can have slaves and master which require different HTTP proxy settings. I can imagine a private Jenkins server using slaves in a public cloud but needing access to a git repository inside the firewall, or a Jenkins server with slaves at two different locations of the same company, with a site specific http proxy at each of the locations.

          There are already some interesting techniques in other Jenkins plugins which might solve that. I understand that the Mercurial plugin can use the Jenkins master / slave communication channel to carry Mercurial source code changes between client and server. Conceptually, the Jenkins git plugin (with the JGit implementation inside it) could be extended to use a similar technique, transporting all git operations through the master node, using only the Jenkins master / slave communication channel for other git work.

          Unfortunately, I don't think it is a common enough use case for me to invest in it (at least not personally). I'm much more interested in making more authentication cases work, and in making submodule authentication work. If either of you would like to propose changes (with tests) to implement the capability, I'm willing to consider them. I don't want to sound rude, or inconsiderate of your concerns, but I want the personal time I spend on the Jenkins project to benefit as many people as I can, and submodule authentication, existing pull requests, and test improvements are already taking all of the time that I can give to the plugin.

          Mark Waite added a comment - ahuxtable and matthiasjenkins I agree that a Jenkins system can have slaves and master which require different HTTP proxy settings. I can imagine a private Jenkins server using slaves in a public cloud but needing access to a git repository inside the firewall, or a Jenkins server with slaves at two different locations of the same company, with a site specific http proxy at each of the locations. There are already some interesting techniques in other Jenkins plugins which might solve that. I understand that the Mercurial plugin can use the Jenkins master / slave communication channel to carry Mercurial source code changes between client and server. Conceptually, the Jenkins git plugin (with the JGit implementation inside it) could be extended to use a similar technique, transporting all git operations through the master node, using only the Jenkins master / slave communication channel for other git work. Unfortunately, I don't think it is a common enough use case for me to invest in it (at least not personally). I'm much more interested in making more authentication cases work, and in making submodule authentication work. If either of you would like to propose changes (with tests) to implement the capability, I'm willing to consider them. I don't want to sound rude, or inconsiderate of your concerns, but I want the personal time I spend on the Jenkins project to benefit as many people as I can, and submodule authentication, existing pull requests, and test improvements are already taking all of the time that I can give to the plugin.
          Mark Waite made changes -
          Issue Type Original: Bug [ 1 ] New: New Feature [ 2 ]

          Matt Clark added a comment - - edited

          Note that below I am commenting from my perspective, I have no idea on the variations of this problem outside of my own domain so by no means do I assume the below is easy or complete.

          From my perspective, the http_proxy and https_proxy settings are set individually. For every build node, I go to node properties and then environmental variables and set values for ftp_proxy, http_proxy and https_proxy.

          It seems very strange that something is able to clobber these values.

          Who is in control of printing the line (seen in my initial cut-paste of the issue):
          Setting http proxy: localhost:3128

          As this is where the problem lies.

          I have many (20-30) different hosts all in different environments and I need to be able to set the environmental variables based on environment (and that effectively means I set this up on each node). Which is fine, it's explicit and allows all possibilities of nodes being in different environments.

          I can see how it helps people to take a default of what the master is set with, but the ordering is wrong. First it should honor the nodes environmental variables and fall back to that of the master iff one attempt fails (what if the builder can't get to the proxy that the master uses to get to the internet, like if my builders are spun up aws instances). The one attempt fails thing sucks, but you've got no other way to test if the builders are actually on the internet without the need for a proxy (I guess you could set http_proxy="" in the node properties and this would override the global proxy setting).

          The solution of not clobbering the node specific environment variables would mean every single variation of nodes being in same and different locations is 100% supported. So stopping whatever does the:
          Setting http proxy: localhost:3128
          Would allow the problem to be solved for everyone.

          IMHO.

          Happy to wrong about any of the above. I am just eager to hear your thoughts on what I see as a way out.

          Matt Clark added a comment - - edited Note that below I am commenting from my perspective, I have no idea on the variations of this problem outside of my own domain so by no means do I assume the below is easy or complete. From my perspective, the http_proxy and https_proxy settings are set individually. For every build node, I go to node properties and then environmental variables and set values for ftp_proxy, http_proxy and https_proxy. It seems very strange that something is able to clobber these values. Who is in control of printing the line (seen in my initial cut-paste of the issue): Setting http proxy: localhost:3128 As this is where the problem lies. I have many (20-30) different hosts all in different environments and I need to be able to set the environmental variables based on environment (and that effectively means I set this up on each node). Which is fine, it's explicit and allows all possibilities of nodes being in different environments. I can see how it helps people to take a default of what the master is set with, but the ordering is wrong. First it should honor the nodes environmental variables and fall back to that of the master iff one attempt fails (what if the builder can't get to the proxy that the master uses to get to the internet, like if my builders are spun up aws instances). The one attempt fails thing sucks, but you've got no other way to test if the builders are actually on the internet without the need for a proxy (I guess you could set http_proxy="" in the node properties and this would override the global proxy setting). The solution of not clobbering the node specific environment variables would mean every single variation of nodes being in same and different locations is 100% supported. So stopping whatever does the: Setting http proxy: localhost:3128 Would allow the problem to be solved for everyone. IMHO. Happy to wrong about any of the above. I am just eager to hear your thoughts on what I see as a way out.

          scattym I agree. Maybe, this is really an issue within jenkins itself, and not the git plugin?

          Matthias Pretzer added a comment - scattym I agree. Maybe, this is really an issue within jenkins itself, and not the git plugin?

            Unassigned Unassigned
            scattym Matt Clark
            Votes:
            19 Vote for this issue
            Watchers:
            27 Start watching this issue

              Created:
              Updated: