Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-48775

Proxy authentication error 407 even if 'check proxy' works

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Critical Critical
    • core

      Proxy configuration doesn't work even if 'check proxy' works! No workaround found.

      Situation: Fresh installed Windows Server 2016 with latest, fresh installed  LTS Jenkins version 2.89.2. Server is behind a proxy. Check of Jenkins proxy configuration says 'Ok' but checking plugins crashes jenkins (see attached screen shot).

      I have a second machine (Windows 7 enterprise) with the same Jenkins version, behind the same proxy, with (really!) the same proxy configuration: Check Jenkins proxy settings says: "Error 407", BUT: Checking plugins works! (WTF!?!) One differrence is the java runtime verision: on this machine = 1.8.0_66-b18.

          [JENKINS-48775] Proxy authentication error 407 even if 'check proxy' works

          From the release notes:

          Default the built-in Jenkins Update Center URL to https://updates.jenkins.io instead of obsolete HTTP endpoint. This requires a JRE compatible with Let's Encrypt, e.g. Oracle JRE 8u101.

          i.e. as you mention, the Java version makes a difference.

          Christopher Orr added a comment - From the  release notes : Default the built-in Jenkins Update Center URL to https://updates.jenkins.io instead of obsolete HTTP endpoint. This requires a JRE compatible with Let's Encrypt , e.g. Oracle JRE 8u101. i.e. as you mention, the Java version makes a difference.

          Gerald Ortner added a comment -

          Hi Christopher,

          at first thank you for your fast answer!
          Unfortunately, the situation on my both Jenkins servers is inverted: That one with JRE 1.8.0_66-b18 is working and the one with 1.8.0_144-b01 isn't working. Even the 'proxy check' is the other way around.

          bg Gerald

           

          Gerald Ortner added a comment - Hi Christopher, at first thank you for your fast answer! Unfortunately, the situation on my both Jenkins servers is inverted: That one with JRE 1.8.0_66-b18 is working and the one with 1.8.0_144-b01 isn't working. Even the 'proxy check' is the other way around. bg Gerald  

          Gerald Ortner added a comment -

          due to the situation that the Jenkins Server with the newer JRE makes the troubles => still an issue.

           

          Gerald Ortner added a comment - due to the situation that the Jenkins Server with the newer JRE makes the troubles => still an issue.  

          Gerald Ortner added a comment -

          Hi rtyler,

          I have new gained insights of this problem:

          After consulting out IT guys, it was clear that the Jenkins server DON'T send any authentication to the proxy (maybe except when pressing 'check proxy').
          So there were made two configurations on the company proxy server:
          1. added 'https://updates.jenkins.io' to the 'DoNotAuthenticate whitelist'
          2. turned off the 'SSL Interception' for this URL
          Now, with this workaround, checking and downloading plugin works! But that can't be the final solution!

          Thank you for your effort!

          BG Gerald

           

          Gerald Ortner added a comment - Hi rtyler, I have new gained insights of this problem: After consulting out IT guys, it was clear that the Jenkins server  DON'T send any authentication to the proxy (maybe except when pressing 'check proxy'). So there were made two configurations on the company proxy server: 1. added ' https://updates.jenkins.io'  to the 'DoNotAuthenticate whitelist' 2. turned off the 'SSL Interception' for this URL Now, with this workaround, checking and downloading plugin works! But that can't be the final solution! Thank you for your effort! BG Gerald  

          Alexey Marin added a comment -

          I can confirm that I have the same problem with Jenkins 2.89.3 running on Oracle JRE 8u161 for x64 Linux: proxy configuration check succeeds, checking for plugins on Jenkins updates produces a stack trace about HTTP error 407.

          Alexey Marin added a comment - I can confirm that I have the same problem with Jenkins 2.89.3 running on Oracle JRE 8u161 for x64 Linux: proxy configuration check succeeds, checking for plugins on Jenkins updates produces a stack trace about HTTP error 407.

          User Name added a comment - - edited

          Jenkins 2.101 here.
          Proxy Settings OK
          Update Center / Advanced  Proxy / Test OK
          wget from within the docker container (when setting the proxy there): OK
          wget from the host machine OK
          Trying to get the plugin update infos - fails.

          Proxy doesn't need an authentification.

          User Name added a comment - - edited Jenkins 2.101 here. Proxy Settings OK Update Center / Advanced  Proxy / Test OK wget from within the docker container (when setting the proxy there): OK wget from the host machine OK Trying to get the plugin update infos - fails. Proxy doesn't need an authentification.

          Gerald Ortner added a comment -

          Now I saw the difference of fetching the updates: The machine with the older JRE version checks '[*http://updates.jenkins-ci.org/update-center.json*'] and the newer one (fresh installled Jenklins 2.89.2) calls '*https://updates.jenkins.io/update-center.json'*!

          So this exlains the different behavier but not what's going on in the newer Jenkins software!

          bg Gerald

          Gerald Ortner added a comment - Now I saw the difference of fetching the updates: The machine with the older JRE version checks ' [*http://updates.jenkins-ci.org/update-center.json*']  and the newer one (fresh installled Jenklins 2.89.2) calls '* https://updates.jenkins.io/update-center.json'* ! So this exlains the different behavier but not what's going on in the newer Jenkins software! bg Gerald

          User Name added a comment -

          Jenkins Docker Container with V2.101 and Java 1.8.0.151 still an issue

          User Name added a comment - Jenkins Docker Container with V2.101 and Java 1.8.0.151 still an issue

          WORKAROUND:

          This is crazy, but will work...

          Run the following in the script console:

          ProxyConfiguration.open(new java.net.URL("http://jenkins.io")).content

          Then the update center should work... unclear why this works at this point in time!

          Stephen Connolly added a comment - WORKAROUND: This is crazy, but will work... Run the following in the script console: ProxyConfiguration.open( new java.net.URL( "http: //jenkins.io" )).content Then the update center should work... unclear why this works at this point in time!

          Interestingly...

          ProxyConfiguration.open(new java.net.URL("https://jenkins.io")).content

          Will give the 

          java.io.IOException: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 407 Proxy Authentication Required"

          error... looks like a JDK bug whereby the HttpsURLConnectionImpl wherein it will fail to setup a connection to the proxy but will reuse an existing one

           

          Stephen Connolly added a comment - Interestingly... ProxyConfiguration.open( new java.net.URL( "https: //jenkins.io" )).content Will give the  java.io.IOException: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 407 Proxy Authentication Required" error... looks like a JDK bug whereby the HttpsURLConnectionImpl wherein it will fail to setup a connection to the proxy but will reuse an existing one  

          Ok, so Http URLs will populate the sun.net.www.protocol.http.AuthenticationInfo.cache.hashtable but Https URLs will not populate, only consult

          Stephen Connolly added a comment - Ok, so Http URLs will populate the sun.net.www.protocol.http.AuthenticationInfo.cache.hashtable but Https URLs will not populate, only consult

          FYI the workaround may only last for a short-ish period of time as the cache seems to get purged under circumstances I have been unable to determine

          Stephen Connolly added a comment - FYI the workaround may only last for a short-ish period of time as the cache seems to get purged under circumstances I have been unable to determine

          Same issue with 2.107.2 version 

          Israel Romero Fijo added a comment - Same issue with 2.107.2 version 

          Harry G. added a comment -

          For us it is fixed, probably either by removal of a duplicate update center we had (same update center twice, disappeared after update and restart) or with the update to 2.107.3 (possibly related in the changelog: "Add new update center root CA certificate. (INFRA-1502)" ?)

          Harry G. added a comment - For us it is fixed, probably either by removal of a duplicate update center we had (same update center twice, disappeared after update and restart) or with the update to 2.107.3 (possibly related in the changelog: "Add new update center root CA certificate. (INFRA-1502)" ?)

          Jorge Yanes added a comment - - edited

          Still happens to me with 2.121.3, behind a proxy that requires NTLM or basic auth. I am using basic auth, but the password expired. I updated my proxy settings and although the proxy test check succeeds, the plugin manager was getting a 407 when checking for updates. I issued  

          ProxyConfiguration.open(new java.net.URL("http://jenkins.io")).content
          

          and that refreshed the caches and it's now working; I guess it will work until the password expires again. EDIT: never mind, restarting jenkins causes the 407 again, even if the credentials are right.

          Please don't dismiss this bug yet. How about adding a simple "ping" to an http resource to force the cache refresh?

          Jorge Yanes added a comment - - edited Still happens to me with 2.121.3, behind a proxy that requires NTLM or basic auth. I am using basic auth, but the password expired. I updated my proxy settings and although the proxy test check succeeds, the plugin manager was getting a 407 when checking for updates. I issued   ProxyConfiguration.open( new java.net.URL( "http: //jenkins.io" )).content and that refreshed the caches and it's now working; I guess it will work until the password expires again. EDIT: never mind, restarting jenkins causes the 407 again, even if the credentials are right. Please don't dismiss this bug yet. How about adding a simple "ping" to an http resource to force the cache refresh?

          Same issue with 2.138.2 version

          Fukusuke Takahashi added a comment - Same issue with 2.138.2 version

          Israel Romero Fijo added a comment - - edited

          I respond to myself here to help anyone with same issue.

          The problem wasn't Jenkins. Problem comes from Java JDK8 and above. Since this version, Oracle has disabled basic auth tunnelling for Basic authentication. If you want to use Basic mode again (security will be compromised), execute Jenkins adding this parameter to your JAVA_OPTS variable:

             "-Djdk.http.auth.tunneling.disabledSchemes="

          default value is

            "-Djdk.http.auth.tunneling.disabledSchemes=Basic", so Basic Schema is DISABLED by default and will return HTTP 407 error even if your credentials were OK.

           Proxy checking must use other method to validate Internet connection...

          I hope this can help you, guys.

           

          Israel Romero Fijo added a comment - - edited I respond to myself here to help anyone with same issue. The problem wasn't Jenkins. Problem comes from Java JDK8 and above. Since this version, Oracle has disabled basic auth tunnelling for Basic authentication. If you want to use Basic mode again (security will be compromised), execute Jenkins adding this parameter to your JAVA_OPTS variable:    "-Djdk.http.auth.tunneling.disabledSchemes=" default value is   "-Djdk.http.auth.tunneling.disabledSchemes=Basic", so Basic Schema is DISABLED by default and will return HTTP 407 error even if your credentials were OK.  Proxy checking must use other method to validate Internet connection... I hope this can help you, guys.  

          Removing myself as assignee. My current work assignments do not provide sufficient bandwidth to review these issues and in the majority of cases I am only assigned by virtue of being the default assignee. For the credentials-api and scm-api related plugins I have permission to allocate time reviewing changes to these APIs themselves to ensure these APIs remain cohesive, but that can be handled through PR reviews rather than assigning issues in JIRA

          Stephen Connolly added a comment - Removing myself as assignee. My current work assignments do not provide sufficient bandwidth to review these issues and in the majority of cases I am only assigned by virtue of being the default assignee. For the credentials-api and scm-api related plugins I have permission to allocate time reviewing changes to these APIs themselves to ensure these APIs remain cohesive, but that can be handled through PR reviews rather than assigning issues in JIRA

          Postponing backport until 2.150.2 at least to give it some soak time.

          Oliver Gondža added a comment - Postponing backport until 2.150.2 at least to give it some soak time.

          Daniel Beck added a comment -

          This introduced JENKINS-54903.

          Daniel Beck added a comment - This introduced JENKINS-54903 .

            Unassigned Unassigned
            go Gerald Ortner
            Votes:
            7 Vote for this issue
            Watchers:
            15 Start watching this issue

              Created:
              Updated:
              Resolved: