Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-63506

Unable to update plugins (ssl error downloading plugin)

    • Icon: Bug Bug
    • Resolution: Won't Fix
    • Icon: Critical Critical
    • core
    • None
    • Jenkins 2.254 running on CentOS 7 (OpenJDK8) with macOS agents

      Recently, the plugin update functionality is returning the following error for all plugins I try to update when trying to download and installed the plugin updates:

      javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
      	at sun.security.ssl.Alerts.getSSLException(Alerts.java:198)
      	at sun.security.ssl.Alerts.getSSLException(Alerts.java:159)
      	at sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:2041)
      	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1145)
      	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1388)
      	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1416)
      	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1400)
      	at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
      	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
      	at sun.net.www.protocol.http.HttpURLConnection.followRedirect0(HttpURLConnection.java:2739)
      	at sun.net.www.protocol.http.HttpURLConnection.followRedirect(HttpURLConnection.java:2651)
      	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1830)
      	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
      	at sun.net.www.protocol.http.HttpURLConnection.getHeaderField(HttpURLConnection.java:3061)
      	at java.net.URLConnection.getHeaderFieldLong(URLConnection.java:629)
      	at java.net.URLConnection.getContentLengthLong(URLConnection.java:501)
      	at java.net.URLConnection.getContentLength(URLConnection.java:485)
      	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getContentLength(HttpsURLConnectionImpl.java:412)
      	at hudson.model.UpdateCenter$UpdateCenterConfiguration.download(UpdateCenter.java:1264)
      Caused: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
      	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
      	at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1950)
      	at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1945)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1944)
      	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1514)
      	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
      	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:268)
      	at hudson.model.UpdateCenter$UpdateCenterConfiguration.download(UpdateCenter.java:1280)
      Caused: java.io.IOException: Failed to load https://updates.jenkins.io/download/plugins/plugin-util-api/1.2.3/plugin-util-api.hpi to /var/lib/jenkins/plugins/plugin-util-api.jpi.tmp
      	at hudson.model.UpdateCenter$UpdateCenterConfiguration.download(UpdateCenter.java:1287)
      Caused: java.io.IOException: Failed to download from https://updates.jenkins.io/download/plugins/plugin-util-api/1.2.3/plugin-util-api.hpi (redirected to: https://get.jenkins.io/plugins/plugin-util-api/1.2.3/plugin-util-api.hpi)
      	at hudson.model.UpdateCenter$UpdateCenterConfiguration.download(UpdateCenter.java:1321)
      	at hudson.model.UpdateCenter$DownloadJob._run(UpdateCenter.java:1869)
      	at hudson.model.UpdateCenter$InstallationJob._run(UpdateCenter.java:2147)
      	at hudson.model.UpdateCenter$DownloadJob.run(UpdateCenter.java:1843)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at hudson.remoting.AtmostOneThreadExecutor$Worker.run(AtmostOneThreadExecutor.java:118)
      	at java.lang.Thread.run(Thread.java:748)
      

      I do not see any change nor have been informed of any company network configuration change (where the CentOS based VM with the Jenkins root node installed on is located) that could explain it.

      The core of the error seems to be this part (it might be a red herring though... has it always been redirecting?): "Caused: java.io.IOException: Failed to download from https://updates.jenkins.io/download/plugins/plugin-util-api/1.2.3/plugin-util-api.hpi (*redirected to: https://get.jenkins.io/plugins*/plugin-util-api/1.2.3/plugin-util-api.hpi)"

      Edit: the "preparation checks" Jenkins does before attempting to update each plugin succeed and I had been able to update plugins regularly up until this week:

      "Preparation	
          Checking internet connectivity
          Checking update center connectivity
          Success"
      

          [JENKINS-63506] Unable to update plugins (ssl error downloading plugin)

          Mark Waite added a comment -

          dshanks you may want to review all the comments in case one of them helps you with your investigation. I'm reasonably confident the issue is not something that the Jenkins project can resolve for you.

          Mark Waite added a comment - dshanks you may want to review all the comments in case one of them helps you with your investigation. I'm reasonably confident the issue is not something that the Jenkins project can resolve for you.

          G'day, I had the exact same issue (centos7, jdk8, jenkins-stable) but was reluctant to upgrade from jdk8 to 11.

          Turns out we had modified our `java.security` file sometime in the past, so yum was carefully putting the updated policy file (configured with all the newer ciphers,etc) in `java.security.rpmnew`. My fix was to overwrite our `java.security` file with the up to date `java.security.rpmnew`. You may want to see what changes you made to your original and translate them across.

          Might also be worth checking for any other left over `.rpmnew` files in your openjdk directory.

          Example fix steps (adjust for your environment):

          $ cd /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64/jre/lib/security/
          
          $ ls
          blacklisted.certs  cacerts  java.policy  java.security  java.security.rpmnew  local_policy.jar  nss.cfg  policy  US_export_policy.jar
          
          # sanity check
          $ diff java.security  java.security.rpmnew
          
          $ mv java.security java.security.20200917
          $ mv java.security.rpmnew java.security
          

          Nick Sonneveld added a comment - G'day, I had the exact same issue (centos7, jdk8, jenkins-stable) but was reluctant to upgrade from jdk8 to 11. Turns out we had modified our `java.security` file sometime in the past, so yum was carefully putting the updated policy file (configured with all the newer ciphers,etc) in `java.security.rpmnew`. My fix was to overwrite our `java.security` file with the up to date `java.security.rpmnew`. You may want to see what changes you made to your original and translate them across. Might also be worth checking for any other left over `.rpmnew` files in your openjdk directory. Example fix steps (adjust for your environment): $ cd /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64/jre/lib/security/ $ ls blacklisted.certs cacerts java.policy java.security java.security.rpmnew local_policy.jar nss.cfg policy US_export_policy.jar # sanity check $ diff java.security java.security.rpmnew $ mv java.security java.security.20200917 $ mv java.security.rpmnew java.security

          I don't know if this is worth documenting in the Jenkins LTS Upgrade Guide? Or performing a sanity check before initiating an update download? These new infrastructure changes might trip up people who have long-lived jenkins servers but not realise their package manager hasn't been keeping their modified java security policy up to date.

          Nick Sonneveld added a comment - I don't know if this is worth documenting in the Jenkins LTS Upgrade Guide? Or performing a sanity check before initiating an update download? These new infrastructure changes might trip up people who have long-lived jenkins servers but not realise their package manager hasn't been keeping their modified java security policy up to date.

          Mark Waite added a comment -

          I like the idea very much sonneveld.

          Is the existence of `java.security.rpmnew` a reliable indicator that the RPM package manager detected user changes and intentionally avoided overwriting the user changes?

          Is there a Java command that could be executed to show that the current java.security configuration does not recognize the SSL certificate from https://updates.jenkins.io/ and could then be used to confirm that the updated `java.security` file now allows it?

          Mark Waite added a comment - I like the idea very much sonneveld . Is the existence of `java.security.rpmnew` a reliable indicator that the RPM package manager detected user changes and intentionally avoided overwriting the user changes? Is there a Java command that could be executed to show that the current java.security configuration does not recognize the SSL certificate from https://updates.jenkins.io/ and could then be used to confirm that the updated `java.security` file now allows it?

          markewaite I tried to find some more official documentation but the best I could find was  https://www.cl.cam.ac.uk/~jw35/docs/rpm_config.html which describes the .rpmnew/.rpmsave behaviour and https://www.linux.com/news/dealing-rpmnew-and-rpmsave-files/ which offers some advice. Basically an rpm can define config files, and if it was modified (and depending on the options specified) then the config file can be replaced (and the original saved as .rpmsave), or the config file is left as if (and the new config is saved as .rpmnew).

          In regards to a java command, there could be a better option, but I put together a test java program based on some sample code:

          import java.io.BufferedReader;
          import java.io.IOException;
          import java.io.InputStream;
          import java.io.InputStreamReader;
          import java.net.HttpURLConnection;
          import java.net.URL;
          
          public class DownloadWebpageExample {
              public static void main(String[] args) {
                  try  {
                      URL url = new URL("https://updates.jenkins.io/download/plugins/plugin-util-api/1.2.5/plugin-util-api.hpi");
                      HttpURLConnection con = (HttpURLConnection) url.openConnection();
                      readStream(con.getInputStream());
                      // Give output for the command line
                  } catch (Exception e) {
                      e.printStackTrace();
                  }
              }
              private static void  readStream(InputStream in)
              {
                  char[] buf = new char[1024];
                  try (BufferedReader reader = new BufferedReader(new InputStreamReader(in));) {
                      while (reader.read(buf, 0, 1024) != -1) { }
                  } catch (IOException e) {
                      e.printStackTrace();
                  }
              }
          }
          

          And ran like so:

          $ javac DownloadWebpageExample.java
          $ java -Dhttps.proxyHost=proxyhostname  -Dhttps.proxyPort=80  -cp . DownloadWebpageExample
          

           

          Nick Sonneveld added a comment - markewaite I tried to find some more official documentation but the best I could find was  https://www.cl.cam.ac.uk/~jw35/docs/rpm_config.html which describes the .rpmnew/.rpmsave behaviour and https://www.linux.com/news/dealing-rpmnew-and-rpmsave-files/ which offers some advice. Basically an rpm can define config files, and if it was modified (and depending on the options specified) then the config file can be replaced (and the original saved as .rpmsave), or the config file is left as if (and the new config is saved as .rpmnew). In regards to a java command, there could be a better option, but I put together a test java program based on some sample code: import java.io.BufferedReader; import java.io.IOException; import java.io.InputStream; import java.io.InputStreamReader; import java.net.HttpURLConnection; import java.net.URL; public class DownloadWebpageExample { public static void main( String [] args) { try { URL url = new URL( "https: //updates.jenkins.io/download/plugins/plugin-util-api/1.2.5/plugin-util-api.hpi" ); HttpURLConnection con = (HttpURLConnection) url.openConnection(); readStream(con.getInputStream()); // Give output for the command line } catch (Exception e) { e.printStackTrace(); } } private static void readStream(InputStream in) { char [] buf = new char [1024]; try (BufferedReader reader = new BufferedReader( new InputStreamReader(in));) { while (reader.read(buf, 0, 1024) != -1) { } } catch (IOException e) { e.printStackTrace(); } } } And ran like so: $ javac DownloadWebpageExample.java $ java -Dhttps.proxyHost=proxyhostname -Dhttps.proxyPort=80 -cp . DownloadWebpageExample  

          John Rocha added a comment - - edited

          I encountered this problem to, using Fedora20, and java 1.8.0_45-b14. There is no option to go to Java 11 on Fedora 20, So Java upgrade is not a solution for me. Neither is OS upgrade due to government requirements. Options?

           

          I succeeded by changing over to Adoptopenjdk 8.

          This was suggested in a comment, and I found there was a linux archive available that I could use with Fedora. 

          John Rocha added a comment - - edited I encountered this problem to, using Fedora20, and java 1.8.0_45-b14. There is no option to go to Java 11 on Fedora 20, So Java upgrade is not a solution for me. Neither is OS upgrade due to government requirements. Options?   I succeeded by changing over to Adoptopenjdk 8. This was suggested in a comment, and I found there was a linux archive available that I could use with Fedora. 

          Daniel Beck added a comment -

          java 1.8.0_45-b14

          New Jenkins installs have required a newer JRE than this for the last three years, see changelog for 2.77.

          Daniel Beck added a comment - java 1.8.0_45-b14 New Jenkins installs have required a newer JRE than this for the last three years, see changelog for 2.77.

          Tim Jacomb added a comment -

          This is caused by out of date java installations not being able to handle let's encrypt certificates. Update your java version and / or trust store.

          Tim Jacomb added a comment - This is caused by out of date java installations not being able to handle let's encrypt certificates. Update your java version and / or trust store.

          please help me also i am using latest version of jenins and latest ubuntu version 20.04 but still ssl connection problem

          shashank chouksey added a comment - please help me also i am using latest version of jenins and latest ubuntu version 20.04 but still ssl connection problem

          Mark Waite added a comment -

          shashank2244 asking for help on a closed issue that is unlikely related to your issue is not very effective. Most of the people who might be able to help will not see your request.

          The Jenkins community forum https://community.jenkins.io/ is a much better place to ask questions and request help.

          In the case that you're describing, it could be that your operating system needs to be updated to the latest package versions (ca-certificates is a package that can cause this type of message if it is too old). It could be that a corporate proxy is doing some harm. It could be that you're running a Java version that is too old. All those things are not issues with Jenkins and are best handled by posting a good question on the community forum with lots of background information. Others will need to know the Jenkins version you are running, the Java version you are running, and will want your confirmation that you've updated the operating system will all the latest patches. If you are behind a corporate proxy, they may need

          Mark Waite added a comment - shashank2244 asking for help on a closed issue that is unlikely related to your issue is not very effective. Most of the people who might be able to help will not see your request. The Jenkins community forum https://community.jenkins.io/ is a much better place to ask questions and request help. In the case that you're describing, it could be that your operating system needs to be updated to the latest package versions (ca-certificates is a package that can cause this type of message if it is too old). It could be that a corporate proxy is doing some harm. It could be that you're running a Java version that is too old. All those things are not issues with Jenkins and are best handled by posting a good question on the community forum with lots of background information. Others will need to know the Jenkins version you are running, the Java version you are running, and will want your confirmation that you've updated the operating system will all the latest patches. If you are behind a corporate proxy, they may need

            Unassigned Unassigned
            panajev Goffredo Marocchi
            Votes:
            4 Vote for this issue
            Watchers:
            13 Start watching this issue

              Created:
              Updated:
              Resolved: