Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-68334

JavaMail 1.6 prevents properly mail rendering with Axis2 + JavaMail 1.4

      I'm hosting a SOAP web service based on pretty old Axis2 1.6.2, which bundles JavaMail 1.4.2. That web service sends mails of type "multipart/mixed" consisting of textual body in plain text and attachments in different formats. I have automatic tests running in Jenkins rendering and sending those mails, but am not looking into those too often.

      Recently I recognized that since around February this year all of those mails are broken, which means while those are sent, they are entirely empty. No textual body, no attachments, the content is simply missing. Looking at my mail history, I can see that this worked in the past and I most likely updated to a newer Jenkins LTS around February this year. The plugins in question have been added only some months ago as well

      https://plugins.jenkins.io/javax-mail-api/#releases
      https://plugins.jenkins.io/javax-activation-api/#releases

      After a lot of debugging, I finally found the issue to be related to JavaMail 1.6 provided by Jenkins: Deleting Jenkins from my Tomcat makes mail sending succeed again, after putting it back in place the content is gone again. The problem seems to be with "com.sun.mail.handlers.multipart_mixed.writeTo", though I'm not too sure yet which of both available classes is loaded. Either the one provided by Jenkins or Axis2. Anyway, both contain some sort of the following check and that seems to silently fail in my case, resulting in empty mail contents:

      For 1.4:

         /**
          * Write the object to the output stream, using the specific MIME type.
          */
         public void writeTo(Object obj, String mimeType, OutputStream os) 
      		throws IOException {
      if (obj instanceof MimeMultipart) {
          try {
      	((MimeMultipart)obj).writeTo(os);
          } catch (MessagingException e) {
      	throw new IOException(e.toString());
          }
      }
         }
      

      For 1.6:

          /**
           * Write the object to the output stream, using the specific MIME type.
           */
          @Override
          public void writeTo(Object obj, String mimeType, OutputStream os) 
      			throws IOException {
      	if (obj instanceof Multipart) {
      	    try {
      		((Multipart)obj).writeTo(os);
      	    } catch (MessagingException e) {
      		IOException ioex =
      		    new IOException("Exception writing Multipart");
      		ioex.initCause(e);
      		throw ioex;
      	    }
      	}
          }
      

      The problem most likely is either "MimeMultipart" being something completely unexpected according to the Eclipse Debugger (com.sun.xml.internal.messaging.saaj.packaging.mime.internet.MimeMultipart), or that either "MimeMultipart" or "Multipart" come from different classloaders. Axis2 has some classloader handling on its own as well. While I'm unable to update Axis2 anyway, if it's a classloader issue in combination with Jenkins, I doubt an update would solve the problem.

      I had a deeper look about how this JavaMail stuff works, especially about how "multipart_mixed" gets chosen and found the class MailcapCommandMap and the associated file "mailcap". The latter is bundled in the JAR of JavaMail and describes the class to use to render multipart/mixed-mails:

      multipart/*;;		x-java-content-handler=com.sun.mail.handlers.multipart_mixed; x-java-fallback-entry=true
      

      That is of interest to me, because in theory it provides a way to make my own implementation being used to render that content. But here is again where your plugin comes into play: Your own JavaMail-JAR is always found first and the activation mechanism results in only your bundled file being considered at all. It doesn't make any difference if I include a modified copy of that file in my own JARs.

      > -Djavax.activation.debug=true
      
      
      > MailcapCommandMap: getResources
      > MailcapCommandMap: URL jar:file:/C:/Users/jenkins/.jenkins/plugins/javax-mail-api/WEB-INF/lib/javax.mail-1.6.2.jar!/META-INF/mailcap
      > new MailcapFile: InputStream
      

      The only other way to change the contents of the file at runtime is by using the API of MailcapCommandMap. And while that doesn't read too difficult, your plugin is standing in the way again because you are replacing the default instance with your own delegating one, actually hiding the otherwise available instance of MailcapCommandMap this way.

      https://github.com/jenkinsci/javax-mail-api-plugin/blob/master/src/main/java/io/jenkins/plugins/javax/activation/DelegatingCommandMap.java
      https://javadoc.jenkins-ci.org/plugin/javax-mail-api/

      > // <faultcode>soapenv:Server</faultcode>
      > // <faultstring>io.jenkins.plugins.javax.activation.DelegatingCommandMap cannot be cast to javax.activation.MailcapCommandMap</faultstring>
      

      So right now I'm stuck and need to have a look at getting the formerly available instance of MailcapCommandMap from your wrapper using reflection or some kind. From my point of view, hiding the MailcapCommandMap is wrong and your wrapper should instead extend MailcapCommandMap and not the base class. Your JAR is bundling the mailcap-file, so the instance of MailcapCommandMap can safely be assumed being available like it could be assumed for decades with the JavaMail-JAR.

      I don't see any other way to fix this, even if you make the delegate accessible using some getter, people wouldn't know unless they would implement against your API. Which isn't possible, because Jenkins is simply not used in production. The only solution is to make your instance fulfill the cast and being fully transparent that way.

      When I'm able to register some different class to be used for multipart/mixed at runtime, I could 1. change the implementation of multipart_mixed.writeTo however I like and 2. take care of possible classloader issues because of Jenkins providing JavaMail 1.6 myself most likely as well.

      Thanks!

          [JENKINS-68334] JavaMail 1.6 prevents properly mail rendering with Axis2 + JavaMail 1.4

          Basil Crow added a comment -

          I would suggest running Jenkins in a separate JVM with java -jar jenkins.war rather than in Tomcat. We do not test the WAR with Tomcat anyway, and we may drop support for such a configuration in the future.

          Basil Crow added a comment - I would suggest running Jenkins in a separate JVM with java -jar jenkins.war rather than in Tomcat. We do not test the WAR with Tomcat anyway, and we may drop support for such a configuration in the future.

          Basil Crow added a comment -

          If you want to open a pull request to javax-activation-api-plugin and it doesn't break Jenkins in any way, I'd be happy to accept it.

          Basil Crow added a comment - If you want to open a pull request to javax-activation-api-plugin and it doesn't break Jenkins in any way, I'd be happy to accept it.

          It's a bit funny that you are mentioning not to break Jenkins, while Jenkins actually breaks anything else with the current implementation... I did some more testing and making the wrapped MailcapCommandMap available to add a custom handler doesn't seem to change a thing. The root cause really seems to be that DelegatingCommandMap forcefully sets its own classloader:

              private static final <T> T runWithContextClassLoader(Supplier<T> supplier) {
                  Thread t = Thread.currentThread();
                  ClassLoader orig = t.getContextClassLoader();
                  t.setContextClassLoader(DelegatingCommandMap.class.getClassLoader());
                  try {
                      return supplier.get();
                  } finally {
                      t.setContextClassLoader(orig);
                  }
              }
          

          https://github.com/jenkinsci/javax-mail-api-plugin/blob/ad463e503451c90be259cc521e5ec106f7498cea/src/main/java/io/jenkins/plugins/javax/activation/DelegatingCommandMap.java#L21

          I've implemented multiple different approaches and all of those (somewhat) worked when actually getting DelegatingCommandMap out of the way. The funny thing is that the easiest implementation is retrieving the original MailcapCommandMap from DelegatingCommandMap using reflection and simply putting that back into place during the time my mails are created. Their content is (mostly) the same as expected, while that is NEVER the case with keeping DelegatingCommandMap around. Using custom handlers for e.g. multipart/mixed being part of my project doesn't work, because those are not known to the classloader of Jenkins. Though, registering handlers themself using the wrapped instance of MailcapCommandMap was successful according to the logs, but handlers simply couldn't be created with ClassNotFoundException. I guess this only works with some kind of Jenkins-plugin sharing the same classloader.

          So right now, it seems that Jenkins is simply not compatible with being deployed into Tomcat with different web apps anymore. I wonder why it is necessary to forcefully set the context classloader at all? This is especially of interest because in my environment using the pretty old Axis2 1.6, Axis2 has its own classloader approach to distinguish different web services from each other as well and that and former versions of Jenkins worked perfectly together. My own mails were generated as expected and Jenkins was able to send mails without any problems as well.

          I suggest you reconsider or at least better document why your custom classloader handling is necessary and what that might break in different deployment scenarios.

          That's especially important because I don't see any fully succeeding workaround being possible at all: Mine is a class implementing AutoCloseable spanning the lifetime of my mail creation. In the CTOR, I'm retrieving the MailcapCommandMap from the DelegetaingCommandMap and putting it into place using CommandMap.setDefaultMap. In the close-method I'm putting the DelegatingCommandMap back, just to be sure. The problem is that CommandMap.setDefaultMap maintains some global resource one has only limited control over. While most of my mails render OK most of the time, some are still broken, most likely because Jenkins did something on its own concurrently in the background, like checking other projects for the need to run tests or stuff like that. It might even be sufficient to access the web-UI already. I've tested with two different setups, one running tests automatically and therefore checking automatically for commits etc., the other one running tests manually only. In the latter case, multiple executions of my tests rendered ALL mails successfully, while with automatic tests there ALWAYS were some broken mails.

          Looks like I won't get any closer to a solution and need to reconsider my setup instead. There's some discussion to migrate to GitLab anyway...

          Thorsten Schöning added a comment - It's a bit funny that you are mentioning not to break Jenkins, while Jenkins actually breaks anything else with the current implementation... I did some more testing and making the wrapped MailcapCommandMap available to add a custom handler doesn't seem to change a thing. The root cause really seems to be that DelegatingCommandMap forcefully sets its own classloader: private static final <T> T runWithContextClassLoader(Supplier<T> supplier) { Thread t = Thread .currentThread(); ClassLoader orig = t.getContextClassLoader(); t.setContextClassLoader(DelegatingCommandMap. class. getClassLoader()); try { return supplier.get(); } finally { t.setContextClassLoader(orig); } } https://github.com/jenkinsci/javax-mail-api-plugin/blob/ad463e503451c90be259cc521e5ec106f7498cea/src/main/java/io/jenkins/plugins/javax/activation/DelegatingCommandMap.java#L21 I've implemented multiple different approaches and all of those (somewhat) worked when actually getting DelegatingCommandMap out of the way. The funny thing is that the easiest implementation is retrieving the original MailcapCommandMap from DelegatingCommandMap using reflection and simply putting that back into place during the time my mails are created. Their content is (mostly) the same as expected, while that is NEVER the case with keeping DelegatingCommandMap around. Using custom handlers for e.g. multipart/mixed being part of my project doesn't work, because those are not known to the classloader of Jenkins. Though, registering handlers themself using the wrapped instance of MailcapCommandMap was successful according to the logs, but handlers simply couldn't be created with ClassNotFoundException. I guess this only works with some kind of Jenkins-plugin sharing the same classloader. So right now, it seems that Jenkins is simply not compatible with being deployed into Tomcat with different web apps anymore. I wonder why it is necessary to forcefully set the context classloader at all? This is especially of interest because in my environment using the pretty old Axis2 1.6, Axis2 has its own classloader approach to distinguish different web services from each other as well and that and former versions of Jenkins worked perfectly together. My own mails were generated as expected and Jenkins was able to send mails without any problems as well. I suggest you reconsider or at least better document why your custom classloader handling is necessary and what that might break in different deployment scenarios. That's especially important because I don't see any fully succeeding workaround being possible at all: Mine is a class implementing AutoCloseable spanning the lifetime of my mail creation. In the CTOR, I'm retrieving the MailcapCommandMap from the DelegetaingCommandMap and putting it into place using CommandMap.setDefaultMap. In the close-method I'm putting the DelegatingCommandMap back, just to be sure. The problem is that CommandMap.setDefaultMap maintains some global resource one has only limited control over. While most of my mails render OK most of the time, some are still broken, most likely because Jenkins did something on its own concurrently in the background, like checking other projects for the need to run tests or stuff like that. It might even be sufficient to access the web-UI already. I've tested with two different setups, one running tests automatically and therefore checking automatically for commits etc., the other one running tests manually only. In the latter case, multiple executions of my tests rendered ALL mails successfully, while with automatic tests there ALWAYS were some broken mails. Looks like I won't get any closer to a solution and need to reconsider my setup instead. There's some discussion to migrate to GitLab anyway...

          Basil Crow added a comment -

          See jenkinsci/jenkins#6165 and Dependencies and Class Loading for context. And I already gave you a solution: use java -jar jenkins.war rather than running Jenkins in Tomcat. We do not test such a mode anyway.

          Basil Crow added a comment - See jenkinsci/jenkins#6165 and Dependencies and Class Loading for context. And I already gave you a solution: use java -jar jenkins.war rather than running Jenkins in Tomcat. We do not test such a mode anyway.

            Unassigned Unassigned
            tschoening Thorsten Schöning
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: