Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-5119

Winstone: Memory leak due to default session timeout of -1

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Blocker Blocker
    • core
    • None
    • Hudson 1.337 (Winstone), Solaris 10 SPARC, Sun JDK 1.6.0_16

      We are running Hudson with its embedded Winstone servlet engine. User authentication within Hudson is done via Hudsons LDAP security realm using our corporate ActiveDirectory via LDAP.

      After running Hudson a few weeks we're running into OutOfMemory exceptions. Here the culprit is a single winstone.WebAppConfiguration instance holding a lot of winstone.WinStoneSession instances each holding an Acegi RememberMeAuthenticationToken instance with 36k data from the authenticated users AD record.

      It seems that Hudson isn't setting Winstones session timeout. Without specifying a session timeout winstone.WebAppConfiguration.makeNewSession() is using a session timeout of -1.

          [JENKINS-5119] Winstone: Memory leak due to default session timeout of -1

          tjuerge added a comment -

          This can be fixed by specifying a session timeout in hudson/main/war/resources/WEB-INF/web.xml:

          Index: resources/WEB-INF/web.xml
          ===================================================================
          --- resources/WEB-INF/web.xml	(revision 24689)
          +++ resources/WEB-INF/web.xml	(working copy)
          @@ -30,6 +30,10 @@
             <display-name>Hudson</display-name>
             <description>Build management system</description>
           
          +  <session-config>
          +    <session-timeout>60</session-timeout>
          +  </session-config>
          +
             <servlet>
               <servlet-name>Stapler</servlet-name>
               <servlet-class>org.kohsuke.stapler.Stapler</servlet-class>
          

          tjuerge added a comment - This can be fixed by specifying a session timeout in hudson/main/war/resources/WEB-INF/web.xml : Index: resources/WEB-INF/web.xml =================================================================== --- resources/WEB-INF/web.xml (revision 24689) +++ resources/WEB-INF/web.xml (working copy) @@ -30,6 +30,10 @@ <display-name>Hudson</display-name> <description>Build management system</description> + <session-config> + <session-timeout>60</session-timeout> + </session-config> + <servlet> <servlet-name>Stapler</servlet-name> <servlet-class>org.kohsuke.stapler.Stapler</servlet-class>

          tjuerge added a comment -

          Answered my question: winstone.HostConfiguration.run() is responsible for calling WebAppCOnfiguration.invalidateExpiredSessions()

          tjuerge added a comment - Answered my question: winstone.HostConfiguration.run() is responsible for calling WebAppCOnfiguration.invalidateExpiredSessions()

          slurdge added a comment -

          We have also the same issue.
          I tried tjuerge patch to web.xml but it did not change anything (by the way I had to remove a sha1 - computed file to do this).
          We have a ActiveDirectory that is configured via LDAP. Every connection leaks a little bit. The problem is worsened by the fact that we use a tray applet that checks the build status. This tray applet authenticates itself with Hudson. I believe that is creating a WinStoneSession each time, leaking memory slowly.
          I've also looked at the memory dump and my findings are the same as tjuerge.
          In one week, Hudson takes the maximum of 512Mb allocated and starts to fail builds.

          Any solution is welcomed, I would be happy to try a build with a fix for this.
          Also, if there is a workaround in forcing the flush of expired sessions, this would allow us to periodically do it and not interrupt operation by restarting Hudson (the only available option so far, but not very good for us).

          slurdge added a comment - We have also the same issue. I tried tjuerge patch to web.xml but it did not change anything (by the way I had to remove a sha1 - computed file to do this). We have a ActiveDirectory that is configured via LDAP. Every connection leaks a little bit. The problem is worsened by the fact that we use a tray applet that checks the build status. This tray applet authenticates itself with Hudson. I believe that is creating a WinStoneSession each time, leaking memory slowly. I've also looked at the memory dump and my findings are the same as tjuerge. In one week, Hudson takes the maximum of 512Mb allocated and starts to fail builds. Any solution is welcomed, I would be happy to try a build with a fix for this. Also, if there is a workaround in forcing the flush of expired sessions, this would allow us to periodically do it and not interrupt operation by restarting Hudson (the only available option so far, but not very good for us).

          We're also seeing a large number of objects 6,143 sessions with 17.9Mb. Can this timeout be added to the standard distribution?

          We're going OOM as often as 3 times a day. When monit restarts Hudson we blow away the expanded WAR directory which messing with the web.xml a bit much pain.

          We are also using the Hudson tray applet. Our primary OOM problem seems to be related to the Cobertura reports, by the way, but this is not a small leak either.

          Hudson ver. 1.353 on Linux, Sun JDK 1.6.0_18

          StevenTheAmusing added a comment - We're also seeing a large number of objects 6,143 sessions with 17.9Mb. Can this timeout be added to the standard distribution? We're going OOM as often as 3 times a day. When monit restarts Hudson we blow away the expanded WAR directory which messing with the web.xml a bit much pain. We are also using the Hudson tray applet. Our primary OOM problem seems to be related to the Cobertura reports, by the way, but this is not a small leak either. Hudson ver. 1.353 on Linux, Sun JDK 1.6.0_18

          VisualVM showing a large number of sessions.

          StevenTheAmusing added a comment - VisualVM showing a large number of sessions.

          Fixed in Winstone 0.9.10-hudson-17 to have the finite default session timeout value. This change still needs to be integrated into Hudson.

          Kohsuke Kawaguchi added a comment - Fixed in Winstone 0.9.10-hudson-17 to have the finite default session timeout value. This change still needs to be integrated into Hudson.

          amrangan added a comment -

          Will this change be integrated into the next release of Hudson (i.e. 1.356) ?

          amrangan added a comment - Will this change be integrated into the next release of Hudson (i.e. 1.356) ?

          Code changed in hudson
          User: : kohsuke
          Path:
          trunk/hudson/main/war/pom.xml
          trunk/www/changelog.html
          http://jenkins-ci.org/commit/30797
          Log:
          [FIXED JENKINS-5119] integrated a newer version of Winstone with memory leak fix and SSL certificate handling improvement.

          SCM/JIRA link daemon added a comment - Code changed in hudson User: : kohsuke Path: trunk/hudson/main/war/pom.xml trunk/www/changelog.html http://jenkins-ci.org/commit/30797 Log: [FIXED JENKINS-5119] integrated a newer version of Winstone with memory leak fix and SSL certificate handling improvement.

          Code changed in hudson
          User: : kohsuke
          Path:
          trunk/hudson/main/war/pom.xml
          trunk/www/changelog.html
          http://jenkins-ci.org/commit/33533
          Log:
          [FIXED JENKINS-5119] integrated another Winstone that fixes session object leak.

          SCM/JIRA link daemon added a comment - Code changed in hudson User: : kohsuke Path: trunk/hudson/main/war/pom.xml trunk/www/changelog.html http://jenkins-ci.org/commit/33533 Log: [FIXED JENKINS-5119] integrated another Winstone that fixes session object leak.

          dogfood added a comment -

          Integrated in hudson_main_trunk #154
          [FIXED JENKINS-5119] integrated another Winstone that fixes session object leak.

          kohsuke :
          Files :

          • /trunk/hudson/main/war/pom.xml
          • /trunk/www/changelog.html

          dogfood added a comment - Integrated in hudson_main_trunk #154 [FIXED JENKINS-5119] integrated another Winstone that fixes session object leak. kohsuke : Files : /trunk/hudson/main/war/pom.xml /trunk/www/changelog.html

            Unassigned Unassigned
            tjuerge tjuerge
            Votes:
            5 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: