Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-22088

Disk space leak with multiple copies of winstone-XXXX.jar in TEMP folder

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • core
    • Windows 8.1 x64, JDK 1.7.0_51, Jenkins 1.553

      Continued operation of Jenkins creates multiple copies of "winstone.jar" with a numeric suffix in TEMP folder (winstone12343543634634.jar). We had about 800Gb of these accumulated, slowing down the host system.

      We are currently running Jenkins directly (`java -jar jenkins.jar`) via Windows scheduled task.

          [JENKINS-22088] Disk space leak with multiple copies of winstone-XXXX.jar in TEMP folder

          Peter Hannon added a comment - - edited

          I am still seeing this issue with Jenkins v 2.121.3. From jenkins release notes: https://jenkins.io/changelog-stable/#v2.107.1 I thought this had been fixed in v2.107.1

          I noticed recently that there were several winstone jars in my /tmp. I manually removed all but the most recent.  I then did a safeRestart of Jenkins and there were then 2 winstone jars in the /tmp directory of the master Jenkins node.

          ls -lrt winstone*.jar
          -rw-r--r-- 1 jenkins users 2146767 2019-03-04 12:23 winstone9180695101677339973.jar
          -rw-r--r-- 1 jenkins users 2146767 2019-04-04 11:21 winstone8639977864837551161.jar
          

          We are running the Jenkins war directly via a shell script

          #!/bin/bash
          nohup /home/jenkins/jdk8/bin/java -jar /home/jenkins/jenkins.war --httpPort=8081 > /home/jenkins/jenkins.log 2>&1 &
          

           

          I'm not sure if it's related but I am also seeing some empty jetty directories with similar timestamps:

          ls -ldrt jetty*
          drwxr-xr-x 2 jenkins users 4096 2019-03-04 12:23 jetty-0.0.0.0-8081-war-_-any-2428298004054280574.dir
          drwxr-xr-x 2 jenkins users 4096 2019-04-04 11:21 jetty-0.0.0.0-8081-war-_-any-7220745071344149958.dir
          

           

           

          Peter Hannon added a comment - - edited I am still seeing this issue with Jenkins v 2.121.3. From jenkins release notes:  https://jenkins.io/changelog-stable/#v2.107.1  I thought this had been fixed in v2.107.1 I noticed recently that there were several winstone jars in my /tmp. I manually removed all but the most recent.  I then did a safeRestart of Jenkins and there were then 2 winstone jars in the /tmp directory of the master Jenkins node. ls -lrt winstone*.jar -rw-r--r-- 1 jenkins users 2146767 2019-03-04 12:23 winstone9180695101677339973.jar -rw-r--r-- 1 jenkins users 2146767 2019-04-04 11:21 winstone8639977864837551161.jar We are running the Jenkins war directly via a shell script #!/bin/bash nohup /home/jenkins/jdk8/bin/java -jar /home/jenkins/jenkins.war --httpPort=8081 > /home/jenkins/jenkins.log 2>&1 &   I'm not sure if it's related but I am also seeing some empty jetty directories with similar timestamps: ls -ldrt jetty* drwxr-xr-x 2 jenkins users 4096 2019-03-04 12:23 jetty-0.0.0.0-8081-war-_-any-2428298004054280574.dir drwxr-xr-x 2 jenkins users 4096 2019-04-04 11:21 jetty-0.0.0.0-8081-war-_-any-7220745071344149958.dir    

          Peter Hannon added a comment -

          ifernandezcalvo thanks for the reply, if you see my comment (the shell script), I'm running this in linux, so the windows service change is not applicable for me.

          Are you saying for a linux environment, this generated winstone jar file needs to be cleaned manually?

          Peter Hannon added a comment - ifernandezcalvo thanks for the reply, if you see my comment (the shell script), I'm running this in linux, so the windows service change is not applicable for me. Are you saying for a linux environment, this generated winstone jar file needs to be cleaned manually?

          ptha sorry, I've replied something unrelated.

          You have to pass an argument to enable the cleanup `--extractedFilesFolder=FOLDER_PATH`

          https://github.com/kuisathaverat/extras-executable-war/blob/8f00f05c22cb4953ed0a034e0d7ef35c0076f99f/src/main/java/Main.java#L229-L232

          Ivan Fernandez Calvo added a comment - ptha sorry, I've replied something unrelated. You have to pass an argument to enable the cleanup `--extractedFilesFolder=FOLDER_PATH` https://github.com/kuisathaverat/extras-executable-war/blob/8f00f05c22cb4953ed0a034e0d7ef35c0076f99f/src/main/java/Main.java#L229-L232

          Peter Hannon added a comment -

          ifernandezcalvo thanks again!

          As per your suggestion, I updated my start shell script to:

           

          #!/bin/bash
          nohup /home/jenkins/jdk8/bin/java -jar /home/jenkins/jenkins.war --httpPort=8081 --extractedFilesFolder=/tmp > /home/jenkins/jenkins.log 2>&1 &
          

          Now the extra winstone jars are cleaned up on startup, I also tested a safeRestart and again there was only 1 winstone jar file (for the running Jenkins).

          So it's solved for me, cheers!

           

          Peter Hannon added a comment - ifernandezcalvo  thanks again! As per your suggestion, I updated my start shell script to:   #!/bin/bash nohup /home/jenkins/jdk8/bin/java -jar /home/jenkins/jenkins.war --httpPort=8081 --extractedFilesFolder=/tmp > /home/jenkins/jenkins.log 2>&1 & Now the extra winstone jars are cleaned up on startup, I also tested a safeRestart and again there was only 1 winstone jar file (for the running Jenkins). So it's solved for me, cheers!  

          This issue is happening with my current Jenkins build 2.156 on windows. Is there any solution for windows for this. 

          Disk is getting full, due to so many winstone*.jar in C:\...\AppData\Local\Temp folder

          Shashank Kulshrestha added a comment - This issue is happening with my current Jenkins build 2.156 on windows. Is there any solution for windows for this.  Disk is getting full, due to so many winstone*.jar in C:\...\AppData\Local\Temp folder

          shashankkulsh Did you read the comment before yours? there you have the solution *--extractedFilesFolder=c:\TMP-FOLDER-PATH*

          Ivan Fernandez Calvo added a comment - shashankkulsh Did you read the comment before yours? there you have the solution * --extractedFilesFolder=c:\TMP-FOLDER-PATH *

          Matt Barna added a comment -

          Is there any plan to fix this? Seems wonky that Jenkins knows where to write the jar, but not where to go to delete it.

          Matt Barna added a comment - Is there any plan to fix this? Seems wonky that Jenkins knows where to write the jar, but not where to go to delete it.

          I just had my Windows 2012R2 Jenkins 2.257 server fall to its knees for the second time in a week after something threw a spanner at Jenkins (started as a service). Jenkins was convinced that it needed to restart itself, which failed due to address already in use, so try again and again and again, dumping a new copy of winstone#.jar each time until out of disk space.

          This bizarre behavior seems to be the result of how the Jenkins service is created by default, first failure: restart, second failure: restart, subsequent failures: restart.

          For now I've set subsequent failures to 'take no action' - hopefully this will ease the pain.

          Stefan Drissen added a comment - I just had my Windows 2012R2 Jenkins 2.257 server fall to its knees for the second time in a week after something threw a spanner at Jenkins (started as a service). Jenkins was convinced that it needed to restart itself, which failed due to address already in use, so try again and again and again, dumping a new copy of winstone#.jar each time until out of disk space. This bizarre behavior seems to be the result of how the Jenkins service is created by default, first failure: restart, second failure: restart, subsequent failures: restart. For now I've set subsequent failures to 'take no action' - hopefully this will ease the pain.

          I am currently experiencing this issue with a jenkins 2.401.1 (LTS) running on Windows server 2016. Still seems to be an active issue...

          Christoph Fetzer added a comment - I am currently experiencing this issue with a jenkins 2.401.1 (LTS) running on Windows server 2016. Still seems to be an active issue...

          The issue was fixed and nobody report a hit in five years, Could you open a new one with the details of your environment and reference this one?

          Ivan Fernandez Calvo added a comment - The issue was fixed and nobody report a hit in five years, Could you open a new one with the details of your environment and reference this one?

            ifernandezcalvo Ivan Fernandez Calvo
            t0yv0 Anton Tayanovskyy
            Votes:
            20 Vote for this issue
            Watchers:
            34 Start watching this issue

              Created:
              Updated:
              Resolved: