Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-20682

Starting Jenkins with defined --webroot or JETTY_HOME not working.

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved (View Workflow)
    • Critical
    • Resolution: Duplicate
    • core
    • None
    • - 8Gb RAM
      - 8 CPUs
      - 300Gb disk
      - Ubuntu 12.04
      Jenkins installed via 'apt-get install -y jenkins'

    Description

      For our environment it is required to load webapp into /var/cache/jenkins.
      With previous version (1.509.3) setting /etc/default/jenkins with:
      JENKINS_ARGS="--webroot=/var/cache/jenkins/war --httpPort=$HTTP_PORT --ajp13Port=$AJP_PORT"

      has always loaded webroot/webapp to this location.
      Installing the latest jenkins with 'apt-get install -y jenkins' using our existing /etc/default/jenkins file not passing the --webroot parameter/location. Starting Jenkins with /etc/init.d/jenkins start

      Jenkins:IIRC suggestions to use (command-line or /etc/default/jenkins):
      $ export JETTY_HOME=/var/cache/jenkins/war
      $ export JENKINS_HOME=/home/username/jenkins
      $ /usr/bin/java -Xmx512m -jar /usr/share/jenkins/jenkins-1.540.war --webroot=/var/cache/jenkins/war --httpPort=8080 --ajp13Port=-1

      However, it still continuously loads into: /tmp/jetty-0.0.0.0-8080-jenkins-1.540.war-any/webapp

      /var/log/jenkins/jenkins.log:
      INFO: Extract jar:file:/usr/share/jenkins/jenkins-1.540.war!/ to /tmp/jetty-0.0.0.0-8080-jenkins-1.540.war-any/webapp

      Using previous 1.509.3 jenkins.war (and prior) versions does not have this behavior.

      Is this change by design?
      Can you please provide a workaround so we can specify --webroot or jetty_home, etc.?

      Attachments

        Issue Links

          Activity

            gmarquez GMarquez created issue -
            gmarquez GMarquez made changes -
            Field Original Value New Value
            Description For our environment it is required to load webapp into /var/cache/jenkins.
            With previous version (1.509.3) setting /etc/default/jenkins with:
            JENKINS_ARGS="--webroot=/var/cache/jenkins/war --httpPort=$HTTP_PORT --ajp13Port=$AJP_PORT"

            has always loaded webroot/webapp to this location.
            Installing the latest jenkins with 'apt-get install -y jenkins' using our existing /etc/default/jenkins file not passing the --webroot parameter/location. Starting Jenkins with /etc/init.d/jenkins start

            Jenkins:IIRC suggestions to use (command-line or /etc/default/jenkins):
              $ export JETTY_HOME=/var/cache/jenkins/war
              $ export JENKINS_HOME=/home/username/jenkins
              $ /usr/bin/java -Xmx512m -jar /usr/share/jenkins/jenkins-1.540.war --webroot=/var/cache/jenkins/war --httpPort=8080 --ajp13Port=-1

            However, it still continuously loads into: /tmp/jetty-0.0.0.0-8080-jenkins-1.540.2.war--any-/webapp

            /var/log/jenkins/jenkins.log:
            INFO: Extract jar:file:/usr/share/jenkins/jenkins-1.540.2.war!/ to /tmp/jetty-0.0.0.0-8080-jenkins-1.540.2.war--any-/webapp

            Using previous 1.509.3 jenkins.war (and prior) versions does not have this behavior.

            Is this change by design?
            Can you please provide a workaround so we can specify --webroot or jetty_home, etc.?
            For our environment it is required to load webapp into /var/cache/jenkins.
            With previous version (1.509.3) setting /etc/default/jenkins with:
            JENKINS_ARGS="--webroot=/var/cache/jenkins/war --httpPort=$HTTP_PORT --ajp13Port=$AJP_PORT"

            has always loaded webroot/webapp to this location.
            Installing the latest jenkins with 'apt-get install -y jenkins' using our existing /etc/default/jenkins file not passing the --webroot parameter/location. Starting Jenkins with /etc/init.d/jenkins start

            Jenkins:IIRC suggestions to use (command-line or /etc/default/jenkins):
              $ export JETTY_HOME=/var/cache/jenkins/war
              $ export JENKINS_HOME=/home/username/jenkins
              $ /usr/bin/java -Xmx512m -jar /usr/share/jenkins/jenkins-1.540.war --webroot=/var/cache/jenkins/war --httpPort=8080 --ajp13Port=-1

            However, it still continuously loads into: /tmp/jetty-0.0.0.0-8080-jenkins-1.540.war--any-/webapp

            /var/log/jenkins/jenkins.log:
            INFO: Extract jar:file:/usr/share/jenkins/jenkins-1.540.2.war!/ to /tmp/jetty-0.0.0.0-8080-jenkins-1.540.war--any-/webapp

            Using previous 1.509.3 jenkins.war (and prior) versions does not have this behavior.

            Is this change by design?
            Can you please provide a workaround so we can specify --webroot or jetty_home, etc.?
            gmarquez GMarquez made changes -
            Description For our environment it is required to load webapp into /var/cache/jenkins.
            With previous version (1.509.3) setting /etc/default/jenkins with:
            JENKINS_ARGS="--webroot=/var/cache/jenkins/war --httpPort=$HTTP_PORT --ajp13Port=$AJP_PORT"

            has always loaded webroot/webapp to this location.
            Installing the latest jenkins with 'apt-get install -y jenkins' using our existing /etc/default/jenkins file not passing the --webroot parameter/location. Starting Jenkins with /etc/init.d/jenkins start

            Jenkins:IIRC suggestions to use (command-line or /etc/default/jenkins):
              $ export JETTY_HOME=/var/cache/jenkins/war
              $ export JENKINS_HOME=/home/username/jenkins
              $ /usr/bin/java -Xmx512m -jar /usr/share/jenkins/jenkins-1.540.war --webroot=/var/cache/jenkins/war --httpPort=8080 --ajp13Port=-1

            However, it still continuously loads into: /tmp/jetty-0.0.0.0-8080-jenkins-1.540.war--any-/webapp

            /var/log/jenkins/jenkins.log:
            INFO: Extract jar:file:/usr/share/jenkins/jenkins-1.540.2.war!/ to /tmp/jetty-0.0.0.0-8080-jenkins-1.540.war--any-/webapp

            Using previous 1.509.3 jenkins.war (and prior) versions does not have this behavior.

            Is this change by design?
            Can you please provide a workaround so we can specify --webroot or jetty_home, etc.?
            For our environment it is required to load webapp into /var/cache/jenkins.
            With previous version (1.509.3) setting /etc/default/jenkins with:
            JENKINS_ARGS="--webroot=/var/cache/jenkins/war --httpPort=$HTTP_PORT --ajp13Port=$AJP_PORT"

            has always loaded webroot/webapp to this location.
            Installing the latest jenkins with 'apt-get install -y jenkins' using our existing /etc/default/jenkins file not passing the --webroot parameter/location. Starting Jenkins with /etc/init.d/jenkins start

            Jenkins:IIRC suggestions to use (command-line or /etc/default/jenkins):
              $ export JETTY_HOME=/var/cache/jenkins/war
              $ export JENKINS_HOME=/home/username/jenkins
              $ /usr/bin/java -Xmx512m -jar /usr/share/jenkins/jenkins-1.540.war --webroot=/var/cache/jenkins/war --httpPort=8080 --ajp13Port=-1

            However, it still continuously loads into: /tmp/jetty-0.0.0.0-8080-jenkins-1.540.war--any-/webapp

            /var/log/jenkins/jenkins.log:
            INFO: Extract jar:file:/usr/share/jenkins/jenkins-1.540.war!/ to /tmp/jetty-0.0.0.0-8080-jenkins-1.540.war--any-/webapp

            Using previous 1.509.3 jenkins.war (and prior) versions does not have this behavior.

            Is this change by design?
            Can you please provide a workaround so we can specify --webroot or jetty_home, etc.?
            gmarquez GMarquez added a comment -

            I decided to do some additional tracking of last known working version of jenkins.war where --webroot parameter took effect.
            I downloaded previous versions and ran test to see which last known version set webroot to defined: /var/cache/jenkins folder.

            Last Version to allow --webroot option was: v 1.534
            jenkins-1.534.war - Pass!
            jenkins-1.535.war - Fail!

            INFO: Extract jar:file:/usr/share/jenkins/jenkins-1.535.war!/ to /tmp/jetty-0.0.0.0-8080-jenkins-1.535.war-any/webapp

            So it appears this change was implemented in v.1.535 - 1.540 (current). I ran several test on 1.537, 1.538, 1.539 and same results.

            gmarquez GMarquez added a comment - I decided to do some additional tracking of last known working version of jenkins.war where --webroot parameter took effect. I downloaded previous versions and ran test to see which last known version set webroot to defined: /var/cache/jenkins folder. Last Version to allow --webroot option was: v 1.534 jenkins-1.534.war - Pass! jenkins-1.535.war - Fail! INFO: Extract jar: file:/usr/share/jenkins/jenkins-1.535.war!/ to /tmp/jetty-0.0.0.0-8080-jenkins-1.535.war- any /webapp So it appears this change was implemented in v.1.535 - 1.540 (current). I ran several test on 1.537, 1.538, 1.539 and same results.
            mc1arke Michael Clarke made changes -
            Component/s cvs [ 15486 ]
            sogabe sogabe added a comment -

            changed component.

            sogabe sogabe added a comment - changed component.
            sogabe sogabe made changes -
            Component/s core [ 15593 ]
            Component/s bulk-builder [ 15774 ]
            Component/s copyarchiver [ 15676 ]
            Component/s credentials [ 16523 ]
            Component/s dashboard-view [ 15679 ]
            Component/s disk-usage [ 15537 ]
            Component/s extended-read-permission [ 16109 ]
            Component/s external-monitor-job [ 17123 ]
            Component/s git [ 15543 ]
            Component/s greenballs [ 15603 ]
            Component/s join [ 15635 ]
            Component/s ldap [ 17122 ]
            Component/s leastload [ 17653 ]
            Component/s mailer [ 17522 ]
            Component/s matrix-auth [ 18131 ]
            Component/s nested-view [ 15701 ]
            Component/s pam-auth [ 17502 ]
            Component/s parameterized-trigger [ 15592 ]
            Component/s prescmbuildstep [ 15980 ]
            Component/s project-inheritance [ 17666 ]
            Component/s publish-over-ssh [ 15792 ]
            Component/s rebuild [ 15744 ]
            Component/s role-strategy [ 15758 ]
            Component/s ssh [ 16049 ]
            Component/s ssh-slaves [ 15578 ]
            Component/s timestamper [ 15749 ]
            Component/s view-job-filters [ 15736 ]
            b_dean Ben Dean added a comment -

            This is a big deal. If the operating system is set up to run tmpwatch with cron (our CentOS box is this way), files will start to get deleted from that /tmp/jetty-... folder and then Jenkins won't work correctly. Usually this is just icons and other images being broken in the UI, but sometimes it could be important jars missing.

            b_dean Ben Dean added a comment - This is a big deal. If the operating system is set up to run tmpwatch with cron (our CentOS box is this way), files will start to get deleted from that /tmp/jetty-... folder and then Jenkins won't work correctly. Usually this is just icons and other images being broken in the UI, but sometimes it could be important jars missing.
            b_dean Ben Dean made changes -
            Link This issue is related to JENKINS-18366 [ JENKINS-18366 ]
            gaige Gaige Paulsen made changes -
            Link This issue is related to JENKINS-20858 [ JENKINS-20858 ]
            swestcott Simon Westcott made changes -
            Assignee Simon Westcott [ swestcott ]
            ohtake_tomohiro OHTAKE Tomohiro made changes -
            Link This issue is related to JENKINS-21061 [ JENKINS-21061 ]

            Agree Ben, this it's a big problem.
            Icons are missing and slaves couldn't start automatically, slave.jar is also missing.
            /tmp is really a bad place for the extracted war file

            stefanbrausch Stefan Brausch added a comment - Agree Ben, this it's a big problem. Icons are missing and slaves couldn't start automatically, slave.jar is also missing. /tmp is really a bad place for the extracted war file

            Assigning to Kohsuke (who made the change to Jetty) for comment. I believe that this is an unintended consequence of moving to Jetty.

            Note that this is not just a case of the war being exploded into TEMPDIR but I have observed that it continues to serve images etc from the old --webroot location based on the last non-Jetty version of Jenkins (see JENKINS-20858 ).

            The symptoms are quite subtle and at a minimum this bug needs resolving before the next LTS release.

            oldelvet Richard Mortimer added a comment - Assigning to Kohsuke (who made the change to Jetty) for comment. I believe that this is an unintended consequence of moving to Jetty. Note that this is not just a case of the war being exploded into TEMPDIR but I have observed that it continues to serve images etc from the old --webroot location based on the last non-Jetty version of Jenkins (see JENKINS-20858 ). The symptoms are quite subtle and at a minimum this bug needs resolving before the next LTS release.
            oldelvet Richard Mortimer made changes -
            Assignee Kohsuke Kawaguchi [ kohsuke ]
            kate_taylor Kate Taylor added a comment -

            We are experiencing the same issue on Ubuntu 12.04.2: although we have moved from using /tmp, Jenkins still loses the icons, then the styles, overnight. Any idea of a timescale for this fix, please?

            kate_taylor Kate Taylor added a comment - We are experiencing the same issue on Ubuntu 12.04.2: although we have moved from using /tmp, Jenkins still loses the icons, then the styles, overnight. Any idea of a timescale for this fix, please?
            thewoolleyman2 Chad Woolley added a comment -

            This is a serious issue, especially because https://issues.jenkins-ci.org/browse/JENKINS-20858 renders Jenkins essentially unusable until a reboot. I am having this issue on multiple Jenkins installations.

            Can we please get a fix for this recently introduced major bug prioritized?

            Thanks...

            thewoolleyman2 Chad Woolley added a comment - This is a serious issue, especially because https://issues.jenkins-ci.org/browse/JENKINS-20858 renders Jenkins essentially unusable until a reboot. I am having this issue on multiple Jenkins installations. Can we please get a fix for this recently introduced major bug prioritized? Thanks...

            This issue still opened for me and my teamworkers. It's the same behavior since https://issues.jenkins-ci.org/browse/JENKINS-18366.

            Could you fix this one as soon as possible. Is it possible to up priority ? "Critical seems to be a good level"

            xter Benjamin Dewez added a comment - This issue still opened for me and my teamworkers. It's the same behavior since https://issues.jenkins-ci.org/browse/JENKINS-18366 . Could you fix this one as soon as possible. Is it possible to up priority ? "Critical seems to be a good level"
            oldelvet Richard Mortimer made changes -
            Priority Major [ 3 ] Critical [ 2 ]

            The Jetty documentation shows how the location of the temporary directories can be controlled
            http://www.eclipse.org/jetty/documentation/current/ref-temporary-directories.html

            I just did a quick experiment by adding a call to WebAppContext#setTempDirectory() in HostConfiguration#create()
            https://github.com/jenkinsci/winstone/blob/master/src/java/winstone/HostConfiguration.java

            i.e.

            wac.setTempDirectory(new File("/var/cache/jenkins/temp"));
            

            When jenkins.war was built with a version of winstone.jar containing that the Jetty temporary directory moved away from /tmp into /var/cache/jenkins/temp

            That isn't a full solution to the problem because it needs hooking into the --webroot argument to jenkins.

            There are other issues relating to how it works with multiple webapps (they should not share the same temp directory) and the usage of the temp folder is slightly different (it uses a /webapp subdirectory) to the old native winstone behaviour.

            I don't know the internal plumbing of Winstone/Jenkins well enough to make a stab at a full fix for this.

            oldelvet Richard Mortimer added a comment - The Jetty documentation shows how the location of the temporary directories can be controlled http://www.eclipse.org/jetty/documentation/current/ref-temporary-directories.html I just did a quick experiment by adding a call to WebAppContext#setTempDirectory() in HostConfiguration#create() https://github.com/jenkinsci/winstone/blob/master/src/java/winstone/HostConfiguration.java i.e. wac.setTempDirectory( new File( "/ var /cache/jenkins/temp" )); When jenkins.war was built with a version of winstone.jar containing that the Jetty temporary directory moved away from /tmp into /var/cache/jenkins/temp That isn't a full solution to the problem because it needs hooking into the --webroot argument to jenkins. There are other issues relating to how it works with multiple webapps (they should not share the same temp directory) and the usage of the temp folder is slightly different (it uses a /webapp subdirectory) to the old native winstone behaviour. I don't know the internal plumbing of Winstone/Jenkins well enough to make a stab at a full fix for this.
            julio_r julio rincon added a comment -

            I bumped into this issue trying to run jenkins in my NAS where the tmp is just 30MBs. Overriding the java system property (java.io.tmpdir) worked for me!

            E.g.

            nohup java -Djava.io.tmpdir=./webroot -jar jenkins.war --httpPort=9999 > ./jenkins.log 2>&1 &

            julio_r julio rincon added a comment - I bumped into this issue trying to run jenkins in my NAS where the tmp is just 30MBs. Overriding the java system property (java.io.tmpdir) worked for me! E.g. nohup java -Djava.io.tmpdir=./webroot -jar jenkins.war --httpPort=9999 > ./jenkins.log 2>&1 &

            The following worked better for me:

            Set the jetty.home to Jenkins directory on the java opts when starting Jenkins

            -Djetty.home=$JENKINS_HOME

            Create a directory called "work" in Jenkins Home directory, or wherever you have set jetty.home

            That will cause Jetty to use that work directory instead of whatever the java.io.tmpdir property is set to.

            gcummings Geoff Cummings added a comment - The following worked better for me: Set the jetty.home to Jenkins directory on the java opts when starting Jenkins -Djetty.home=$JENKINS_HOME Create a directory called "work" in Jenkins Home directory, or wherever you have set jetty.home That will cause Jetty to use that work directory instead of whatever the java.io.tmpdir property is set to.

            See JENKINS-22442
            It should fix the issue reported here.

            oldelvet Richard Mortimer added a comment - See JENKINS-22442 It should fix the issue reported here.

            Duplicate of JENKINS-22442

            oldelvet Richard Mortimer added a comment - Duplicate of JENKINS-22442
            oldelvet Richard Mortimer made changes -
            Resolution Duplicate [ 3 ]
            Status Open [ 1 ] Resolved [ 5 ]
            danielbeck Daniel Beck made changes -
            Link This issue is duplicated by JENKINS-21061 [ JENKINS-21061 ]
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 152491 ] JNJira + In-Review [ 194241 ]

            People

              kohsuke Kohsuke Kawaguchi
              gmarquez GMarquez
              Votes:
              13 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: