Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-8043

"Reload Configuration from Disk" loses labels for swarm-clients

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • swarm-plugin
    • None

        [JENKINS-8043] "Reload Configuration from Disk" loses labels for swarm-clients

        Uwe Stuehler added a comment - - edited

        Could be a duplicate of JENKINS-3896.

        As of 1.439 the swarm clients not only lose their labels after "Reload Configuration from Disk".

        This is what we have observed:

        1. Swarm client nodes appear online in the Build Executor Status, but no new jobs get executed on them.
        2. Visiting the configuration page for any swarm client node results in an error.

        We found two ways to resolve this situation:

        1. Restarting a single slave-client.jar process on any of the swarm client nodes immediately lets the master execute jobs on the other swarm client nodes, as well as on the restarted node once it is back online.
        2. Restarting the Jenkins master resolves the situation as well. All swarm clients reconnect, become online and execute jobs again.

        Uwe Stuehler added a comment - - edited Could be a duplicate of JENKINS-3896 . As of 1.439 the swarm clients not only lose their labels after "Reload Configuration from Disk". This is what we have observed: Swarm client nodes appear online in the Build Executor Status, but no new jobs get executed on them. Visiting the configuration page for any swarm client node results in an error. We found two ways to resolve this situation: Restarting a single slave-client.jar process on any of the swarm client nodes immediately lets the master execute jobs on the other swarm client nodes, as well as on the restarted node once it is back online. Restarting the Jenkins master resolves the situation as well. All swarm clients reconnect, become online and execute jobs again.

        Uwe Stuehler added a comment -

        Backtrace from accessing the configuration page of a swarm client node after reloading the configuration from disk:

        Status Code: 500
        Exception: 
        Stacktrace:
        org.apache.commons.jelly.JellyTagException: jar:file:/var/run/jenkins/war/WEB-INF/lib/jenkins-core-1.439.jar!/hudson/model/Computer/configure.jelly:43:71:  No page found 'configure-entries.jelly' for class hudson.slaves.SlaveComputer
         at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:124)
         at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:270)
         at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
         at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98)
         at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
         at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:270)
         at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
         at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
         at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
         at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
         at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
         at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98)
         at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
         at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:270)
         at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
         at org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98)
         at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
         at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
         at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
         at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
         at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98)
         at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
         at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:270)
         at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
         at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
         at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
         at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
         at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
         at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
         at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
         at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
         at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
         at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
         at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
         at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
         at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
         at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:63)
         at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63)
         at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53)
         at org.kohsuke.stapler.jelly.JellyFacet$1.dispatch(JellyFacet.java:92)
         at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:561)
         at org.kohsuke.stapler.Stapler.invoke(Stapler.java:646)
         at org.kohsuke.stapler.MetaClass$12.dispatch(MetaClass.java:377)
         at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:561)
         at org.kohsuke.stapler.Stapler.invoke(Stapler.java:646)
         at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:196)
         at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
         at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:561)
         at org.kohsuke.stapler.Stapler.invoke(Stapler.java:646)
         at org.kohsuke.stapler.Stapler.invoke(Stapler.java:477)
         at org.kohsuke.stapler.Stapler.service(Stapler.java:159)
         at javax.servlet.http.HttpServlet.service(HttpServlet.java:45)
         at winstone.ServletConfiguration.execute(ServletConfiguration.java:249)
         at winstone.RequestDispatcher.forward(RequestDispatcher.java:335)
         at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:378)
         at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:95)
         at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87)
         at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
         at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
         at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47)
         at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
         at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
         at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
         at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
         at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
         at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166)
         at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
         at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
         at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
         at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
         at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
         at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
         at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
         at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173)
         at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
         at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:61)
         at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
         at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
         at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66)
         at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
         at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
         at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
         at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
         at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
         at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
         at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
         at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
         at winstone.RequestDispatcher.forward(RequestDispatcher.java:333)
         at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:244)
         at winstone.RequestHandlerThread.run(RequestHandlerThread.java:150)
         at java.lang.Thread.run(Thread.java:662)
        
        
        Generated by Winstone Servlet Engine v0.9.10 at Fri Nov 18 16:39:54 CET 2011
        

        Uwe Stuehler added a comment - Backtrace from accessing the configuration page of a swarm client node after reloading the configuration from disk: Status Code: 500 Exception: Stacktrace: org.apache.commons.jelly.JellyTagException: jar:file:/var/run/jenkins/war/WEB-INF/lib/jenkins-core-1.439.jar!/hudson/model/Computer/configure.jelly:43:71: No page found 'configure-entries.jelly' for class hudson.slaves.SlaveComputer at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:124) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:270) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98) at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:270) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119) at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98) at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:270) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98) at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:270) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:63) at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63) at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53) at org.kohsuke.stapler.jelly.JellyFacet$1.dispatch(JellyFacet.java:92) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:561) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:646) at org.kohsuke.stapler.MetaClass$12.dispatch(MetaClass.java:377) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:561) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:646) at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:196) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:561) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:646) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:477) at org.kohsuke.stapler.Stapler.service(Stapler.java:159) at javax.servlet.http.HttpServlet.service(HttpServlet.java:45) at winstone.ServletConfiguration.execute(ServletConfiguration.java:249) at winstone.RequestDispatcher.forward(RequestDispatcher.java:335) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:378) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:95) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87) at winstone.FilterConfiguration.execute(FilterConfiguration.java:195) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47) at winstone.FilterConfiguration.execute(FilterConfiguration.java:195) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:61) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249) at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164) at winstone.FilterConfiguration.execute(FilterConfiguration.java:195) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81) at winstone.FilterConfiguration.execute(FilterConfiguration.java:195) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) at winstone.RequestDispatcher.forward(RequestDispatcher.java:333) at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:244) at winstone.RequestHandlerThread.run(RequestHandlerThread.java:150) at java.lang.Thread.run(Thread.java:662) Generated by Winstone Servlet Engine v0.9.10 at Fri Nov 18 16:39:54 CET 2011

        Marc Günther added a comment - - edited

        Just posted a fix here: https://github.com/jenkinsci/jenkins/pull/408

        The problem is, that swarm clients from the Jenkins.slaves array are not stored into the config.xml (being an EphemeralNode). But when Reloading from Disk, the array is overwritten with only the entries from the file. The swarm clients are then lost from the slaves array, but still exist in the list of computers (Jenkins.computers). This leads to the inconsistent behaviour described above.

        The fix simply remembers all existing swarm clients, and re-adds them to the slaves array after the reload finished.

        Marc Günther added a comment - - edited Just posted a fix here: https://github.com/jenkinsci/jenkins/pull/408 The problem is, that swarm clients from the Jenkins.slaves array are not stored into the config.xml (being an EphemeralNode). But when Reloading from Disk, the array is overwritten with only the entries from the file. The swarm clients are then lost from the slaves array, but still exist in the list of computers (Jenkins.computers). This leads to the inconsistent behaviour described above. The fix simply remembers all existing swarm clients, and re-adds them to the slaves array after the reload finished.

        Code changed in jenkins
        User: Marc Guenther
        Path:
        core/src/main/java/jenkins/model/Jenkins.java
        http://jenkins-ci.org/commit/jenkins/24c31d138fe7b9ff14870b921220bdf709af20cc
        Log:
        [Fixed JENKINS-8043] "Reload Configuration from Disk" broke swarm clients

        When Reloading Configuration from Disk in Jenkins, all swarm clients
        were in an unusable state afterwards (eg. missing their labels).
        This happened because Swarm Clients were not written into the
        configuration file, and got removed when reloading the config.

        This patch re-adds all previously existing swarm clients after
        reloading the config file.

        SCM/JIRA link daemon added a comment - Code changed in jenkins User: Marc Guenther Path: core/src/main/java/jenkins/model/Jenkins.java http://jenkins-ci.org/commit/jenkins/24c31d138fe7b9ff14870b921220bdf709af20cc Log: [Fixed JENKINS-8043] "Reload Configuration from Disk" broke swarm clients When Reloading Configuration from Disk in Jenkins, all swarm clients were in an unusable state afterwards (eg. missing their labels). This happened because Swarm Clients were not written into the configuration file, and got removed when reloading the config. This patch re-adds all previously existing swarm clients after reloading the config file.

        dogfood added a comment -

        Integrated in jenkins_main_trunk #1597
        [Fixed JENKINS-8043] "Reload Configuration from Disk" broke swarm clients (Revision 24c31d138fe7b9ff14870b921220bdf709af20cc)

        Result = SUCCESS
        Seiji Sogabe : 24c31d138fe7b9ff14870b921220bdf709af20cc
        Files :

        • core/src/main/java/jenkins/model/Jenkins.java

        dogfood added a comment - Integrated in jenkins_main_trunk #1597 [Fixed JENKINS-8043] "Reload Configuration from Disk" broke swarm clients (Revision 24c31d138fe7b9ff14870b921220bdf709af20cc) Result = SUCCESS Seiji Sogabe : 24c31d138fe7b9ff14870b921220bdf709af20cc Files : core/src/main/java/jenkins/model/Jenkins.java

        dogfood added a comment -

        Integrated in jenkins_ui-changes_branch #21
        [Fixed JENKINS-8043] "Reload Configuration from Disk" broke swarm clients (Revision 24c31d138fe7b9ff14870b921220bdf709af20cc)

        Result = SUCCESS
        Seiji Sogabe : 24c31d138fe7b9ff14870b921220bdf709af20cc
        Files :

        • core/src/main/java/jenkins/model/Jenkins.java

        dogfood added a comment - Integrated in jenkins_ui-changes_branch #21 [Fixed JENKINS-8043] "Reload Configuration from Disk" broke swarm clients (Revision 24c31d138fe7b9ff14870b921220bdf709af20cc) Result = SUCCESS Seiji Sogabe : 24c31d138fe7b9ff14870b921220bdf709af20cc Files : core/src/main/java/jenkins/model/Jenkins.java

        closing

        Peter Jönsson added a comment - closing

          mindjiver Peter Jönsson
          voorth voorth
          Votes:
          4 Vote for this issue
          Watchers:
          3 Start watching this issue

            Created:
            Updated:
            Resolved: