Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-3696

Dead Node Causes Exceptions w/ Node Config Site

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved (View Workflow)
    • Critical
    • Resolution: Fixed
    • core
    • None
    • Platform: Sun, OS: SunOS

    Description

      We encountered an issue causing one of our test nodes to fall into the Dead
      state. Our theory at the moment is that some one was trying to reconfigure a
      node while it was in use, specifically we think they tried to rename it. We
      believe that caused the test node to get into the dead state. After that
      happened we were unable to access the node configuration pages for any of the
      test nodes in the system. When ever we try to access the configuration page for
      a node we get a status code 500 page with a stack trace which I will include below.

      We are running everything on SunOS 10, version 1.303 of Hudson, and we use ssh
      to connect our slaves.

      I am pasting three excerpts from our logs. The first is the output from
      nohup.out which I believe shows the exceptions that occurred while some one was
      configuring one of the test nodes. The second section of logs is from hudson.log
      which shows more exceptions with regards to the test node. The third log is the
      error the client receives in the browser.

      --------------------
      NOHUP.OUT
      --------------------

      May 15, 2009 10:55:15 AM hudson.slaves.SlaveComputer tryReconnect
      INFO: Attempting to reconnect qatest28a.qa - ACE I18N
      Exception in thread "Executor #0 for qatest28a.qa" java.lang.NullPointerException
      	at hudson.slaves.SlaveComputer.taskCompleted(SlaveComputer.java:206)
      	at hudson.model.Executor.run(Executor.java:130)
      May 15, 2009 10:55:25 AM hudson.slaves.CommandLauncher launch
      INFO: slave agent launched for qatest28a.qa - ACE I18N
      May 15, 2009 10:55:35 AM hudson.triggers.SafeTimerTask run
      SEVERE: Timer task hudson.slaves.ComputerRetentionWork@1d8b7abe failed
      java.lang.NullPointerException
      	at hudson.slaves.SlaveComputer.getRetentionStrategy(SlaveComputer.java:400)
      	at hudson.slaves.ComputerRetentionWork.doRun(ComputerRetentionWork.java:63)
      	at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54)
      	at java.util.TimerThread.mainLoop(Timer.java:512)
      	at java.util.TimerThread.run(Timer.java:462)
      May 15, 2009 10:56:35 AM hudson.triggers.SafeTimerTask run
      SEVERE: Timer task hudson.slaves.ComputerRetentionWork@1d8b7abe failed
      java.lang.NullPointerException
      	at hudson.slaves.SlaveComputer.getRetentionStrategy(SlaveComputer.java:400)
      	at hudson.slaves.ComputerRetentionWork.doRun(ComputerRetentionWork.java:63)
      	at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54)
      	at java.util.TimerThread.mainLoop(Timer.java:512)
      	at java.util.TimerThread.run(Timer.java:462)
      May 15, 2009 10:56:56 AM hudson.ExpressionFactory2$JexlExpression evaluate
      WARNING: Caught exception evaluating: it.manualLaunchAllowed. Reason:
      java.lang.reflect.InvocationTargetException
      java.lang.reflect.InvocationTargetException
      	at sun.reflect.GeneratedMethodAccessor570.invoke(Unknown Source)
      	at
      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      	at java.lang.reflect.Method.invoke(Method.java:585)
      	at org.apache.commons.jexl.util.PropertyExecutor.execute(PropertyExecutor.java:125)
      	at
      org.apache.commons.jexl.util.introspection.UberspectImpl$VelGetterImpl.invoke(UberspectImpl.java:314)
      	at
      org.apache.commons.jexl.parser.ASTArrayAccess.evaluateExpr(ASTArrayAccess.java:185)
      	at org.apache.commons.jexl.parser.ASTIdentifier.execute(ASTIdentifier.java:75)
      	at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83)
      	at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57)
      	at
      org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:51)
      	at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80)
      	at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:71)
      	at
      org.apache.commons.jelly.expression.ExpressionSupport.evaluateRecurse(ExpressionSupport.java:61)
      	at
      org.apache.commons.jelly.expression.ExpressionSupport.evaluateAsBoolean(ExpressionSupport.java:71)
      	at org.apache.commons.jelly.tags.core.IfTag.doTag(IfTag.java:41)
      	at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:262)
      	at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      	at
      org.kohsuke.stapler.jelly.CustomTagLibrary$StaplerDynamicTag$1.run(CustomTagLibrary.java:147)
      	at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
      	at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:262)
      	at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      	at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:186)
      	at org.apache.commons.jelly.tags.core.IfTag.doTag(IfTag.java:42)
      	at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:262)
      	at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      	at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:186)
      	at org.apache.commons.jelly.tags.core.JellyTag.doTag(JellyTag.java:45)
      	at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:262)
      	at org.apache.commons.jelly.impl.DynamicTag.doTag(DynamicTag.java:81)
      	at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:262)
      	at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      	at
      org.kohsuke.stapler.jelly.CustomTagLibrary$StaplerDynamicTag$1.run(CustomTagLibrary.java:147)
      	at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
      	at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:262)
      	at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      	at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:186)
      	at org.apache.commons.jelly.impl.StaticTag.doTag(StaticTag.java:65)
      	at org.apache.commons.jelly.impl.StaticTagScript.run(StaticTagScript.java:112)
      	at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      	at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:186)
      	at org.apache.commons.jelly.impl.StaticTag.doTag(StaticTag.java:65)
      	at org.apache.commons.jelly.impl.StaticTagScript.run(StaticTagScript.java:112)
      	at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      	at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:186)
      	at org.apache.commons.jelly.impl.StaticTag.doTag(StaticTag.java:65)
      	at org.apache.commons.jelly.impl.StaticTagScript.run(StaticTagScript.java:112)
      	at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      	at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:186)
      	at org.apache.commons.jelly.impl.StaticTag.doTag(StaticTag.java:65)
      	at org.apache.commons.jelly.impl.StaticTagScript.run(StaticTagScript.java:112)
      	at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      	at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:186)
      	at org.apache.commons.jelly.impl.StaticTag.doTag(StaticTag.java:65)
      	at org.apache.commons.jelly.impl.StaticTagScript.run(StaticTagScript.java:112)
      	at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      	at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:186)
      	at org.apache.commons.jelly.tags.core.JellyTag.doTag(JellyTag.java:45)
      	at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:262)
      	at org.apache.commons.jelly.impl.DynamicTag.doTag(DynamicTag.java:81)
      	at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:262)
      	at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      	at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:186)
      	at org.apache.commons.jelly.tags.core.JellyTag.doTag(JellyTag.java:45)
      	at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:262)
      	at
      org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:29)
      	at
      org.kohsuke.stapler.jelly.JellyClassTearOff.serveIndexJelly(JellyClassTearOff.java:43)
      	at org.kohsuke.stapler.jelly.JellyFacet.handleIndexRequest(JellyFacet.java:75)
      	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:475)
      	at org.kohsuke.stapler.MetaClass$8.dispatch(MetaClass.java:221)
      	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:486)
      	at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:150)
      	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:30)
      	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:486)
      	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:404)
      	at org.kohsuke.stapler.Stapler.service(Stapler.java:117)
      	at javax.servlet.http.HttpServlet.service(HttpServlet.java:45)
      	at winstone.ServletConfiguration.execute(ServletConfiguration.java:249)
      	at winstone.RequestDispatcher.forward(RequestDispatcher.java:335)
      	at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:378)
      	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:91)
      	at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:83)
      	at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
      	at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
      	at hudson.plugins.audit_trail.AuditTrailFilter.doFilter(AuditTrailFilter.java:35)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
      	at
      

      Attachments

        Activity

          I believe this is already fixed.

          kohsuke Kohsuke Kawaguchi added a comment - I believe this is already fixed.

          People

            kohsuke Kohsuke Kawaguchi
            elismaga elismaga
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: