-
Bug
-
Resolution: Cannot Reproduce
-
Major
-
Jenkins ver. 1.574-SNAPSHOT (git 17d9093)
When tailing the log of a newly launched ssh unix slave, Jenkins suddenly stopped showing the log and instead reported:
javax.servlet.ServletException: java.lang.NumberFormatException: For input string: "5584start=208664" at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:778) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:858) at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:210) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:728) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:858) at org.kohsuke.stapler.MetaClass$12.dispatch(MetaClass.java:390) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:728) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:858) at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:210) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:728) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:858) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:631) at org.kohsuke.stapler.Stapler.service(Stapler.java:225) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:686) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1494) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:96) at com.cloudbees.jenkins.support.slowrequest.SlowRequestFilter.doFilter(SlowRequestFilter.java:37) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:99) at jenkins.metrics.impl.MetricsFilter.doFilter(MetricsFilter.java:117) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:99) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:88) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:48) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:174) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:79) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249) at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:46) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1474) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:533) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:370) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) at org.eclipse.jetty.ajp.Ajp13Connection.access$2900(Ajp13Connection.java:45) at org.eclipse.jetty.ajp.Ajp13Connection$RequestHandler.headerComplete(Ajp13Connection.java:237) at org.eclipse.jetty.ajp.Ajp13Parser.parseNext(Ajp13Parser.java:505) at org.eclipse.jetty.ajp.Ajp13Parser.parseAvailable(Ajp13Parser.java:158) at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264) at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:701) Caused by: java.lang.NumberFormatException: For input string: "5584start=208664" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Long.parseLong(Long.java:438) at java.lang.Long.parseLong(Long.java:478) at org.kohsuke.stapler.framework.io.LargeText.doProgressText(LargeText.java:256) at hudson.console.AnnotatedLargeText.doProgressiveHtml(AnnotatedLargeText.java:91) at sun.reflect.GeneratedMethodAccessor141.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:622) at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:298) at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:161) at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:96) at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:120) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:728) ... 74 more
as an "Oops!" report and stopped following the log.
I've recently been seeing this happening a lot, and it seems new in Jenkins ver. 1.574-SNAPSHOT . I'm on git rev 17d9093.
[JENKINS-23971] NumberFormatException when tailing node log (ssh unix slave, ec2)
Are you behind a reverse proxy? If so, does this also happen when accessing Jenkins directly?
Scott Roberts
added a comment - We're running Jenkins version 1.564 and we just saw this for the first time today as well.
Error while serving http://build.rctanalytics.com/job/Connection%20Broker/41/promotion/Production/promotionBuild/6/logText/progressiveHtml
...
Caused by: java.lang.NumberFormatException: For input string: "1251start=24"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Long.parseLong(Long.java:438)
at java.lang.Long.parseLong(Long.java:478)
at org.kohsuke.stapler.framework.io.LargeText.doProgressText(LargeText.java:256)
at hudson.console.AnnotatedLargeText.doProgressiveHtml(AnnotatedLargeText.java:91)
Our version is 1.564, but we have updated plugins if those might be the cause:
Active Directory plugin
This plugin enables authentication through Active Directory on Windows environment.
1.38
Downgrade to 1.37
Ant Plugin
This plugin adds Apache Ant support to Jenkins.
1.2
Artifactory Plugin
This plugin allows deploying maven artifacts and build info to Artifactory.
2.2.3
Downgrade to 2.2.1
Copy Artifact Plugin
Adds a build step to copy artifacts from another project.
1.31
Downgrade to 1.30
Credentials Plugin
This plugin allows you to store credentials in Jenkins.
1.15
Downgrade to 1.15
Unpin
CVS Plug-in
Integrates Jenkins with CVS version control system using a modified version of the Netbeans cvsclient.
2.12
Downgrade to 2.11
Unpin
Deploy to container Plugin
This plugin allows you to deploy a war to a container after a successful build.
Glassfish 3.x remote deployment
1.10
Downgrade to 1.9
External Monitor Job Type Plugin
Adds the ability to monitor the result of externally executed jobs.
1.2
Downgrade to 1.1
Unpin
GIT client plugin
Shared library plugin for other Git related Jenkins plugins.
1.10.0
Downgrade to 1.9.2
GIT plugin
This plugin integrates GIT with Jenkins.
2.2.2
Downgrade to 2.2.1
Git server plugin
Allows Jenkins to act as a Git server.
1.3
Downgrade to 1.2
Gradle plugin
This plugin allows Jenkins to invoke Gradle build scripts directly.
1.24
Downgrade to 1.23
Grails plugin
This plugin supports building Grails applications from Jenkins.
1.7
Hudson Groovy builder
This plugin executes Groovy code.
1.19
Downgrade to 1.18
instant-messaging plugin
This plugin provides abstract support for build notification via instant-messaging.
1.29
Downgrade to 1.28
Jabber notifier plugin
Sends build notifications to jabber contacts and/or chatrooms. Also allows control of builds via a jabber 'bot'.
Note that the instant-messaging plugin 1.24 is a requirement for this plugin. Please make sure that it is installed, too!
1.25
Javadoc Plugin
This plugin adds Javadoc support to Jenkins.
1.1
LDAP Plugin
Security realm based on LDAP authentication.
1.10.2
Downgrade to 1.8
Unpin
Mailer Plugin
This plugin allows you to configure email notifications. This is a break-out of the original core based email component.
1.9
Downgrade to 1.8
Unpin
MapDB API Plugin
This plugin provides a shared dependency on the MapDB library so that other plugins can co-operate when using this library.
1.0.1.0
Matrix Authorization Strategy Plugin
Offers matrix-based security authorization strategies (global and per-project).
1.2
Downgrade to 1.1
Unpin
Matrix Project Plugin
Multi-configuration (matrix) project type.
1.3
Downgrade to 1.2
Unpin
Maven Integration plugin
Jenkins plugin for building Maven 2/3 jobs via a special project type.
2.5
Downgrade to 2.3
Unpin
OWASP Markup Formatter Plugin
Uses the OWASP Java HTML Sanitizer to allow safe-seeming HTML markup to be entered in project descriptions and the like.
1.2
Downgrade to 1.1
Unpin
PAM Authentication plugin
Adds Unix Pluggable Authentication Module (PAM) support to Jenkins.
1.1
Downgrade to 1.0
Unpin
promoted builds plugin
This plugin implements a "promoted build" feature where a build of one job can be marked as "promoted" when it passes certain criteria.
2.17
Downgrade to 2.13
S3 publisher plugin
This is a plugin to upload files to Amazon S3 buckets.
0.6
Downgrade to 0.5
SCM API Plugin
This plugin provides a new enhanced API for interacting with SCM systems.
0.2
Scriptler
Scriptler allows you to store/edit/execute groovy scripts on any of the slaves/nodes... no need for copy paste groovy code anymore. Beside administer your scripts, Scritpler also provides a way to share scripts between users via hosted script catalogs on the internet.
2.7
Downgrade to 2.6.1
SSH Agent Plugin
This plugin allows you to provide SSH credentials to builds via a ssh-agent in Jenkins.
1.4.1
Downgrade to 1.3
SSH Credentials Plugin
This plugin allows you to store SSH credentials in Jenkins.
1.7.1
Downgrade to 1.6.1
Unpin
SSH Slaves plugin
This plugin allows you to manage slaves running on *nix machines over SSH.
1.6
Downgrade to 1.2
Unpin
Subversion Plug-in
This plugin adds the Subversion support (via SVNKit) to Jenkins.
2.4.1
Downgrade to 2.4
Unpin
Subversion Revert Plugin
This plugin automatically performs Subversion reverting when the build status changes to unstable or failed.
1.3
Token Macro Plugin
This plug-in adds reusable macro expansion capability for other plug-ins to use.
1.10
Downgrade to 1.8.1
Translation Assistance plugin
This plugin adds an additional dialog box in every page, which enables people to contribute localizations for the messages they are seeing in the current page.
1.11
Downgrade to 1.10
Unpin
Windows Slaves Plugin
Allows you to connect to Windows machines and start slave agents on them.
1.0
Scott Roberts
added a comment - We're running Jenkins version 1.564 and we just saw this for the first time today as well.
Error while serving http://build.rctanalytics.com/job/Connection%20Broker/41/promotion/Production/promotionBuild/6/logText/progressiveHtml
...
Caused by: java.lang.NumberFormatException: For input string: "1251start=24"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Long.parseLong(Long.java:438)
at java.lang.Long.parseLong(Long.java:478)
at org.kohsuke.stapler.framework.io.LargeText.doProgressText(LargeText.java:256)
at hudson.console.AnnotatedLargeText.doProgressiveHtml(AnnotatedLargeText.java:91)
Our version is 1.564, but we have updated plugins if those might be the cause:
Active Directory plugin
This plugin enables authentication through Active Directory on Windows environment.
1.38
Downgrade to 1.37
Ant Plugin
This plugin adds Apache Ant support to Jenkins.
1.2
Artifactory Plugin
This plugin allows deploying maven artifacts and build info to Artifactory.
2.2.3
Downgrade to 2.2.1
Copy Artifact Plugin
Adds a build step to copy artifacts from another project.
1.31
Downgrade to 1.30
Credentials Plugin
This plugin allows you to store credentials in Jenkins.
1.15
Downgrade to 1.15
Unpin
CVS Plug-in
Integrates Jenkins with CVS version control system using a modified version of the Netbeans cvsclient.
2.12
Downgrade to 2.11
Unpin
Deploy to container Plugin
This plugin allows you to deploy a war to a container after a successful build.
Glassfish 3.x remote deployment
1.10
Downgrade to 1.9
External Monitor Job Type Plugin
Adds the ability to monitor the result of externally executed jobs.
1.2
Downgrade to 1.1
Unpin
GIT client plugin
Shared library plugin for other Git related Jenkins plugins.
1.10.0
Downgrade to 1.9.2
GIT plugin
This plugin integrates GIT with Jenkins.
2.2.2
Downgrade to 2.2.1
Git server plugin
Allows Jenkins to act as a Git server.
1.3
Downgrade to 1.2
Gradle plugin
This plugin allows Jenkins to invoke Gradle build scripts directly.
1.24
Downgrade to 1.23
Grails plugin
This plugin supports building Grails applications from Jenkins.
1.7
Hudson Groovy builder
This plugin executes Groovy code.
1.19
Downgrade to 1.18
instant-messaging plugin
This plugin provides abstract support for build notification via instant-messaging.
1.29
Downgrade to 1.28
Jabber notifier plugin
Sends build notifications to jabber contacts and/or chatrooms. Also allows control of builds via a jabber 'bot'.
Note that the instant-messaging plugin 1.24 is a requirement for this plugin. Please make sure that it is installed, too!
1.25
Javadoc Plugin
This plugin adds Javadoc support to Jenkins.
1.1
LDAP Plugin
Security realm based on LDAP authentication.
1.10.2
Downgrade to 1.8
Unpin
Mailer Plugin
This plugin allows you to configure email notifications. This is a break-out of the original core based email component.
1.9
Downgrade to 1.8
Unpin
MapDB API Plugin
This plugin provides a shared dependency on the MapDB library so that other plugins can co-operate when using this library.
1.0.1.0
Matrix Authorization Strategy Plugin
Offers matrix-based security authorization strategies (global and per-project).
1.2
Downgrade to 1.1
Unpin
Matrix Project Plugin
Multi-configuration (matrix) project type.
1.3
Downgrade to 1.2
Unpin
Maven Integration plugin
Jenkins plugin for building Maven 2/3 jobs via a special project type.
2.5
Downgrade to 2.3
Unpin
OWASP Markup Formatter Plugin
Uses the OWASP Java HTML Sanitizer to allow safe-seeming HTML markup to be entered in project descriptions and the like.
1.2
Downgrade to 1.1
Unpin
PAM Authentication plugin
Adds Unix Pluggable Authentication Module (PAM) support to Jenkins.
1.1
Downgrade to 1.0
Unpin
promoted builds plugin
This plugin implements a "promoted build" feature where a build of one job can be marked as "promoted" when it passes certain criteria.
2.17
Downgrade to 2.13
S3 publisher plugin
This is a plugin to upload files to Amazon S3 buckets.
0.6
Downgrade to 0.5
SCM API Plugin
This plugin provides a new enhanced API for interacting with SCM systems.
0.2
Scriptler
Scriptler allows you to store/edit/execute groovy scripts on any of the slaves/nodes... no need for copy paste groovy code anymore. Beside administer your scripts, Scritpler also provides a way to share scripts between users via hosted script catalogs on the internet.
2.7
Downgrade to 2.6.1
SSH Agent Plugin
This plugin allows you to provide SSH credentials to builds via a ssh-agent in Jenkins.
1.4.1
Downgrade to 1.3
SSH Credentials Plugin
This plugin allows you to store SSH credentials in Jenkins.
1.7.1
Downgrade to 1.6.1
Unpin
SSH Slaves plugin
This plugin allows you to manage slaves running on *nix machines over SSH.
1.6
Downgrade to 1.2
Unpin
Subversion Plug-in
This plugin adds the Subversion support (via SVNKit) to Jenkins.
2.4.1
Downgrade to 2.4
Unpin
Subversion Revert Plugin
This plugin automatically performs Subversion reverting when the build status changes to unstable or failed.
1.3
Token Macro Plugin
This plug-in adds reusable macro expansion capability for other plug-ins to use.
1.10
Downgrade to 1.8.1
Translation Assistance plugin
This plugin adds an additional dialog box in every page, which enables people to contribute localizations for the messages they are seeing in the current page.
1.11
Downgrade to 1.10
Unpin
Windows Slaves Plugin
Allows you to connect to Windows machines and start slave agents on them.
1.0 If this happens fairly often (what web browser(s)?), try to keep the web developer console open, logging all requests. The following happens when viewing the log of a build in progress/slave log/promotion log:
- The browser sends AJAX requests to Jenkins with the POST parameter 'start' to only load data that hasn't been received yet.
- Jenkins response contains a HTTP header named 'X-Text-Size' which serves as the new starting point
The cause for the issue is a garbled value sent by the client (it appears to be two values concatenated).
It would be interesting to see where it goes wrong, so if it happens, check the log of past requests. Is it first the client to fail, or the server?
Craig Ringer
added a comment - Thanks for the tip Daniel. I'm using Chrome 35.0.1916.27 at the moment. I'll keep an eye on the dev console.
Craig Ringer
added a comment - Thanks for the tip Daniel. I'm using Chrome 35.0.1916.27 at the moment. I'll keep an eye on the dev console. Any news?
Would also be possible to look in the access logs for error responses to /progressivHtml to look for a commonality (same host, same browser, same job, ...).
Resolving as Cannot Reproduce after a few months without update. Reopen if this happens again; make sure to provide the requested information.
This doesn't seem to disrupt the ssh communication channel; slave/master communication continues and logs are captured as normal. Re-entering the node's page and opening the log again shows the rest of the log.