Bruno Laoueillé added a comment - 2018-01-02 14:10 Hi, I've exactly the same problem.   Agent log : janv. 02, 2018 3:05:49 PM hudson.remoting.jnlp.Main createEngine INFO: Setting up slave: Windows Slave janv. 02, 2018 3:05:49 PM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir INFO: Using D:\Jenkins\remoting as a remoting work directory Both error and output logs will be printed to D:\Jenkins\remoting janv. 02, 2018 3:05:49 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve INFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping] janv. 02, 2018 3:06:00 PM hudson.remoting.jnlp.GuiListener$1 run INFO: Connecting to ciserver:8081 (retrying:2) java.io.IOException: Failed to connect to ciserver:8081 {{ at org.jenkinsci.remoting.engine.JnlpAgentEndpoint.open(JnlpAgentEndpoint.java:242)}} {{ at hudson.remoting.Engine.connect(Engine.java:686)}} {{ at hudson.remoting.Engine.innerRun(Engine.java:547)}} {{ at hudson.remoting.Engine.run(Engine.java:469)}} Caused by: java.net.ConnectException: Connection refused: connect {{ at sun.nio.ch.Net.connect0(Native Method)}} {{ at sun.nio.ch.Net.connect(Unknown Source)}} {{ at sun.nio.ch.Net.connect(Unknown Source)}} {{ at sun.nio.ch.SocketChannelImpl.connect(Unknown Source)}} {{ at java.nio.channels.SocketChannel.open(Unknown Source)}} {{ at org.jenkinsci.remoting.engine.JnlpAgentEndpoint.open(JnlpAgentEndpoint.java:203)}} {{ ... 3 more}}   And in master log nothing appears...   Regards, Bruno.

Oleg Nenashev added a comment - 2018-01-02 15:45 Which agent version do you use?

Bruno Laoueillé added a comment - 2018-01-02 16:17 For me : I've uninstalled my Windows Service agent and click again on "Launch agent from browser". I guess it's the latest with this method ? In the agent.jar manifest file it's written 3.15 for the version...

Oleg Nenashev added a comment - 2018-01-02 17:29 - edited My first guess is that Remoting receiver thread is not active due to whatever reason. Or maybe not, the HTTPs receiver should be working Please provide... 1) Jenkins log, starting from the master startup 2) Thread dump for the master If you could just generate a bundle using https://wiki.jenkins.io/display/JENKINS/Support+Core+Plugin , it would be great

Bruno Laoueillé added a comment - 2018-01-02 17:46 Hi,   I hope it's what you asked : support_2018-01-02_17.45.19.zip master-logs.txt

Oleg Nenashev added a comment - 2018-01-02 17:56 Some investigation while I am waiting for the info... 1) No changes in JnlpAgentEndpoint since 2.95 2) No changes in Engine#connect() 3) AFAICT https://github.com/jenkinsci/remoting/pull/210 is the most plausible source of regressions, but I do not see the bogus logic so far. Please also provide agent startup arguments

Oleg Nenashev added a comment - 2018-01-02 17:59 A full support bundle would help though. In particular I am interested whether the agent uses HTTPS and what are the certificate check flags

Bruno Laoueillé added a comment - 2018-01-02 18:05 Yes here :  support_2018-01-02_18.01.52.zip

Oleg Nenashev added a comment - 2018-01-02 18:14 Yes, for JNLP agent I really need startup arguments. Other info: 8081 is a TCP agent listener port TCP Agent Listener thread is running There is a proxy configured on the instance. Likely it's not related since there are other community votes "TCP agent listener port=8081" id=53 (0x35) state=RUNNABLE cpu=0% (running in native) at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at hudson.TcpSlaveAgentListener.run(TcpSlaveAgentListener.java:157)

Bruno Laoueillé added a comment - 2018-01-02 18:20 OK, How can I retrieve agent startup arguments ?   On startup page for the agent start through the browser I have java -jar agent.jar -jnlpUrl http://myserver:10004/computer/Windows%20Slave/slave-agent.jnlp -secret **************************************************************** -workDir "D:\Jenkins"   (Note: I've replaced the real server name by myserver, and changed the secret code).  

Oleg Nenashev added a comment - 2018-01-02 19:31 Yes, it's what I needed. Now it's a time to reproduce it

Goffredo Marocchi added a comment - 2018-01-02 20:03 - edited Seeing something very similar. In a job: First time build. Skipping changelog. No emails were triggered. FATAL: command execution failed Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from mdbuildserver01217.gamesys.corp/10.194.194.147:60268 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1696) at hudson.remoting.UserResponse.retrieve(UserRequest.java:313) at hudson.remoting.Channel.call(Channel.java:909) at hudson.Launcher$RemoteLauncher.launch(Launcher.java:1053) at hudson.Launcher$ProcStarter.start(Launcher.java:450) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:109) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744) at hudson.model.Build$BuildExecution.build(Build.java:206) at hudson.model.Build$BuildExecution.doRun(Build.java:163) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504) at hudson.model.Run.execute(Run.java:1727) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) java.lang.NoSuchMethodError: hudson.Launcher$RemoteLaunchCallable.getOpenChannelOrFail()Lhudson/remoting/Channel; at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1292) at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1260) at hudson.remoting.UserRequest.perform(UserRequest.java:207) at hudson.remoting.UserRequest.perform(UserRequest.java:53) at hudson.remoting.Request$2.run(Request.java:358) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at hudson.remoting.Engine$1$1.run(Engine.java:98) at java.lang. Thread .run( Thread .java:748) Caused: java.io.IOException: Remote call on JNLP4-connect connection from mdbuildserver01217.gamesys.corp/10.194.194.147:60268 failed at hudson.remoting.Channel.call(Channel.java:917) at hudson.Launcher$RemoteLauncher.launch(Launcher.java:1053) at hudson.Launcher$ProcStarter.start(Launcher.java:450) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:109) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744) at hudson.model.Build$BuildExecution.build(Build.java:206) at hudson.model.Build$BuildExecution.doRun(Build.java:163) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504) at hudson.model.Run.execute(Run.java:1727) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) Build step 'Execute shell' marked build as failure   Starting the agent up: Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main createEngine INFO: Setting up slave: <SlaveName> Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main$CuiListener <init> INFO: Jenkins agent is running in headless mode. Jan 02, 2018 7:55:08 PM hudson.remoting.Engine startEngine WARNING: No Working Directory. Using the legacy JAR Cache location: /Users/mdbuildserver/.jenkins/cache/jars Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Locating server among [https: //<jenkins URL>/] Jan 02, 2018 7:55:08 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve INFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping] Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Agent discovery successful   Agent address: <jenkins URL>/   Agent port:    42650   Identity:      fb:f3:24:67:05:be:94:06:49:2a:aa:64:c2:83:32:c7 Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Handshaking Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Connecting to nps-jenkins.gamesys.co.uk:42650 Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Trying protocol: JNLP4-connect Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Remote identity confirmed: fb:f3:24:67:05:be:94:06:49:2a:aa:64:c2:83:32:c7 Jan 02, 2018 7:55:08 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Connected Jan 02, 2018 7:55:08 PM hudson.remoting.UserRequest perform WARNING: LinkageError while performing UserRequest:hudson.slaves.ChannelPinger$SetUpRemotePing@2905 java.lang.NoSuchMethodError: hudson.slaves.ChannelPinger$SetUpRemotePing.getOpenChannelOrFail()Lhudson/remoting/Channel;     at hudson.slaves.ChannelPinger$SetUpRemotePing.call(ChannelPinger.java:137)     at hudson.slaves.ChannelPinger$SetUpRemotePing.call(ChannelPinger.java:121)     at hudson.remoting.UserRequest.perform(UserRequest.java:207)     at hudson.remoting.UserRequest.perform(UserRequest.java:53)     at hudson.remoting.Request$2.run(Request.java:358)     at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)     at java.util.concurrent.FutureTask.run(FutureTask.java:266)     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)     at hudson.remoting.Engine$1$1.run(Engine.java:98)     at java.lang. Thread .run( Thread .java:748) Jan 02, 2018 7:55:08 PM hudson.remoting.UserRequest perform WARNING: LinkageError while performing UserRequest:jenkins.slaves.StandardOutputSwapper$ChannelSwapper@3b16348f java.lang.NoSuchMethodError: jenkins.slaves.StandardOutputSwapper$ChannelSwapper.getOpenChannelOrFail()Lhudson/remoting/Channel;     at jenkins.slaves.StandardOutputSwapper$ChannelSwapper.call(StandardOutputSwapper.java:42)     at jenkins.slaves.StandardOutputSwapper$ChannelSwapper.call(StandardOutputSwapper.java:39)     at hudson.remoting.UserRequest.perform(UserRequest.java:207)     at hudson.remoting.UserRequest.perform(UserRequest.java:53)     at hudson.remoting.Request$2.run(Request.java:358)     at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)     at java.util.concurrent.FutureTask.run(FutureTask.java:266)     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)     at hudson.remoting.Engine$1$1.run(Engine.java:98)     at java.lang. Thread .run( Thread .java:748) Jan 02, 2018 7:55:09 PM com.youdevise.hudson.slavestatus.SlaveListener call INFO: Slave-status listener starting Jan 02, 2018 7:55:09 PM com.youdevise.hudson.slavestatus.SocketHTTPListener waitForConnection INFO: Slave-status listener ready on port 3141   Agent is launched like so: java -jar agent.jar -jnlpUrl https: //<Jenkins URL>/computer/<AgentName>/slave-agent.jnlp -secret <Agent Secret>  

Oleg Nenashev added a comment - 2018-01-02 20:12 Goffredo Marocchi could you please create a separate ticket for now? I am not 100% sure the issues are same, but in the case of your stacktraces I know what to fix

Oleg Nenashev added a comment - 2018-01-02 20:20 Regarding the issue from Goffredo Marocchi , updating agent to Remoting 3.15 is a workaround, working on a fix

Goffredo Marocchi added a comment - 2018-01-02 20:34 Thanks Oleg Nenashev if you want me to create a separate ticket I will do so right away and link it here.

Goffredo Marocchi added a comment - 2018-01-02 20:38 Done: https://issues.jenkins-ci.org/browse/JENKINS-48761

Oleg Nenashev added a comment - 2018-01-02 21:18 There is a chance that my patch for JENKINS-48761 solves this issue as well

Oleg Nenashev added a comment - 2018-01-03 22:52 Most likely it is fixed in Jenkins 2.100, the release is in progress. Retrospective thread: https://groups.google.com/forum/#!topic/jenkinsci-dev/-VQ5YU0yYhg

SCM/JIRA link daemon added a comment - 2018-01-04 00:01 Code changed in jenkins User: Oleg Nenashev Path: content/_data/changelogs/weekly.yml http://jenkins-ci.org/commit/jenkins.io/ac42d83bf51e56e826991839a5471eccdd767543 Log: Changelog: noting JENKINS-48761 / JENKINS-48754 in Jenkins 2.100. The PR also adds warning banners to 2.98/2.99

SCM/JIRA link daemon added a comment - 2018-01-04 00:01 Code changed in jenkins User: R. Tyler Croy Path: content/_data/changelogs/weekly.yml http://jenkins-ci.org/commit/jenkins.io/99fe6d097fd1cc1f6797ad7b816c98dec3261a37 Log: Merge pull request #1305 from oleg-nenashev/changelog/2.100 Changelog: noting JENKINS-48761 / JENKINS-48754 in Jenkins 2.100. Compare: https://github.com/jenkins-infra/jenkins.io/compare/62362762ff55...99fe6d097fd1

Bruno Laoueillé added a comment - 2018-01-04 06:53 Hi, Thank you for your actions. I have updated to the 2.100 version, restarted Jenkins, then go to my slave and refreshed my browser cache, finally I launch again the agent from the browser and unfortunally the result was identical. It doesn't work. I also tried to download the agent.jar and launch by command line, we have exactly the same thing. Same stack trace on the agent side... I edited the downloaded agent.jar, it was still in 3.5 version, is it normal ?   However we are totally blocked without the possibility to come back to the 2.94 version (we can only come back to 2.99), do you think we have a possibility to come back to this version through the web interface (because we have no access to the docker administration where Jenkins is installed).    Thank you in advance for your help.    Regards, Bruno.

Oleg Nenashev added a comment - 2018-01-04 08:27 - edited Thanks for the quick reply, digging in. > I edited the downloaded agent.jar, it was still in 3.5 version, is it normal ? It's not. The version should be 3.15, maybe it's just a typo. But yes, all changes in 2.100 were on the master side, Remoting has not been changed. > However we are totally blocked without the possibility to come back to the 2.94 version (we can only come back to 2.99), do you think we have a possibility to come back to this version through the web interface (because we have no access to the docker administration where Jenkins is installed). I am not sure the version rollback from Jenkins UI operates correctly in Docker. My recommendation would be to replace a container. Thanks for the pointers regarding Docker though. Do you use the standard Jenkins image or your custom one?

Bruno Laoueillé added a comment - 2018-01-04 08:38 - edited For the version 3.5 it's my bad, sorry, it's the 3.15 (bad typing).   EDIT: For the Docker image, I dont know sorry, it's not me who managed it. I can ask to another person.

Oleg Nenashev added a comment - 2018-01-04 09:10 - edited OK, Docker... So far I can say the following: Master runs on the 8082 port according to the bundle, but the web UI suggests the 10004 port. It is a whatever port mapping or external proxy, I'd guess Agent fails to connect to TcpAgentListener port, the connection is refused. Probably it's just because the port is not exposed TcpAgentListener runs with port set to 0 . It means the it will be taking a random port every restart Since you run the master in Docker, I am not sure how it is supposed to work. You would have to expose each port on the Docker host. I doubt you do that. Please check which port is exposed up for TcpAgentListener, and make sure the same port is configured in the security global config. Since you run in Docker, I would definitely recommend setting port flags so that the port won't be changeable from UI: -Djenkins.model.Jenkins.slaveAgentPort=${YOUR_EXPOSED_PORT} -Djenkins.model.Jenkins.slaveAgentPortEnforce=true Example for Docker "TCP agent listener port=0" id=86 (0x56) state=RUNNABLE cpu=0% (running in native) at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at hudson.TcpSlaveAgentListener.run(TcpSlaveAgentListener.java:157)

Bruno Laoueillé added a comment - 2018-01-05 10:23 Yes it's ok for that, moreover the configuration has not changed on that. We have put the 8081 port for Tcp agents with only JNLP/4 and it is accessible and configured on our Docker. (It was working well before updates).

Oleg Nenashev added a comment - 2018-01-18 08:18 Back to this story... > We have put the 8081 port How did you do that? The threaddump definitely suggests you're running with a random port

Bruno Laoueillé added a comment - 2018-01-18 08:28 Hi, Sorry for late answer. But you are right, the configuration of the connection through tuneling was not set for the slave. I forgot it after my second installation (it was set only in the global security part). So after correcting this part the new version has worked perfectly. Thank you for your support. Regards, Bruno.

Oleg Nenashev added a comment - 2018-01-18 08:35 Thanks for the reply! So this is not a defect. I will close it as a duplicate of JENKINS-48761 , because the most of the voters/watchers followed the summary of the issue && timing was very coincidental