Ross Rowe added a comment - 2012-03-20 11:11 Okay, thanks for that, I've updated the summary and description to cover this. In this case, I think the change should be fairly straightforward...I think I'll try change the plugin to launch Sauce Connect on the Master, then all you will need to do is configure the Job to include the host name of the Master in the Sauce Connect Advanced Options. I'll let you know when I've got a snapshot version ready to test.

Ross Rowe added a comment - 2012-03-20 11:13 Actually, you probably won't even need to update the configuration...the plugin is currently defaulting the Selenium host to 'localhost', I'll change the plugin to default the host to the Master's host name.

SCM/JIRA link daemon added a comment - 2012-03-20 11:41 Code changed in jenkins User: Ross Rowe Path: src/main/java/hudson/plugins/sauce_ondemand/SauceOnDemandBuildWrapper.java http://jenkins-ci.org/commit/sauce-ondemand-plugin/048fdffd64761c4072bf029d273dcd06658ea2af Log: JENKINS-13152 Updated plugin to launch Sauce Connect on the master rather than the slave

R. Tyler Croy added a comment - 2012-03-21 00:09 Testing the snapshot plugin, getting this exception on startup: Mar 20, 2012 5:08:30 PM jenkins.InitReactorRunner$1 onTaskFailed SEVERE: Failed Loading job multiderp java.lang.NullPointerException at hudson.matrix.MatrixProject.onLoad(MatrixProject.java:411) at hudson.model.Items.load(Items.java:115) at jenkins.model.Jenkins$15.run(Jenkins.java:2421) at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:146) at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:259) at jenkins.model.Jenkins$6.runTask(Jenkins.java:840) at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:187) at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:94) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang. Thread .run( Thread .java:679) Mar 20, 2012 5:08:30 PM jenkins.InitReactorRunner$1 onAttained

R. Tyler Croy added a comment - 2012-03-21 00:25 Also with the snapshot version of the plugin, this issue is not fixed. I created a simple matrix job with one axis, with "a" "b" "c" as the variants. My Jenkins instance has a master node with 2 executors, and a slave node with 3 executors (just because). Build #1, parent job's console: Started by user anonymous Building remotely on loopback in workspace /home/tyler/.jenkins/loopback-root/workspace/matrix-test Triggering c Triggering b Triggering a c completed with result FAILURE b completed with result SUCCESS a completed with result FAILURE Finished: FAILURE Build #1, a's console Started by upstream project "matrix-test" build number 1 Building remotely on loopback in workspace /home/tyler/.jenkins/loopback-root/workspace/matrix-test/sides/a Starting Sauce OnDemand SSH tunnels Launching Sauce Connect [/usr/lib64/jvm/java-1.6.0-openjdk-1.6.0/jre/bin/java, -cp, /home/tyler/sauce-connect.jar, com.saucelabs.sauceconnect.SauceConnect, rcroy, MEGA-REDACTED, -P, 4445] [-u, rcroy, -k, MEGA-REDACTED, -d, sauce-connect.proxy, -s, 127.0.0.1, -p, 80, --ssh-port, 443, -b, -- rest -url, https: //saucelabs.com/ rest /v1, --se-port, 4445, --squid-opts, ] * Debug messages will be sent to sauce_connect.log .---------------------------------------------------. | Have questions or need help with Sauce Connect? | | Contact us: http: //support.saucelabs.com/forums | | Terms of Service: http: //saucelabs.com/tos | ----------------------------------------------------- 2012-03-20 17:14:21,371 - / Starting \ 2012-03-20 17:14:21,380 - Please wait for "You may start your tests" to start your tests. 2012-03-20 17:14:21,394 - Forwarding: [ 'sauce-connect.proxy' ]:[ '80' ] -> 127.0.0.1:[ '46556' ] 2012-03-20 17:14:21,408 - Succesfully connected to local server 127.0.0.1:46556 in 9ms { 'shutdown_time' : None, 'owner' : 'rcroy' , 'status' : 'booting' , 'use_caching_proxy' : True, 'user_shutdown' : None, 'metadata' : { 'PythonVersion' : '2.5.1' , 'OwnerPorts' : [ '42769' ], 'OwnerHost' : '127.0.0.1' , 'Release' : '3.0-r18' , 'Ports' : [ '80' ], 'Build' : '26' , 'Platform' : 'Java-1.6.0_24-OpenJDK_64-Bit_Server_VM,_20.0-b12,_Sun_Microsystems_Inc.-on-Linux-3.2.10-15-desktop-amd64' , 'ScriptRelease' : 26, 'ScriptName' : 'sauce_connect' }, 'use_kgp' : True, 'id' : 'e45e2b2cf72f4a5eb60fad424a8c90f5' , 'ssh_port' : 443, 'domain_names' : [ 'sauce-connect.proxy' ], 'creation_time' : 1332288852, 'host' : None} 2012-03-20 17:14:21,956 - Shutting down other tunnel remote VMs using requested domains 2012-03-20 17:14:22,671 - { "squid_config" :[ ""]," use_caching_proxy ": true ," fast_fail_regexps ":[" "]," ssh_port ":443," metadata ":{" PythonVersion ":" 2.5.1 "," OwnerHost ":" 127.0.0.1 "," Release ":" 3.0-r18 "," OwnerPorts ":[" 46556 "]," Ports ":[" 80 "]," Platform ":" Java-1.6.0_24-OpenJDK_64-Bit_Server_VM,_20.0-b12,_Sun_Microsystems_Inc.-on-Linux-3.2.10-15-desktop-amd64 "," Build ":" 26 "," ScriptRelease ":26," ScriptName ":" sauce_connect "}," use_kgp ": true ," domain_names ":[" sauce-connect.proxy"]} 2012-03-20 17:14:22,986 - Tunnel remote VM is provisioned (7550384b9b16446b96e060693c2ef802) 2012-03-20 17:14:23,336 - Tunnel remote VM is new .. 2012-03-20 17:14:26,592 - Tunnel remote VM was shut down 2012-03-20 17:14:26,598 - Requesting new tunnel { 'shutdown_time' : None, 'owner' : 'rcroy' , 'status' : 'halting' , 'use_caching_proxy' : True, 'user_shutdown' : True, 'metadata' : { 'PythonVersion' : '2.5.1' , 'OwnerPorts' : [ '46556' ], 'OwnerHost' : '127.0.0.1' , 'Release' : '3.0-r18' , 'Ports' : [ '80' ], 'Build' : '26' , 'Platform' : 'Java-1.6.0_24-OpenJDK_64-Bit_Server_VM,_20.0-b12,_Sun_Microsystems_Inc.-on-Linux-3.2.10-15-desktop-amd64' , 'ScriptRelease' : 26, 'ScriptName' : 'sauce_connect' }, 'use_kgp' : True, 'id' : '7550384b9b16446b96e060693c2ef802' , 'ssh_port' : 443, 'domain_names' : [ 'sauce-connect.proxy' ], 'creation_time' : 1332288862, 'host' : None} { 'shutdown_time' : None, 'owner' : 'rcroy' , 'status' : 'halting' , 'use_caching_proxy' : True, 'user_shutdown' : True, 'metadata' : { 'PythonVersion' : '2.5.1' , 'OwnerPorts' : [ '42769' ], 'OwnerHost' : '127.0.0.1' , 'Release' : '3.0-r18' , 'Ports' : [ '80' ], 'Build' : '26' , 'Platform' : 'Java-1.6.0_24-OpenJDK_64-Bit_Server_VM,_20.0-b12,_Sun_Microsystems_Inc.-on-Linux-3.2.10-15-desktop-amd64' , 'ScriptRelease' : 26, 'ScriptName' : 'sauce_connect' }, 'use_kgp' : True, 'id' : 'e45e2b2cf72f4a5eb60fad424a8c90f5' , 'ssh_port' : 443, 'domain_names' : [ 'sauce-connect.proxy' ], 'creation_time' : 1332288852, 'host' : None} 2012-03-20 17:14:27,006 - Shutting down other tunnel remote VMs using requested domains 2012-03-20 17:14:27,953 - { "squid_config" :[ ""]," use_caching_proxy ": true ," fast_fail_regexps ":[" "]," ssh_port ":443," metadata ":{" PythonVersion ":" 2.5.1 "," OwnerHost ":" 127.0.0.1 "," Release ":" 3.0-r18 "," OwnerPorts ":[" 46556 "]," Ports ":[" 80 "]," Platform ":" Java-1.6.0_24-OpenJDK_64-Bit_Server_VM,_20.0-b12,_Sun_Microsystems_Inc.-on-Linux-3.2.10-15-desktop-amd64 "," Build ":" 26 "," ScriptRelease ":26," ScriptName ":" sauce_connect "}," use_kgp ": true ," domain_names ":[" sauce-connect.proxy"]} 2012-03-20 17:14:28,259 - Tunnel remote VM is provisioned (17a284b1ec2b44b7b454a9904fbde261) 2012-03-20 17:14:28,510 - Tunnel remote VM is new .. 2012-03-20 17:14:31,801 - Tunnel remote VM is deploying .. 2012-03-20 17:14:35,003 - Tunnel remote VM is booting .. 2012-03-20 17:15:00,733 - Tunnel remote VM is running at maki8035.miso.saucelabs.com 2012-03-20 17:15:00,750 - Succesfully connected to local server 127.0.0.1:46556 in 0ms 2012-03-20 17:15:00,756 - Starting connection to tunnel host... 2012-03-20 17:15:00,759 - Connecting to tunnel host maki8035.miso.saucelabs.com as rcroy 2012-03-20 17:15:00,882 - Forwarding Selenium with ephemeral port 57965 2012-03-20 17:15:00,901 - Selenium HTTP proxy listening on port 4445 2012-03-20 17:15:01,019 - Successful handshake with Sauce Connect server 2012-03-20 17:15:01,032 - Tunnel host version: 0.1.0, remote endpoint ID: 1472c6c2a09943b3978d7f99f7c5c386 2012-03-20 17:15:01,038 - Connected! You may start your tests. [a] $ /bin/sh -xe /tmp/hudson8735648732809226617.sh + echo ls ls + sleep 50 Shutting down Sauce OnDemand SSH tunnels FATAL: null java.lang.IllegalMonitorStateException at java.util.concurrent.locks.ReentrantLock$Sync.tryRelease(ReentrantLock.java:155) at java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1262) at java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:459) at com.saucelabs.ci.sauceconnect.SauceConnectTwoManager.closeTunnelsForPlan(SauceConnectTwoManager.java:60) at hudson.plugins.sauce_ondemand.SauceOnDemandBuildWrapper$TunnelHolder.close(SauceOnDemandBuildWrapper.java:230) at hudson.plugins.sauce_ondemand.SauceOnDemandBuildWrapper$1.tearDown(SauceOnDemandBuildWrapper.java:126) at hudson.model.Build$RunnerImpl.doRun(Build.java:150) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:473) at hudson.model.Run.run(Run.java:1410) at hudson.matrix.MatrixRun.run(MatrixRun.java:146) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:238) Build #1, b's console Started by upstream project "matrix-test" build number 1 Building on master in workspace /home/tyler/.jenkins/jobs/matrix-test/workspace/sides/b Starting Sauce OnDemand SSH tunnels Launching Sauce Connect [/usr/lib64/jvm/java-1.6.0-openjdk-1.6.0/jre/bin/java, -cp, /home/tyler/.jenkins/plugins/sauce-ondemand/WEB-INF/lib/sauce-connect-3.0.18.jar, com.saucelabs.sauceconnect.SauceConnect, rcroy, MEGA-REDACTED, -P, 4445] [-u, rcroy, -k, MEGA-REDACTED, -d, sauce-connect.proxy, -s, 127.0.0.1, -p, 80, --ssh-port, 443, -b, -- rest -url, https: //saucelabs.com/ rest /v1, --se-port, 4445, --squid-opts, ] * Debug messages will be sent to sauce_connect.log .---------------------------------------------------. | Have questions or need help with Sauce Connect? | | Contact us: http: //support.saucelabs.com/forums | | Terms of Service: http: //saucelabs.com/tos | ----------------------------------------------------- 2012-03-20 17:14:12,007 - / Starting \ 2012-03-20 17:14:12,012 - Please wait for "You may start your tests" to start your tests. 2012-03-20 17:14:12,026 - Forwarding: [ 'sauce-connect.proxy' ]:[ '80' ] -> 127.0.0.1:[ '42769' ] 2012-03-20 17:14:12,051 - Succesfully connected to local server 127.0.0.1:42769 in 14ms 2012-03-20 17:14:12,454 - { "squid_config" :[ ""]," use_caching_proxy ": true ," fast_fail_regexps ":[" "]," ssh_port ":443," metadata ":{" PythonVersion ":" 2.5.1 "," OwnerHost ":" 127.0.0.1 "," Release ":" 3.0-r18 "," OwnerPorts ":[" 42769 "]," Ports ":[" 80 "]," Platform ":" Java-1.6.0_24-OpenJDK_64-Bit_Server_VM,_20.0-b12,_Sun_Microsystems_Inc.-on-Linux-3.2.10-15-desktop-amd64 "," Build ":" 26 "," ScriptRelease ":26," ScriptName ":" sauce_connect "}," use_kgp ": true ," domain_names ":[" sauce-connect.proxy"]} 2012-03-20 17:14:12,688 - Tunnel remote VM is provisioned (e45e2b2cf72f4a5eb60fad424a8c90f5) 2012-03-20 17:14:12,913 - Tunnel remote VM is new .. 2012-03-20 17:14:16,118 - Tunnel remote VM is deploying .. 2012-03-20 17:14:19,322 - Tunnel remote VM is booting .. 2012-03-20 17:14:23,513 - Tunnel remote VM was shut down 2012-03-20 17:14:23,515 - Requesting new tunnel { 'shutdown_time' : None, 'owner' : 'rcroy' , 'status' : 'deploying' , 'use_caching_proxy' : True, 'user_shutdown' : None, 'metadata' : { 'PythonVersion' : '2.5.1' , 'OwnerPorts' : [ '46556' ], 'OwnerHost' : '127.0.0.1' , 'Release' : '3.0-r18' , 'Ports' : [ '80' ], 'Build' : '26' , 'Platform' : 'Java-1.6.0_24-OpenJDK_64-Bit_Server_VM,_20.0-b12,_Sun_Microsystems_Inc.-on-Linux-3.2.10-15-desktop-amd64' , 'ScriptRelease' : 26, 'ScriptName' : 'sauce_connect' }, 'use_kgp' : True, 'id' : '7550384b9b16446b96e060693c2ef802' , 'ssh_port' : 443, 'domain_names' : [ 'sauce-connect.proxy' ], 'creation_time' : 1332288862, 'host' : None} { 'shutdown_time' : None, 'owner' : 'rcroy' , 'status' : 'halting' , 'use_caching_proxy' : True, 'user_shutdown' : True, 'metadata' : { 'PythonVersion' : '2.5.1' , 'OwnerPorts' : [ '42769' ], 'OwnerHost' : '127.0.0.1' , 'Release' : '3.0-r18' , 'Ports' : [ '80' ], 'Build' : '26' , 'Platform' : 'Java-1.6.0_24-OpenJDK_64-Bit_Server_VM,_20.0-b12,_Sun_Microsystems_Inc.-on-Linux-3.2.10-15-desktop-amd64' , 'ScriptRelease' : 26, 'ScriptName' : 'sauce_connect' }, 'use_kgp' : True, 'id' : 'e45e2b2cf72f4a5eb60fad424a8c90f5' , 'ssh_port' : 443, 'domain_names' : [ 'sauce-connect.proxy' ], 'creation_time' : 1332288852, 'host' : None} 2012-03-20 17:14:24,134 - Shutting down other tunnel remote VMs using requested domains 2012-03-20 17:14:25,197 - { "squid_config" :[ ""]," use_caching_proxy ": true ," fast_fail_regexps ":[" "]," ssh_port ":443," metadata ":{" PythonVersion ":" 2.5.1 "," OwnerHost ":" 127.0.0.1 "," Release ":" 3.0-r18 "," OwnerPorts ":[" 42769 "]," Ports ":[" 80 "]," Platform ":" Java-1.6.0_24-OpenJDK_64-Bit_Server_VM,_20.0-b12,_Sun_Microsystems_Inc.-on-Linux-3.2.10-15-desktop-amd64 "," Build ":" 26 "," ScriptRelease ":26," ScriptName ":" sauce_connect "}," use_kgp ": true ," domain_names ":[" sauce-connect.proxy"]} 2012-03-20 17:14:25,450 - Tunnel remote VM is provisioned (b9982cee921b48dc81bf5c42acac565f) 2012-03-20 17:14:25,706 - Tunnel remote VM is new .. 2012-03-20 17:14:35,489 - Tunnel remote VM is deploying .. 2012-03-20 17:14:38,690 - Tunnel remote VM is booting .. 2012-03-20 17:15:04,460 - Tunnel remote VM is running at maki10149.miso.saucelabs.com 2012-03-20 17:15:04,480 - Succesfully connected to local server 127.0.0.1:42769 in 2ms 2012-03-20 17:15:04,484 - Starting connection to tunnel host... 2012-03-20 17:15:04,486 - Connecting to tunnel host maki10149.miso.saucelabs.com as rcroy 2012-03-20 17:15:04,632 - Forwarding Selenium with ephemeral port 54526 2012-03-20 17:15:04,638 - Selenium HTTP proxy listening on port 4445 2012-03-20 17:15:04,766 - Successful handshake with Sauce Connect server 2012-03-20 17:15:04,805 - Tunnel host version: 0.1.0, remote endpoint ID: 64d87539b0ad4dbcb67e09113b009abc 2012-03-20 17:15:04,809 - Connected! You may start your tests. [b] $ /bin/sh -xe /tmp/hudson7895239882625682910.sh + echo ls ls + sleep 50 Shutting down Sauce OnDemand SSH tunnels Finished: SUCCESS Build #1, c's console Started by upstream project "matrix-test" build number 1 Building on master in workspace /home/tyler/.jenkins/jobs/matrix-test/workspace/sides/c Starting Sauce OnDemand SSH tunnels [c] $ /bin/sh -xe /tmp/hudson4799966040448883389.sh + echo ls ls + sleep 50 Shutting down Sauce OnDemand SSH tunnels FATAL: null java.lang.NullPointerException at com.saucelabs.ci.sauceconnect.SauceConnectTwoManager.closeTunnelsForPlan(SauceConnectTwoManager.java:51) at hudson.plugins.sauce_ondemand.SauceOnDemandBuildWrapper$TunnelHolder.close(SauceOnDemandBuildWrapper.java:230) at hudson.plugins.sauce_ondemand.SauceOnDemandBuildWrapper$1.tearDown(SauceOnDemandBuildWrapper.java:126) at hudson.model.Build$RunnerImpl.doRun(Build.java:150) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:473) at hudson.model.Run.run(Run.java:1410) at hudson.matrix.MatrixRun.run(MatrixRun.java:146) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:238) a failed, b succeeeded, c failed.

Ross Rowe added a comment - 2012-03-21 01:54 Okay, thanks for letting me know. Could you try stopping jenkins, removing the ~/.jenkins/plugins/sauce-ondemand directory, then restart jenkins? It looks like Sauce Connect is still being started on the Slave, which shouldn't happen with the snapshot version, so I think the old plugin logic is still active. I'll do some further investigation into the NullPointerExceptions, could you send me the config.xml files for your projects and I'll try to replicate the issues. Cheers, Ross

R. Tyler Croy added a comment - 2012-03-21 02:42 This was with a local test instance that never had any Sauce OnDemand plugin installed.

Ross Rowe added a comment - 2012-03-21 03:02 Okay, I'll do some more investigation and let you know what I find.

SCM/JIRA link daemon added a comment - 2012-03-21 04:56 Code changed in jenkins User: Ross Rowe Path: src/main/java/hudson/plugins/sauce_ondemand/SauceOnDemandBuildWrapper.java http://jenkins-ci.org/commit/sauce-ondemand-plugin/fe935119aba73ff57e9c36bf7b3d3bb5a94b14d4 Log: JENKINS-13152 Ensure that tunnels are closed on the master, not the slave

Ross Rowe added a comment - 2012-03-26 10:56 This should now be resolved in version 1.12