Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-17330

FilePath.installIfNecessaryFrom routes download over remoting channel

    XMLWordPrintable

Details

    Description

      Installing a large downloaded tool on a network slave can be quite slow:

      "Executor #... for ... : executing ... #... / waiting for hudson.remoting.Channel@...:HTTP full-duplex channel ..." Id=... Group=main TIMED_WAITING on hudson.remoting.UserRequest@...
      	at java.lang.Object.wait(Native Method)
      	-  waiting on hudson.remoting.UserRequest@...
      	at hudson.remoting.Request.call(Request.java:146)
      	at hudson.remoting.Channel.call(Channel.java:672)
      	at hudson.FilePath.act(FilePath.java:831)
      	at hudson.FilePath.act(FilePath.java:824)
      	at hudson.FilePath.unzipFrom(FilePath.java:485)
      	at hudson.FilePath.installIfNecessaryFrom(FilePath.java:718)
      	at hudson.tools.DownloadFromUrlInstaller.performInstallation(DownloadFromUrlInstaller.java:72)
      	at com.soasta.jenkins.MakeAppTouchTestableInstaller.performInstallation(MakeAppTouchTestableInstaller.java:49)
      	at com.soasta.jenkins.MakeAppTouchTestable.perform(MakeAppTouchTestable.java:94)
      	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
      	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:728)
      	at hudson.model.Build$BuildExecution.build(Build.java:199)
      	at hudson.model.Build$BuildExecution.doRun(Build.java:160)
      	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
      	at hudson.model.Run.execute(Run.java:1505)
      	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
      	at hudson.model.ResourceController.execute(ResourceController.java:88)
      	at hudson.model.Executor.run(Executor.java:237)
      

      JENKINS-7813 is partly to blame, but the main issue here is that the URL is being opened on master, a RemoteInputStream constructed, and the archive unpacked on the slave. It would be much faster to open the URL directly from the slave and use the remoting channel only for control.

      In typical cases the slave can access a tool URL. There are some rarer cases where it is really intentional that the URL is resolved from the master; it might be a file-protocol URL, the slave might have no public internet access, the download location might be limited to particular IPs, etc. So installIfNecessaryFrom should first try to open from the slave, then fall back to the current implementation.

      Attachments

        Issue Links

          Activity

            jglick Jesse Glick created issue -
            scm_issue_link SCM/JIRA link daemon made changes -
            Field Original Value New Value
            Resolution Fixed [ 1 ]
            Status Open [ 1 ] Resolved [ 5 ]
            vjuranek vjuranek made changes -
            Labels lts-candidate performance 1.509.1-fixed lts-candidate performance
            vjuranek vjuranek made changes -
            Labels 1.509.1-fixed lts-candidate performance 1.509.1-fixed performance
            jglick Jesse Glick made changes -
            Link This issue is related to JENKINS-18664 [ JENKINS-18664 ]
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 148164 ] JNJira + In-Review [ 192751 ]

            People

              jglick Jesse Glick
              jglick Jesse Glick
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: