Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-63517

Very slow archiveArtifacts on agents


    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Major Major
    • plugin-proposals
    • None

      We're using ssh-slaves-plugin for our linux build nodes, and experiencing abysmal agent-master throughput (~13*M*bps), despite their (verified) 10*G*bps link. The full set of our build artifacts are huge (~10GB) right now, and yes we are compressing (with zip step) the ones that aren't already compressed.

      There are several Jenkins core issues tracking the sluggishness of archiving artifacts from agent to master over the last decade, but none of them appear to have ever been resolved properly, so I thought I'd create this as an "Improvement" in this component which seems to be the right place for it. It would seem that since the agent session is an ssh session that we should be able to achieve parity with scp (and in fact, use scp?), when archiving artifacts.

      As suggested here, the problem may be "that Jenkins archives via its control channel (e.g. ssh slave - using java SSH implementation JSCH). The java ssh just can't get anywhere near 1Gb/s network speed that native SSH can manage easily."

      If however, this becomes a "can't fix" or "won't fix", has anyone achieved a reasonable workaround for archiving artifacts on jenkins agents?

      We could replace our usage of `archiveArtifacts` steps with a custom groovy call to use the system scp to the correct "artifacts" location on the master, but this of course feels hacky, and I'd really rather not have to reinvent this wheel. I found [the publish-over-ssh plugin|https://github.com/jenkinsci/publish-over-ssh-plugin], but it doesn't seem to be maintained anymore. Anyone using that, or some other alternative to native archiveArtifacts step?

      FWIW, we're using Debian Buster, Jenkins 2.222.3, and this java version on all master/agents:

      jenkins@jenkins-testing-agent-2:~$ java --version
      openjdk 11.0.8 2020-07-14
      OpenJDK Runtime Environment (build 11.0.8+10-post-Debian-1deb10u1)
      OpenJDK 64-Bit Server VM (build 11.0.8+10-post-Debian-1deb10u1, mixed mode, sharing)

      Thanks for your time.

            Unassigned Unassigned
            timblaktu Tim Black
            2 Vote for this issue
            3 Start watching this issue