Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-65934

Maven Container Slow to Download Artifacts - Times Out

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Critical Critical
    • kubernetes-plugin
    • None
    • Jenkins 2.289.1, Kubernetes Plugin 1.30.0

      I am using a pod template that has a Maven container to perform builds. The time it takes for this container to pull artifacts from either a private Nexus repo or Maven Central is extremely slow (over 30 minutes then eventually fails in contrast to less than a minute by other clients not invoked by the Kubernetes plugin - e.g. developer machines). It's so slow that Maven reports the following error (this is just one example):

      GET request of: org/junit/jupiter/junit-jupiter-params/5.6.2/junit-jupiter-params-5.6.2.jar from nexus failed: Premature end of Content-Length delimited message body (expected: 562,271; received: 302,935)
      

      The Nexus logs report the following error (see this for more info).

      java.util.concurrent.TimeoutException: Idle timeout expired: 30000/30000 ms
      

       
      I executed into the Maven container (via the below command) invoked by the Kubernetes plugin and checked the CPU/memory utilization using the top command and found the java process that's running the mvn install command to be barely using any resources (e.g. 1% cpu).

      kubectl exec -n cicd -it test-spring-boot-demo-78-mdr23-tvz1d-mxgkc -c maven /bin/bash
      

      Where things get really interesting is when I manually run the same Maven command defined in the sh step. Invoking the mvn install command executes at warp speed and finishes in about 54 seconds. When looking at the resource allocation of the java process while the command is running, I see almost 100% CPU utilization. It downloads artifacts about the same speed from either Nexus or Maven Central.

      So the takeaway is that the mvn install command invoked by the Kubernetes plugin runs slow to the point of failure but executing that command manually from a different shell/session runs very fast and works as intended.

      I also tried to set org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator.stdinBufferSize to a higher value but that didn't seem to help.

      Any thoughts on what this might be?

            Unassigned Unassigned
            getchadeagle Daniel
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: