Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-72792

Builds hang sometimes for withMaven and ssh-agent steps execution

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Critical Critical
    • kubernetes-plugin
    • None
    • Jenkins: 2.426.3
      Kubernetes plugin: 4174.v4230d0ccd951

      Approximately 5% of builds executed on our Jenkins instance using Kubernetes plugin are hanging. This issue seems to occurs during the execution of `withMaven` step (from the Pipeline Maven Integration plugin) and `ssh-agent` step (from the SSH Agent plugin) itselves.

      We have another Jenkins instance (with the same version of core and the same plugins) that runs agents as Swarm services via the Docker Swarm plugin, and it do not exhibit this issue.

      It's challenging to determine whether the problem lies within the Kubernetes plugin itself or possibly in other plugins when executing steps with an awareness of running within `container` step.

      Currently, after examining the logs from these plugins (SSH Agent and Pipeline Maven Integration) combined with Log Recorder tracking logs from org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator at the FINEST level, we have identified a potential issue with concurrent process execution within the Kubernetes plugin.

      In the event of a specific step hanging, the log shows an entry: "onOpen: java.util.concurrent.CountDownLatch@2a09a562[Count = 1]". In all other cases, the counter indicates a value of 0, and if I understand correctly, the thread continues to wait for the countDown() call, which does not occur.

      What could be causing this behavior for certain pipeline steps?

      The mentioned log is invoked here: ContainerExecDecorator.java#L484

      Changes related to instantiating and invoking CountDownLatch were introduced as part of the issue: JENKINS-67664

            Unassigned Unassigned
            ljackiewicz Ɓukasz Jackiewicz
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: