Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-49119

Support suppression of SSH server inactivity timeouts

XMLWordPrintable

    • Icon: Improvement Improvement
    • Resolution: Incomplete
    • Icon: Minor Minor
    • None
    • Jenkins 2.89.3
      Publish Over SSH 1.18
      CentOS Linux 7.3

      Our team runs a lot of Jenkins jobs using Publish over SSH. Many of them use long-running SQL queries and all timeouts in Publish over SSH have been disabled. However, we are running into server SSH inactivity timeouts, set as ClientAliveInterval 1500 in /etc/ssh/sshd_config for security reasons.

      One workaround we tried is to configure server-alives on a client side of the Jenkins server in ~/.ssh/config of the Jenkins user:

      Host *
      ServerAliveInterval 120
      ServerAliveCountMax 30
      ConnectTimeout 30
      

      This approach did not work. What worked is a wait/print loop in every remote script that needs to be executed. It works like this:

      # Run the main script
      Technical/Template_Job/template.sh &
      pid=$!
      
      # Wait on a background job completion. Query status every 10 minutes.
      declare -i elapsed=0
      # `ps -p ${pid}` works on macOS and CentOS. On both OSes `ps ${pid}` works as well.
      while ps -p ${pid} >/dev/null; do
        sleep 1
        if (( ++elapsed % 600 == 0 )); then
          echo "Waiting for the completion of the main script. $((elapsed / 60))m and counting ..."
        fi
      done
      
      # Return the exit code of the terminated background process. This works in Bash 4.4 despite what Bash docs say:
      # "If neither jobspec nor pid specifies an active child process of the shell, the return status is 127."
      wait ${pid}
      

      This works, but it's a hassle to be adding wait/print loops into each script we execute. It'd be very nice if Publish over SSH supports periodic server-alive messages or some other option to suppress server inactivity timeouts.

            Unassigned Unassigned
            dk Dima Korobskiy
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: