Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-29752

Failure of WithContainerStepTest#stop in ubuntu trusty

    XMLWordPrintable

Details

    Description

      When running WithContainerStepTest#stop on Ubuntu trusty, the build abort that should happen never come. The reason is that WithContainerStep#kill tries to look up processes with JENKINS_SERVER_COOKIE=durable-<durableId>. But it doesn't work because it is indirectly referenced. (notice jsc=... then JENKINS_SERVER_COOKIE=$jsc)

      Running docker exec 9c405bc9aa649ebc4fa172e681d2ec02a0ace3b8a39be777a34bc1bc3a0f1d60 ps -A -o "pid,command" e
        PID COMMAND
          1 cat
         10 sh -c echo $$ > '/tmp/junit5445757940323164914/junit3199707453429220714/workspace/p/.ba6cf8f6/pid'; jsc=durable-8a78a05454f21cb427cda0326a8bd396; JENKINS_SERVER_COOKIE=$jsc '/tmp/junit5445757940323164914/junit3199707453429220714/workspace/p/.ba6cf8f6/script.sh' > '/tmp/junit5445757940323164914/junit3199707453429220714/workspace/p/.ba6cf8f6/jenkins-log.txt' 2>&1; echo $? > '/tmp/junit5445757940323164914/junit3199707453429220714/workspace/p/.ba6cf8f6/jenkins-result.txt'
         17 /bin/sh -xe /tmp/junit5445757940323164914/junit3199707453429220714/workspace/p/.ba6cf8f6/script.sh
         18 sleep 999
         26 ps -A -o pid,command e PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/apache2/bin HOSTNAME=9c405bc9aa64 TERM=xterm BUILD_DISPLAY_NAME=#1 BUILD_ID=2015-08-03_16-50-59 BUILD_NUMBER=1 BUILD_TAG=jenkins-p-1 BUILD_URL=http://localhost:52715/jenkins/job/p/1/ CLASSPATH= HUDSON_HOME=/tmp/junit5445757940323164914/junit3199707453429220714 HUDSON_SERVER_COOKIE=8e1085b66cb45a24 HUDSON_URL=http://localhost:52715/jenkins/ JENKINS_HOME=/tmp/junit5445757940323164914/junit3199707453429220714 JENKINS_SERVER_COOKIE=8e1085b66cb45a24 JENKINS_URL=http://localhost:52715/jenkins/ JOB_NAME=p JOB_URL=http://localhost:52715/jenkins/job/p/ HTTPD_PREFIX=/usr/local/apache2 HTTPD_VERSION=2.4.12 HTTPD_BZ2_URL=https://www.apache.org/dist/httpd/httpd-2.4.12.tar.bz2 HOME=/
      Killing []
      

      Attachments

        Issue Links

          Activity

            vlatombe Vincent Latombe created issue -
            vlatombe Vincent Latombe made changes -
            Field Original Value New Value
            Description When running WithContainerStepTest#stop on Ubuntu trusty, the build abort that should happen never come. The reason is that WithContainerStep#kill tries to look up processes with JENKINS_SERVER_COOKIE=durable-<durableId>. But it doesn't work because it is indirectly referenced.

            {code}
            Running docker exec 9c405bc9aa649ebc4fa172e681d2ec02a0ace3b8a39be777a34bc1bc3a0f1d60 ps -A -o "pid,command" e
              PID COMMAND
                1 cat
               10 sh -c echo $$ > '/tmp/junit5445757940323164914/junit3199707453429220714/workspace/p/.ba6cf8f6/pid'; jsc=durable-8a78a05454f21cb427cda0326a8bd396; JENKINS_SERVER_COOKIE=$jsc '/tmp/junit5445757940323164914/junit3199707453429220714/workspace/p/.ba6cf8f6/script.sh' > '/tmp/junit5445757940323164914/junit3199707453429220714/workspace/p/.ba6cf8f6/jenkins-log.txt' 2>&1; echo $? > '/tmp/junit5445757940323164914/junit3199707453429220714/workspace/p/.ba6cf8f6/jenkins-result.txt'
               17 /bin/sh -xe /tmp/junit5445757940323164914/junit3199707453429220714/workspace/p/.ba6cf8f6/script.sh
               18 sleep 999
               26 ps -A -o pid,command e PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/apache2/bin HOSTNAME=9c405bc9aa64 TERM=xterm BUILD_DISPLAY_NAME=#1 BUILD_ID=2015-08-03_16-50-59 BUILD_NUMBER=1 BUILD_TAG=jenkins-p-1 BUILD_URL=http://localhost:52715/jenkins/job/p/1/ CLASSPATH= HUDSON_HOME=/tmp/junit5445757940323164914/junit3199707453429220714 HUDSON_SERVER_COOKIE=8e1085b66cb45a24 HUDSON_URL=http://localhost:52715/jenkins/ JENKINS_HOME=/tmp/junit5445757940323164914/junit3199707453429220714 JENKINS_SERVER_COOKIE=8e1085b66cb45a24 JENKINS_URL=http://localhost:52715/jenkins/ JOB_NAME=p JOB_URL=http://localhost:52715/jenkins/job/p/ HTTPD_PREFIX=/usr/local/apache2 HTTPD_VERSION=2.4.12 HTTPD_BZ2_URL=https://www.apache.org/dist/httpd/httpd-2.4.12.tar.bz2 HOME=/
            Killing []
            {code}
            When running WithContainerStepTest#stop on Ubuntu trusty, the build abort that should happen never come. The reason is that WithContainerStep#kill tries to look up processes with JENKINS_SERVER_COOKIE=durable-<durableId>. But it doesn't work because it is indirectly referenced. (notice jsc=... then JENKINS_SERVER_COOKIE=$jsc)

            {code}
            Running docker exec 9c405bc9aa649ebc4fa172e681d2ec02a0ace3b8a39be777a34bc1bc3a0f1d60 ps -A -o "pid,command" e
              PID COMMAND
                1 cat
               10 sh -c echo $$ > '/tmp/junit5445757940323164914/junit3199707453429220714/workspace/p/.ba6cf8f6/pid'; jsc=durable-8a78a05454f21cb427cda0326a8bd396; JENKINS_SERVER_COOKIE=$jsc '/tmp/junit5445757940323164914/junit3199707453429220714/workspace/p/.ba6cf8f6/script.sh' > '/tmp/junit5445757940323164914/junit3199707453429220714/workspace/p/.ba6cf8f6/jenkins-log.txt' 2>&1; echo $? > '/tmp/junit5445757940323164914/junit3199707453429220714/workspace/p/.ba6cf8f6/jenkins-result.txt'
               17 /bin/sh -xe /tmp/junit5445757940323164914/junit3199707453429220714/workspace/p/.ba6cf8f6/script.sh
               18 sleep 999
               26 ps -A -o pid,command e PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/apache2/bin HOSTNAME=9c405bc9aa64 TERM=xterm BUILD_DISPLAY_NAME=#1 BUILD_ID=2015-08-03_16-50-59 BUILD_NUMBER=1 BUILD_TAG=jenkins-p-1 BUILD_URL=http://localhost:52715/jenkins/job/p/1/ CLASSPATH= HUDSON_HOME=/tmp/junit5445757940323164914/junit3199707453429220714 HUDSON_SERVER_COOKIE=8e1085b66cb45a24 HUDSON_URL=http://localhost:52715/jenkins/ JENKINS_HOME=/tmp/junit5445757940323164914/junit3199707453429220714 JENKINS_SERVER_COOKIE=8e1085b66cb45a24 JENKINS_URL=http://localhost:52715/jenkins/ JOB_NAME=p JOB_URL=http://localhost:52715/jenkins/job/p/ HTTPD_PREFIX=/usr/local/apache2 HTTPD_VERSION=2.4.12 HTTPD_BZ2_URL=https://www.apache.org/dist/httpd/httpd-2.4.12.tar.bz2 HOME=/
            Killing []
            {code}
            jglick Jesse Glick added a comment -

            I am not sure what “indirectly referenced” means. This Bourne shell syntax sets the environment variable on the specified subprocess. Most likely it is there. The problem here is that ps e is apparently not working as advertised: some of the processes are listing their environment variables (presumably all of them), while others are listing none. Perhaps this is a bug in ps in some versions of Linux.

            jglick Jesse Glick added a comment - I am not sure what “indirectly referenced” means. This Bourne shell syntax sets the environment variable on the specified subprocess. Most likely it is there. The problem here is that ps e is apparently not working as advertised: some of the processes are listing their environment variables (presumably all of them), while others are listing none . Perhaps this is a bug in ps in some versions of Linux.
            jglick Jesse Glick added a comment -

            Attaching WiP patch to replace ps e with more direct access to the proc filesystem. IIRC tfennelly had a similar problem at some point, though I was not able to reproduce it myself. (Currently I run Utopic.)

            jglick Jesse Glick added a comment - Attaching WiP patch to replace ps e with more direct access to the proc filesystem. IIRC tfennelly had a similar problem at some point, though I was not able to reproduce it myself. (Currently I run Utopic.)
            jglick Jesse Glick made changes -
            Attachment JENKINS-29752.diff [ 30407 ]
            jglick Jesse Glick made changes -
            Priority Minor [ 4 ] Major [ 3 ]
            jglick Jesse Glick made changes -
            Link This issue duplicates JENKINS-28606 [ JENKINS-28606 ]
            jglick Jesse Glick made changes -
            Resolution Duplicate [ 3 ]
            Status Open [ 1 ] Resolved [ 5 ]
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 164827 ] JNJira + In-Review [ 197554 ]

            People

              jglick Jesse Glick
              vlatombe Vincent Latombe
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: