Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-49278

cat command in docker agents not detected correctly

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Minor Minor
    • docker-workflow-plugin
    • None
    • Jenkins 2.104 on Docker 17.10.0-ce on CentOS 7.4.1708 (Kernel 3.10.0-693.2.2.el7.x86_64)

      When using a declarative Jenkins pipeline with a stage that uses a Docker agent, I get a confusing error message in the Jenkins log:

      $ docker top 08e1c013e07083492ad0f03285f1a7d30063fb15e0cf39be7b55af6d1a03c829
      ERROR: The container started but didn't run the expected command. Please double check your ENTRYPOINT does execute the command passed as docker run argument. See https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/#entrypoint for entrypoint best practices.
      

      The build continues normally and the cat command is actually running inside the container, so everything is fine except that the error message occurs although it shouldn't.

      Comparing the code in listProcess in https://github.com/jenkinsci/docker-workflow-plugin/blob/master/src/main/java/org/jenkinsci/plugins/docker/workflow/client/DockerClient.java with the output of docker top shows the likely cause of that error:

      docker top prints the following fields

      UID                 PID                 PPID                C                   STIME               TTY                 TIME                CMD
      build               19799               19784               0                   22:23               pts/0               00:00:00            cat
      

      However, the Java client assumes that only PID, USER, TIME and COMMAND is printed. I suggest that the process list is determined by using an explicit format specifier like

      docker container top ${CONTAINER_ID} -eo pid,comm
      

          [JENKINS-49278] cat command in docker agents not detected correctly

          Hendrik Halkow created issue -
          Hendrik Halkow made changes -
          Status Original: Open [ 1 ] New: In Progress [ 3 ]

          which OS are your using as docker host ?

          Nicolas De Loof added a comment - which OS are your using as docker host ?

          Hendrik Halkow added a comment - - edited

          CentOS 7.4. (See 'Environment')

          Hendrik Halkow added a comment - - edited CentOS 7.4. (See 'Environment')
          Nicolas De Loof made changes -
          Assignee New: Nicolas De Loof [ ndeloof ]

          Code changed in jenkins
          User: Hendrik M Halkow
          Path:
          src/main/java/org/jenkinsci/plugins/docker/workflow/client/DockerClient.java
          http://jenkins-ci.org/commit/docker-workflow-plugin/f53608309af64f70471683377c248789c64f400c
          Log:
          JENKINS-49278 Get process list within containers by explicitly specifying ps output.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Hendrik M Halkow Path: src/main/java/org/jenkinsci/plugins/docker/workflow/client/DockerClient.java http://jenkins-ci.org/commit/docker-workflow-plugin/f53608309af64f70471683377c248789c64f400c Log: JENKINS-49278 Get process list within containers by explicitly specifying ps output.
          Nicolas De Loof made changes -
          Resolution New: Fixed [ 1 ]
          Status Original: In Progress [ 3 ] New: Resolved [ 5 ]

          marc young added a comment -

          This is officially preventing builds from working for us.

          11:53:41 $ docker run -t -d -u 10000:10000 -u 0 -w /var/jenkins_home/workspace/Environment_Rebuild_master-H4NNFES5KCILFZZIYUFIHNUT7P6FIWBPBEZA4TEKUDL2E2ILLNRA/terraform --volumes-from 381af2f7bed3bd8749f916422f7776367afc2537d86429fb8408a47172523d7b -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** hashicorp/terraform:0.11.2 cat
          11:53:42 $ docker top 2463f11ecc8add34688abd78af438e3c65a657ab242b15d3ccb16edece928f13
          11:53:43 ERROR: The container but didn't run the expected command. Please double check your ENTRYPOINT does execute the command passed as docker run argument. See https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/#entrypoint for entrypoint best practices.
          
          

          This was a surprise to update jenkins and have very important builds just stop working as expected with no good way to fix them internally

          marc young added a comment - This is officially preventing builds from working for us. 11:53:41 $ docker run -t -d -u 10000:10000 -u 0 -w / var /jenkins_home/workspace/Environment_Rebuild_master-H4NNFES5KCILFZZIYUFIHNUT7P6FIWBPBEZA4TEKUDL2E2ILLNRA/terraform --volumes-from 381af2f7bed3bd8749f916422f7776367afc2537d86429fb8408a47172523d7b -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** hashicorp/terraform:0.11.2 cat 11:53:42 $ docker top 2463f11ecc8add34688abd78af438e3c65a657ab242b15d3ccb16edece928f13 11:53:43 ERROR: The container but didn't run the expected command. Please double check your ENTRYPOINT does execute the command passed as docker run argument. See https: //docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/#entrypoint for entrypoint best practices. This was a surprise to update jenkins and have very important builds just stop working as expected with no good way to fix them internally

          myoung34

          I don't think this is breaking anything.  At least, I can see the error everywhere but it isn't breaking anything.

          Christian Höltje added a comment - myoung34 I don't think this is breaking anything.  At least, I can see the error everywhere but it isn't breaking anything.

          Marc Philipp added a comment -

          I can confirm that this breaks builds for us, too. Any known workaround?

          Marc Philipp added a comment - I can confirm that this breaks builds for us, too. Any known workaround?

            Unassigned Unassigned
            hendrikhalkow Hendrik Halkow
            Votes:
            4 Vote for this issue
            Watchers:
            36 Start watching this issue

              Created:
              Updated: