Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-2729

Hudson failed to kill run-away GF?

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • other
    • None
    • Platform: All, OS: All

    Description

      This is on:

      hudson@wssqe27 /proc $ uname -a
      SunOS wssqe27 5.11 snv_70b sun4u sparc SUNW,Sun-Blade-1500

      pargs -e shows it has the cookie:

      hudson@wssqe27 /proc $ pargs -e 6107
      6107: /files/hudson/tools/native/java1.6.0_06/bin/java -cp
      /files/hudson/workspace/SQ
      ...
      envp[9]: HUDSON_COOKIE=f200d29d-19cf-4b01-833d-63bc8c5c1a1c
      ...

      Perhaps the killer doesn't work on Sparc?

      Attachments

        Activity

          Adding Sudipa to the CC list.

          After more investigation, the killer seems to be working correctly.

          The issue now seems to be that if a build succeeds but leave a daemon behind,
          Hudson won't notice.

          So the question is, whether I should kill run-away processes or not. Maybe the
          daemon process forked from a build is intentional, which makes this hard...

          kohsuke Kohsuke Kawaguchi added a comment - Adding Sudipa to the CC list. After more investigation, the killer seems to be working correctly. The issue now seems to be that if a build succeeds but leave a daemon behind, Hudson won't notice. So the question is, whether I should kill run-away processes or not. Maybe the daemon process forked from a build is intentional, which makes this hard...

          This is fixed in 1.283.

          The side effect of this is that if someone has been intentionally spawning
          processes, those will be killed, too. This is based on environment variables, so
          to work around this, you can spawn processes with one of the environment
          variables different from what Hudson sets. For example,

          BUILD_ID=dontKillMe catalina.sh start

          will start Tomcat in such a way that it'll escape the killing.

          kohsuke Kohsuke Kawaguchi added a comment - This is fixed in 1.283. The side effect of this is that if someone has been intentionally spawning processes, those will be killed, too. This is based on environment variables, so to work around this, you can spawn processes with one of the environment variables different from what Hudson sets. For example, BUILD_ID=dontKillMe catalina.sh start will start Tomcat in such a way that it'll escape the killing.
          mdp mdp added a comment -

          This workaround is often linked to, but it seems BUILD_ID no longer works.
          Experiments show that changing/unsetting some of JENKINS_COOKIE, JENKINS_SERVER_COOKIE, HUDSON_COOKIE or HUDSON_SERVER_COOKIE is now required.

          mdp mdp added a comment - This workaround is often linked to, but it seems BUILD_ID no longer works. Experiments show that changing/unsetting some of JENKINS_COOKIE, JENKINS_SERVER_COOKIE, HUDSON_COOKIE or HUDSON_SERVER_COOKIE is now required.
          mdp mdp added a comment -

          It seems that I was mistaken and unsetting BUILD_ID does indeed work, at least for free-style jobs.
          Where BUILD_ID failed me and unsetting *_COOKIE worked was in a maven job where the project starts a daemon using maven-exec-plugin.

          mdp mdp added a comment - It seems that I was mistaken and unsetting BUILD_ID does indeed work, at least for free-style jobs. Where BUILD_ID failed me and unsetting *_COOKIE worked was in a maven job where the project starts a daemon using maven-exec-plugin.
          cbrown Carlton Brown added a comment -

          Can someone clarify where this workaround is supposed to be specified? Is it in the environment of the Jenkins web container, or of the slave, or of the process itself? (For example do I need to pass it in an env key of an Ant exec or java task).

          cbrown Carlton Brown added a comment - Can someone clarify where this workaround is supposed to be specified? Is it in the environment of the Jenkins web container, or of the slave, or of the process itself? (For example do I need to pass it in an env key of an Ant exec or java task).

          People

            Unassigned Unassigned
            kohsuke Kohsuke Kawaguchi
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: