setsid(1) doesn't appear available on Solaris (source) nor FreeBSD (source).

My proposal would be to write an equivalent command in Python.

Here's the functioning Python version.

This would be also a good place to do what we do with shell today; redirect stdout/err to a file, wait for the completion of the child process, then record exit code.

This lets us intercept various signals like SIGINT, SIGTERM, so that we can forward the signal to the child process and also record the fact in the exit code file.

Our current shell script wrapper doesn't do that, so in case the shell is killed (like Ctrl+C that got me into this), we fail to capture what killed it, and instead we get non-descriptive -1 (from BourneShellScript):

int _pid = pid(workspace);
if (_pid > 0 && !ProcessLiveness.isAlive(workspace.getChannel(), _pid)) {
    return -1; // arbitrary code to distinguish from 0 (success) and 1+ (observed failure)
}

If this saves durable tasks running on master or via CommandLauncher from e.g. Ctrl-C then great. My main concern is availability across POSIXish platforms. Does Mac OS X have it? /usr/bin?

jglick What do you mean by it?
MacOS X has the setsid() function in libc, but not a setsid command-line utility.

For Python, MacOS X 10.10.5 has this in /usr/bin:

-rwxr-xr-x  2 root  wheel  58416 Jul 14 22:53 /usr/bin/python
-rwxr-xr-x  5 root  wheel    925 Sep  9  2014 /usr/bin/python-config
lrwxr-xr-x  1 root  wheel     75 Mar 12  2015 /usr/bin/python2.6 -> ../../System/Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6
lrwxr-xr-x  1 root  wheel     82 Mar 12  2015 /usr/bin/python2.6-config -> ../../System/Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6-config
lrwxr-xr-x  1 root  wheel     75 Mar 12  2015 /usr/bin/python2.7 -> ../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7
lrwxr-xr-x  1 root  wheel     82 Mar 12  2015 /usr/bin/python2.7-config -> ../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7-config
-rwxr-xr-x  2 root  wheel  58416 Jul 14 22:53 /usr/bin/pythonw
lrwxr-xr-x  1 root  wheel     76 Mar 12  2015 /usr/bin/pythonw2.6 -> ../../System/Library/Frameworks/Python.framework/Versions/2.6/bin/pythonw2.6
lrwxr-xr-x  1 root  wheel     76 Mar 12  2015 /usr/bin/pythonw2.7 -> ../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/pythonw2.7

kohsuke I'm still having tons of problems with durable task plugin due to this problem.  I think setsid is the way to go on this.

The idea to write a setsid.py wrapper in Python is intriguing.  It would definitely work.

My concern is that now Python becomes a core dependency for running Jenkins Pipeline.

I have no problem with that because I love Python and use Python heavily in my projects.

However, from a devops perspective this might be a problem because people now need to add Python to their dependencies for running Jenkins.  This becomes more interesting as more Jenkins deployments are running in a dockerized type of world.

Would it be possible to write a small setsid wrapper in JNA?

jnr-posix has a wrapper for setsid() so this might be doable in Java:

https://github.com/jnr/jnr-posix/blob/master/src/main/java/jnr/posix/POSIX.java#L107

The JVM is likely to be too slow. A (statically-compiled) Go wrapper is probably most practical.

The advantages of Golang for this purpose are clear: small size; fast startup; static linking to minimize the chance of divergent runtime behaviors; access to platform-dependent system APIs like this one; and of course the availability of cross-compilation so we could have a Dockerized build process creating binaries for all supported platforms (whatever subset of go tool dist list) at once.

There will however still be some exotic platforms not supported by Golang, such as z/OS, and probably some cases where sh is being used in a semi-POSIX environment like Cygwin that might choke on lower-level calls. So I think the current shell wrapper still needs to be available as a fallback (not sure how you decide when to use it) and given some smoke test coverage, though it could be stripped of all its advanced features like PID tracking.

Shell wrapper script has now been replaced by pre-compiled golang binaries (for Unix and Darwin). The binary allows the script to have its own session id and removes some polling overhead. This should increase the survivability of long-running scripts when jenkins terminates unexpectedly. This does not change the survivability of the script when the underlying environment decides to terminate processes due to things like low memory.
Note: Outside of *NIX systems, the behavior is unchanged.

The binary itself is ~2.5MB per binary. There are 4 pre-compiled binaries (32 and 64bit versions for unix and darwin) while the memory footprint is ~800KB heavier than the shell wrapper.
Here is a high-level breakdown of the memory footprint:
Original version:
two wrapper shell processes that are spawned, each process between 610-640KB
a single 548KB sleep process used to poll the log output file
Go binary version:
a single golang binary ~2560KB