• Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Major Major
    • core, xcode-plugin
    • None

      JENKINS-9399 changed the Jenkins from using LaunchAgents to LaunchDaemons because, at the time, there were problems with running as an agent and using LaunchDaemons seems like the right approach.

      Two years later, it's pretty clear that using LaunchDaemons creates a whole host of problems which (I argue) are worse than using LaunchAgents in the first place. LaunchDaemons cannot launch GUI applications which means you cannot unit test iOS applications or launch with XCode "instructions"... both of which are in high demand.

      Anyway, I believe I discovered a way for us to have the best of both worlds: using LaunchAgents, having it auto-launch on boot, and preventing it from running under other users. I believe this will solve all outstanding problems.

      1. Configure Jenkins to use LaunchAgents (reverse JENKINS-9399).
      2. Install the plist into ~/Library/LaunchAgents instead of /Library/LaunchAgents to prevent it from getting triggered by other users logging in.
      3. Configure the Jenkins user to auto-login on startup, but return to the login menu immediately: http://www.tuaw.com/2011/03/07/terminally-geeky-use-automatic-login-more-securely/

          [JENKINS-21237] Jenkins should use LaunchAgents under OSX

          Sami Tikka added a comment -

          We might want to gather some requirements or user stories for the Jenkins Mac installer.

          My humble opinion is the only people who want to run Jenkins on Mac are iOS developers. (And Mac developers but there are few of those and their needs might be very similar to iOS developers.)

          People who do cross-platform development and need to build their software for multiple platforms need to set up Windows, Mac and Linux build servers. They will probably run Jenkins master either on Windows or Linux and run a slave on Mac. Thus we do not need to worry about these people.

          The iOS developers have some special needs (which I do not know exactly because I haven't done any real development for iOS so feel free to add to the list or set me straight.)

          • Builds must be codesigned. The developer certificate resides in the developer's keychain. The keychain is only accessible within the developer's GUI login session.
          • Automated tests need to execute iOS simulator which is a GUI app. GUI apps can only run in GUI login session.

          It seems to me the most common use cases for Jenkins master on Mac require that Jenkins has access to GUI login session. This means Jenkins master must be executed either as a launch agent (which are meant for background processes) or within a context of a GUI app wrapper (if we want something visible in Dock or menu bar.)

          The security aspect of having a Jenkins master with a logged-in user with a GUI session must be addressed separately. I do not believe we can provide a solution that fits everyone's needs. Maybe we can just give hints on how to solve it: put the Mac in a locked room, set up screensaver with short timeout or the trick mentioned above in the ticket description or maybe something else.

          Sami Tikka added a comment - We might want to gather some requirements or user stories for the Jenkins Mac installer. My humble opinion is the only people who want to run Jenkins on Mac are iOS developers. (And Mac developers but there are few of those and their needs might be very similar to iOS developers.) People who do cross-platform development and need to build their software for multiple platforms need to set up Windows, Mac and Linux build servers. They will probably run Jenkins master either on Windows or Linux and run a slave on Mac. Thus we do not need to worry about these people. The iOS developers have some special needs (which I do not know exactly because I haven't done any real development for iOS so feel free to add to the list or set me straight.) Builds must be codesigned. The developer certificate resides in the developer's keychain. The keychain is only accessible within the developer's GUI login session. Automated tests need to execute iOS simulator which is a GUI app. GUI apps can only run in GUI login session. It seems to me the most common use cases for Jenkins master on Mac require that Jenkins has access to GUI login session. This means Jenkins master must be executed either as a launch agent (which are meant for background processes) or within a context of a GUI app wrapper (if we want something visible in Dock or menu bar.) The security aspect of having a Jenkins master with a logged-in user with a GUI session must be addressed separately. I do not believe we can provide a solution that fits everyone's needs. Maybe we can just give hints on how to solve it: put the Mac in a locked room, set up screensaver with short timeout or the trick mentioned above in the ticket description or maybe something else.

          Randall Wood added a comment -
          • All login plist/hooks can be disabled when accessing the console of a OS X system that is being restarted. I know that a number of users of Jenkins on OS X are using Jenkins on systems that may be installed in areas not under the administrator's physical control. This is a real concern for many OS X developers who are not completely independent.
          • Software that exists in a user's directory is always easier to exploit than software installed using root privileges.

          Randall Wood added a comment - All login plist/hooks can be disabled when accessing the console of a OS X system that is being restarted. I know that a number of users of Jenkins on OS X are using Jenkins on systems that may be installed in areas not under the administrator's physical control. This is a real concern for many OS X developers who are not completely independent. Software that exists in a user's directory is always easier to exploit than software installed using root privileges.

          Randall Wood added a comment -

          @Sami Tikka:

          The only need addressed by running Jenkins in a GUI login session is running automated tests in the iOS simulator.
          Code signing for iOS and OS X builds can be done entirely from the CLI, even remotely over SSH.

          Randall Wood added a comment - @Sami Tikka: The only need addressed by running Jenkins in a GUI login session is running automated tests in the iOS simulator. Code signing for iOS and OS X builds can be done entirely from the CLI, even remotely over SSH.

          Sami Tikka added a comment -

          @Randall: OK, so it can be done, but is it well documented and supported by Xcode? If it's not, lots of programmers are going to try building their iOS projects on Jenkins and will be unhappy. If it just works, then we should be good with the existing installer that sets up a launch daemon. Apparently there are complications because there used to be lots of people asking how to get signing to work when building under Jenkins. (There might still be but I have been away from Jenkins community for a while.)

          Well, there's the iOS simulator... What do we do about that?

          Sami Tikka added a comment - @Randall: OK, so it can be done, but is it well documented and supported by Xcode? If it's not, lots of programmers are going to try building their iOS projects on Jenkins and will be unhappy. If it just works, then we should be good with the existing installer that sets up a launch daemon. Apparently there are complications because there used to be lots of people asking how to get signing to work when building under Jenkins. (There might still be but I have been away from Jenkins community for a while.) Well, there's the iOS simulator... What do we do about that?

          cowwoc added a comment -

          @Randall

          • First rule of security: anyone with physical access to a computer can subvert any protection you put in (they can remove the HD and probe it using an external Linux/OSX installation). I don't think it makes sense to protect against such a case.
          • I can't believe you'd want to give Jenkins more permissions than it need (root privileges) because when it inevitably gets exploited (Jenkins is not the most secure software in the world) your entire OS just got exploited as well. I'd much rather run Jenkins under a normal (non-admin) user account.

          @Sami Tikka: I'm trying to run the iOS simulator under a Jenkins slave under OSX. As far as I know, this requires us to install Jenkins under ~/LaunchAgents.

          cowwoc added a comment - @Randall First rule of security: anyone with physical access to a computer can subvert any protection you put in (they can remove the HD and probe it using an external Linux/OSX installation). I don't think it makes sense to protect against such a case. I can't believe you'd want to give Jenkins more permissions than it need (root privileges) because when it inevitably gets exploited (Jenkins is not the most secure software in the world) your entire OS just got exploited as well. I'd much rather run Jenkins under a normal (non-admin) user account. @Sami Tikka: I'm trying to run the iOS simulator under a Jenkins slave under OSX. As far as I know, this requires us to install Jenkins under ~/LaunchAgents.

          Randall Wood added a comment -
          • Installing something so that it is writable only as root does not, in any way, imply that that software has root privileges, it in fact, reduces the privalegs that software has. I never said I wanted Jenkins to have more permissions (i.e. run as root), I said I wanted the jenkins service account to not be able to replace Jenkins with arbitrary software (i.e. Jenkins must be installed by a user other than the jenkins service account, who can assume root privileges to do so, but the jenkins service account does not get those privileges).
          • I am fully aware of the limits of physical protection, and I am aware that there are basic minimum good security practices for the physical protection of computing assets, but also know that there is a difference between completely ignoring security concerns, protocols, and regulations as a software provider, or placing the burden of making a decision to operate a computer in an unsafe manner on the user.

          Randall Wood added a comment - Installing something so that it is writable only as root does not, in any way, imply that that software has root privileges, it in fact, reduces the privalegs that software has. I never said I wanted Jenkins to have more permissions (i.e. run as root), I said I wanted the jenkins service account to not be able to replace Jenkins with arbitrary software (i.e. Jenkins must be installed by a user other than the jenkins service account, who can assume root privileges to do so, but the jenkins service account does not get those privileges). I am fully aware of the limits of physical protection, and I am aware that there are basic minimum good security practices for the physical protection of computing assets, but also know that there is a difference between completely ignoring security concerns, protocols, and regulations as a software provider, or placing the burden of making a decision to operate a computer in an unsafe manner on the user.

          Randall Wood added a comment -

          Re code signing: This is best handled by the Xcode Plugin. See https://wiki.jenkins-ci.org/display/JENKINS/Xcode+Plugin#XcodePlugin-Signing

          Re UI Automation scripts (using iOS Simulator): Everything I have found suggests this requires that the jenkins process must run as an administrative user with elevated privileges. I do not know how elevated.

          My suggestion is that we stick to a standard OS X .pkg installer that creates a launchd.plist within the Jenkins installation area, and provides the user with instructions to copy the .plist file to /Library/LaunchDaemons or to ${HOME}/Library/LaunchAgents bases on the user's needs. Alternately, a tool (a shell script so it could be run via SSH) that guides the user though making that decision could be provided.

          Another possible suggestion is that a full (master) install of Jenkins should always be a LaunchDaemon (gets less privileges than the LaunchAgent) and that a Jenkins JNLP slave should be run (possibly on the same machine) for UI Automation. This suggestion is because the master listens for inbound connections, while a JNLP slave initiates its connection to the master, but does not listen for inbound connections.

          Randall Wood added a comment - Re code signing: This is best handled by the Xcode Plugin. See https://wiki.jenkins-ci.org/display/JENKINS/Xcode+Plugin#XcodePlugin-Signing Re UI Automation scripts (using iOS Simulator): Everything I have found suggests this requires that the jenkins process must run as an administrative user with elevated privileges. I do not know how elevated. My suggestion is that we stick to a standard OS X .pkg installer that creates a launchd.plist within the Jenkins installation area, and provides the user with instructions to copy the .plist file to /Library/LaunchDaemons or to ${HOME}/Library/LaunchAgents bases on the user's needs. Alternately, a tool (a shell script so it could be run via SSH) that guides the user though making that decision could be provided. Another possible suggestion is that a full (master) install of Jenkins should always be a LaunchDaemon (gets less privileges than the LaunchAgent) and that a Jenkins JNLP slave should be run (possibly on the same machine) for UI Automation. This suggestion is because the master listens for inbound connections, while a JNLP slave initiates its connection to the master, but does not listen for inbound connections.

          cowwoc added a comment -

          So... I made a major discovery.

          If you want to run a OSX slave that can interact with the GUI, all you need to do is log in that user locally. Meaning, you can:

          1. Launch an SSH connection, try launching the iOS simulator and it will fail.
          2. Log in the user locally
          3. Try launching the iOS simulator from the SSH connection from step 1, and it'll work.

          So, for the easiest configuration of an OSX slave I recommend the following:

          1. Launch the slave from SSH as you would Linux (this is the easiest option to configure and maintain)
          2. If you don't need access to the GUI, stop here. Otherwise, do the following:
          3. Enable automatic logins for the Jenkins user as described at http://apple.stackexchange.com/q/43413/21181
          4. Install the Slave Setup plugin
          5. Under "Manage Jenkins -> Configure System -> Slave Setups" add a new entry
          6. Set "setup files directory" to an absolute directory on your Jenkins Master computer, say "/var/jenkins/scripts/osx"
          7. Set "setup script after copy" to:
            chmod u+x prepare-slave.sh
            ./prepare-slave.sh
            
          8. Create the following file in the scripts directory:
            #!/bin/bash
            /usr/bin/security unlock-keychain -p jenkins-keychain-password
            
            # Wait for the user to log in locally before allowing jobs to run, otherwise iOS simulator will not launch
            TIME_ELAPSED=0
            while true; do
              pgrep -u builds -f -l -q "/System/Library/CoreServices/SystemUIServer\.app/Contents/MacOS/SystemUIServer"
              [ $? -eq 0 ] && break
              ((TIME_ELAPSED++))
              echo "Waiting for \"builds\" to log in locally [$TIME_ELAPSED / 60]..."
              if [ $TIME_ELAPSED -ge 60 ]; then
                exit 1
              fi
              sleep 1
            done
            echo
            

          The script at the end guarantees that the user finishes logging in locally before Jenkins will launch any jobs on the node. I suggest adding this documentation under https://wiki.jenkins-ci.org/display/JENKINS/Distributed+builds#Distributedbuilds-Differentwaysofstartingslaveagents

          cowwoc added a comment - So... I made a major discovery. If you want to run a OSX slave that can interact with the GUI, all you need to do is log in that user locally. Meaning, you can: Launch an SSH connection, try launching the iOS simulator and it will fail. Log in the user locally Try launching the iOS simulator from the SSH connection from step 1, and it'll work. So, for the easiest configuration of an OSX slave I recommend the following: Launch the slave from SSH as you would Linux (this is the easiest option to configure and maintain) If you don't need access to the GUI, stop here. Otherwise, do the following: Enable automatic logins for the Jenkins user as described at http://apple.stackexchange.com/q/43413/21181 Install the Slave Setup plugin Under "Manage Jenkins -> Configure System -> Slave Setups" add a new entry Set "setup files directory" to an absolute directory on your Jenkins Master computer, say "/var/jenkins/scripts/osx" Set "setup script after copy" to: chmod u+x prepare-slave.sh ./prepare-slave.sh Create the following file in the scripts directory: #!/bin/bash /usr/bin/security unlock-keychain -p jenkins-keychain-password # Wait for the user to log in locally before allowing jobs to run, otherwise iOS simulator will not launch TIME_ELAPSED=0 while true ; do pgrep -u builds -f -l -q "/ System /Library/CoreServices/SystemUIServer\.app/Contents/MacOS/SystemUIServer" [ $? -eq 0 ] && break ((TIME_ELAPSED++)) echo "Waiting for \" builds\ " to log in locally [$TIME_ELAPSED / 60]..." if [ $TIME_ELAPSED -ge 60 ]; then exit 1 fi sleep 1 done echo The script at the end guarantees that the user finishes logging in locally before Jenkins will launch any jobs on the node. I suggest adding this documentation under https://wiki.jenkins-ci.org/display/JENKINS/Distributed+builds#Distributedbuilds-Differentwaysofstartingslaveagents

          Chris Felix added a comment -

          I know this thread is old, but I was able to give a standard Mac user called jenkins permission to run the ios simulator by adding it to the _developer group

          sudo dscl . append /Groups/_developer GroupMembership jenkins

          Reference: https://blog.pivotal.io/pivotal-labs/labs/ios-ci-jenkins

          Chris Felix added a comment - I know this thread is old, but I was able to give a standard Mac user called jenkins permission to run the ios simulator by adding it to the _developer group sudo dscl . append /Groups/_developer GroupMembership jenkins Reference: https://blog.pivotal.io/pivotal-labs/labs/ios-ci-jenkins

          Russell Dobda added a comment -

          Russell Dobda added a comment - This issue seems to now be a showstopper. We can no longer even create iOS builds anymore because of the setup as of XCode 12. https://stackoverflow.com/questions/62875323/xcode-12-beta-cannot-find-simulator-runtime/64503950#64503950 and https://developer.apple.com/forums/thread/664731#664731021 and https://developer.apple.com/documentation/xcode-release-notes/xcode-12_2-beta-release-notes

            Unassigned Unassigned
            cowwoc cowwoc
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated: