• Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Major Major
    • xcode-plugin
    • None

      At the beginning of a build, after the keychain is copied the following commands are issued:
      security list-keychain -s <KeychainName>
      security default-keychain -d user <KeychainName>

      This creates some unintended effects.

      Imagine two jobs longjob and shortjob that are started in that order on jenkins.
      when longjob starts it sets the keychain-list and default keychain to its keychain, displays the signing identities and all seems right with the world. longjob starts its build..

      shortjob starts and sets the keychain-list and default keychain to its keychain, displays the signing identities and all seems right with the world.

      longjob is nearly finished and tries to sign the build, but alas, it can't find its keychain (because its keychain is not in the searchlist) and its signing credentials are not in shortjob's keychain (which is the default keychain, and the only keychain in the list) so longjob fails with a signing error.

      shortjob signs its build correctly and succeeds.

      A job's keychain should be added to the keychain list, instead of replacing the list.

          [JENKINS-24980] Keychain Management issue.

          lacostej added a comment -

          We have several mechanisms in the plugin today. Some should be deprecated in favor of others. The developerProfile Loader seems a better mechanism right now and doesn't have these issues (to my knowledge)

          lacostej added a comment - We have several mechanisms in the plugin today. Some should be deprecated in favor of others. The developerProfile Loader seems a better mechanism right now and doesn't have these issues (to my knowledge)

          To me the bigger concern is the only indication the user's keychain settings will be modified is in the global section and that implies the keychain will either be left on the search list or removed from the search list. Nowhere does it say "This plugin will change your default keychain and replace the search least while building".

          Since xcodebuild does not seem to support keychains other than on the search list (OTHER_CODE_SIGN_FLAGS is supposed to allow it, but I have never seen it work), I don't think the plugin should try to support it.

          The minimum the plugin should do is check the keychain search list and either warn/fail if the keychain is missing.

          Timothy Rundle added a comment - To me the bigger concern is the only indication the user's keychain settings will be modified is in the global section and that implies the keychain will either be left on the search list or removed from the search list. Nowhere does it say "This plugin will change your default keychain and replace the search least while building". Since xcodebuild does not seem to support keychains other than on the search list (OTHER_CODE_SIGN_FLAGS is supposed to allow it, but I have never seen it work), I don't think the plugin should try to support it. The minimum the plugin should do is check the keychain search list and either warn/fail if the keychain is missing.

          lacostej added a comment -

          Timothy thanks for your input.

          I am using OTHER_CODE_SIGN_FLAGS with the plugin. It works for me in combination of using the developer profile. What problems are you having with it ? My main issue is that it makes it more cumbersome for job configuration. Note: the OTHER_CODE_SIGN_FLAGS only work if the keychain as also been added to the search list! From the codesign documentation

          SIGNING IDENTITIES
          To be used for code signing, a digital identity must be stored in
          a keychain that is on the calling user's keychain search list.

          See http://prod.lists.apple.com/archives/xcode-users/2014/Dec/msg00041.html

          WRT atomicity, see http://lists.apple.com/archives/xcode-users/2014/Dec/msg00040.html to which someone answered to me offlist with 'man lockfile'for an idea on how to implement adding this in an atomic way

          See also https://github.com/jenkinsci/xcode-plugin/pull/56 for a pull request that I received with similar intent, and closed because I didn't like the implementation.

          To summarise,

          • we could add things atomically (using my patch in comment of pr #56, or using lockfile command)
          • we right now lack the ability to remove after usage (see below)
          • there's still the issue of deciding how long the keychain should be opened for use (needs to be somewhat configurable)
          • there's still the issue of making this keychain available at the same time for other processes

          I don't know yet how to solve it cleanly, and I don't know which use cases are the most important for the people. See also https://groups.google.com/d/msg/jenkinsci-dev/JrHvfNc3cQI/a4rAY1L8Ug0J for my request for input, so far without feedback.

          lacostej added a comment - Timothy thanks for your input. I am using OTHER_CODE_SIGN_FLAGS with the plugin. It works for me in combination of using the developer profile. What problems are you having with it ? My main issue is that it makes it more cumbersome for job configuration. Note: the OTHER_CODE_SIGN_FLAGS only work if the keychain as also been added to the search list! From the codesign documentation SIGNING IDENTITIES To be used for code signing, a digital identity must be stored in a keychain that is on the calling user's keychain search list. See http://prod.lists.apple.com/archives/xcode-users/2014/Dec/msg00041.html WRT atomicity, see http://lists.apple.com/archives/xcode-users/2014/Dec/msg00040.html to which someone answered to me offlist with 'man lockfile'for an idea on how to implement adding this in an atomic way See also https://github.com/jenkinsci/xcode-plugin/pull/56 for a pull request that I received with similar intent, and closed because I didn't like the implementation. To summarise, we could add things atomically (using my patch in comment of pr #56, or using lockfile command) we right now lack the ability to remove after usage (see below) there's still the issue of deciding how long the keychain should be opened for use (needs to be somewhat configurable) there's still the issue of making this keychain available at the same time for other processes I don't know yet how to solve it cleanly, and I don't know which use cases are the most important for the people. See also https://groups.google.com/d/msg/jenkinsci-dev/JrHvfNc3cQI/a4rAY1L8Ug0J for my request for input, so far without feedback.

          I found once I added the keychain to the search list codesign works whether or not the keychain is set in OTHER_CODE_SIGN_FLAGS.

          I haven't used developer profile, will test it out when I get a chance.

          Timothy Rundle added a comment - I found once I added the keychain to the search list codesign works whether or not the keychain is set in OTHER_CODE_SIGN_FLAGS. I haven't used developer profile, will test it out when I get a chance.

          lacostej added a comment -

          I think I was forced to add the OTHER_CODE_SIGN_FLAGS because it appeared that if a key is in 2 keychains, but one is locked, you sometimes need to be extra specific of the one you want to use.

          lacostej added a comment - I think I was forced to add the OTHER_CODE_SIGN_FLAGS because it appeared that if a key is in 2 keychains, but one is locked, you sometimes need to be extra specific of the one you want to use.

          OK, that makes sense. I haven't tried that scenario.

          I still have concerns about the plugin modifying the search list. You proposed patch on pr#56 seems safer than the current implementation. Do you try to remove the keychain later or once added it stays on the search list?

          Timothy Rundle added a comment - OK, that makes sense. I haven't tried that scenario. I still have concerns about the plugin modifying the search list. You proposed patch on pr#56 seems safer than the current implementation. Do you try to remove the keychain later or once added it stays on the search list?

          lacostej added a comment -

          Right I do nothing hence the reason for not pushing. I suspect that if we want to provide keychain management, we need to properly implement

          • adding to search list
          • timeout
          • removal to search list

          Maybe there should be one module that deals with the keychain.
          And then another one that deals with xcode. The first one would find a way to tell the xcode one which keychain to use.

          I am unsure of the implementation given the different solutions today.

          This requires some changes (probably have a buildwrapper or whatever this is called in the jenkins world).

          I would prefer some consensus first

          lacostej added a comment - Right I do nothing hence the reason for not pushing. I suspect that if we want to provide keychain management, we need to properly implement adding to search list timeout removal to search list Maybe there should be one module that deals with the keychain. And then another one that deals with xcode. The first one would find a way to tell the xcode one which keychain to use. I am unsure of the implementation given the different solutions today. This requires some changes (probably have a buildwrapper or whatever this is called in the jenkins world). I would prefer some consensus first

          Tom Jones added a comment -

          Currently plugin does this:

          launcher.launch().envs(envs).cmds("/usr/bin/security", "list-keychains", "-s", keychainPath).stdout(listener).pwd(projectRoot).join();

          which is the equivalent of this:

          security list-keychains -s ${KEYCHAIN_PATH}

          Needs to do this:
          security list-keychains | xargs security list-keychains -s "${KEYCHAIN_PATH}"

          And then when finished building, the plugin should do the equivalent of this:
          security delete-keychain "${KEYCHAIN_PATH}"

          Not entirely sure how the first part would be done in java.

          Unrelated matter that this also fixes, the Keychain is left in the workspace, and its password is shown in plain text in the environment variables, which allows a developer on your team to download the keychain, unlock and extract the private keys (which raises the question, why have a password?)

          Tom Jones added a comment - Currently plugin does this: launcher.launch().envs(envs).cmds("/usr/bin/security", "list-keychains", "-s", keychainPath).stdout(listener).pwd(projectRoot).join(); which is the equivalent of this: security list-keychains -s ${KEYCHAIN_PATH} Needs to do this: security list-keychains | xargs security list-keychains -s "${KEYCHAIN_PATH}" And then when finished building, the plugin should do the equivalent of this: security delete-keychain "${KEYCHAIN_PATH}" Not entirely sure how the first part would be done in java. Unrelated matter that this also fixes, the Keychain is left in the workspace, and its password is shown in plain text in the environment variables, which allows a developer on your team to download the keychain, unlock and extract the private keys (which raises the question, why have a password?)

          lacostej added a comment -

          Tom,

          a few things:

          • WRT to adding, yes there's some code more or less ready to support it. Nice use of the xargs for adding to the list It's still not 100% atomic, but the risk is low though.
          • I wasn't aware of the password being available in the log. I don't seem to see it in mine but I am maybe using another way of using the keychains. I checked the code and the command that use the keychain passwords are supposed to mask it when printing to the log.
          • I also do not see the keychain in my workspace

          lacostej added a comment - Tom, a few things: WRT to adding, yes there's some code more or less ready to support it. Nice use of the xargs for adding to the list It's still not 100% atomic, but the risk is low though. I wasn't aware of the password being available in the log. I don't seem to see it in mine but I am maybe using another way of using the keychains. I checked the code and the command that use the keychain passwords are supposed to mask it when printing to the log. I also do not see the keychain in my workspace

          ajohns added a comment -

          As I see it, a lot of the keychain issues with the plugin could be solved by supporting the --keychain flag of codesign utility when signing the app.

          Relying on a default keychain setting will always result in conflicts unless you only have a single executor.

          In the CodeSign area of setup, if a keychain file is specified, the plugin should add the --keychain flag to the codesign command. It's really unfortunate that the plugin does not currently do this, or even allow extra parameters to be configured for this command.

          ajohns added a comment - As I see it, a lot of the keychain issues with the plugin could be solved by supporting the --keychain flag of codesign utility when signing the app. Relying on a default keychain setting will always result in conflicts unless you only have a single executor. In the CodeSign area of setup, if a keychain file is specified, the plugin should add the --keychain flag to the codesign command. It's really unfortunate that the plugin does not currently do this, or even allow extra parameters to be configured for this command.

            Unassigned Unassigned
            tmjdisorder Tom Jones
            Votes:
            2 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: