Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-61111

Soft-deleted AWS Secrets Manager secrets still appear in Jenkins

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Reported by Ethan Stein

      Given I have an AWS secret that is being used as a Jenkins credential,
      When I soft-delete the secret (mark it as deleted) and it is still in its recovery window,
      Then the secret is still seen in Jenkins.

      Soft-deleted secrets should be hidden from Jenkins instead, as they are not intended to be used.

        Attachments

          Activity

          Hide
          chriskilding Chris Kilding added a comment - - edited

          This is interesting because in theory we already have integration tests for this very scenario, which pass, demonstrating that it works:

          • CredentialsProviderIT#shouldTolerateDeletedCredentials
          • CredentialsProviderIT#shouldTolerateRecentlyDeletedCredentials

          If that’s correct then what Ethan saw was probably a one-off event. If not, something may be wrong in this test’s soft deletion logic.

          Show
          chriskilding Chris Kilding added a comment - - edited This is interesting because in theory we already have integration tests for this very scenario, which pass, demonstrating that it works: CredentialsProviderIT#shouldTolerateDeletedCredentials CredentialsProviderIT#shouldTolerateRecentlyDeletedCredentials If that’s correct then what Ethan saw was probably a one-off event. If not, something may be wrong in this test’s soft deletion logic.
          Hide
          chriskilding Chris Kilding added a comment - - edited

          The bit I missed was that the bug description does not say whether the deletion was recent i.e. within the 5 minute cache window.

          It is expected behaviour that if a secret is deleted within the cache window, its entry will continue to be shown (though it will not be functional). It is indeed an idiosyncrasy in the user experience that we would rather not have, but it is unavoidable given the current polling strategy of integrating with Secrets Manager: there is no way for Jenkins to know that the secret is (soft-)deleted until it refreshes the cache and calls Secrets Manager again.

          When the time comes to refresh the cache, the ListSecretsOperation dutifully filters out soft-deleted secrets. This has been tested and is known to work.

          Show
          chriskilding Chris Kilding added a comment - - edited The bit I missed was that the bug description does not say whether the deletion was recent i.e. within the 5 minute cache window. It is expected behaviour that if a secret is deleted within the cache window, its entry will continue to be shown (though it will not be functional). It is indeed an idiosyncrasy in the user experience that we would rather not have, but it is unavoidable given the current polling strategy of integrating with Secrets Manager: there is no way for Jenkins to know that the secret is (soft-)deleted until it refreshes the cache and calls Secrets Manager again. When the time comes to refresh the cache, the ListSecretsOperation dutifully filters out soft-deleted secrets. This has been tested and is known to work.
          Hide
          chriskilding Chris Kilding added a comment -

          In the absence of further information I'll have to assume that Ethan soft-deleted a secret and loaded the Jenkins credentials page within the 5-minute cache window. As mentioned, this is not something that can be fixed as long as we have to poll Secrets Manager for data.

          If, in the future, AWS allows Secrets Manager API clients to subscribe for updates on a push basis, then we could revisit this.

          Show
          chriskilding Chris Kilding added a comment - In the absence of further information I'll have to assume that Ethan soft-deleted a secret and loaded the Jenkins credentials page within the 5-minute cache window. As mentioned, this is not something that can be fixed as long as we have to poll Secrets Manager for data. If, in the future, AWS allows Secrets Manager API clients to subscribe for updates on a push basis, then we could revisit this.

            People

            Assignee:
            chriskilding Chris Kilding
            Reporter:
            chriskilding Chris Kilding
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: