Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-60652

Plugin breaks credential list views when it cannot contact AWS

    XMLWordPrintable

    Details

    • Similar Issues:
    • Released As:
      0.1.3

      Description

      When this plugin cannot contact AWS it breaks screens that list credentials.

      Examples:

      This can easily reproduced by just installing the plugin on a local instance and not configuring anything more.  

      The exception looks like this:

      com.amazonaws.SdkClientException: Unable to find a region via the region provider chain. Must provide an explicit region in the builder or setup environment to supply a region.
      	at com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:462)
      	at com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:424)
      	at com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46)
      	at io.jenkins.plugins.credentials.secretsmanager.AwsCredentialsProvider.fetchCredentials(AwsCredentialsProvider.java:103)
      	at com.google.common.base.Suppliers$ExpiringMemoizingSupplier.get(Suppliers.java:173)
      	at io.jenkins.plugins.credentials.secretsmanager.AwsCredentialsProvider.getCredentials(AwsCredentialsProvider.java:61)
      	at com.cloudbees.plugins.credentials.CredentialsProvider.getCredentials(CredentialsProvider.java:1147)
      	at com.cloudbees.plugins.credentials.CredentialsProvider.getCredentials(CredentialsProvider.java:1222)
      	at com.cloudbees.plugins.credentials.CredentialsProvider.lookupCredentials(CredentialsProvider.java:549)
      	at com.cloudbees.plugins.credentials.CredentialsProvider.findCredentialById(CredentialsProvider.java:906)
      	at com.cloudbees.plugins.credentials.CredentialsProvider.findCredentialById(CredentialsProvider.java:850)
      	at org.jenkinsci.plugins.credentialsbinding.MultiBinding.getCredentials(MultiBinding.java:144)
      	at org.jenkinsci.plugins.credentialsbinding.impl.UsernamePasswordMultiBinding.bind(UsernamePasswordMultiBinding.java:75)
      	at org.jenkinsci.plugins.credentialsbinding.impl.BindingStep$Execution2.doStart(BindingStep.java:135) 

       

        Attachments

          Issue Links

            Activity

            Hide
            chriskilding Chris Kilding added a comment -

            You see this error on the global configuration page because it may attempt to use credential providers (not just this one, others too) before they are ready, where the mechanism for configuring the provider is also on the global configuration page. I reported this issue here.

            Show
            chriskilding Chris Kilding added a comment - You see this error on the global configuration page because it may attempt to use credential providers (not just this one, others too) before they are ready, where the mechanism for configuring the provider is also on the global configuration page. I reported this issue here.
            Hide
            chriskilding Chris Kilding added a comment - - edited

            From what you've said I think there are 3 broad error cases to look at:

            • Listing credentials: there is indeed a bug here. Instead of throwing the exception it should probably return an uncached empty credential list, and log a warning in the Jenkins logs.
            • Get credential value within this provider: The credentials provider standard document insists that remote providers must throw an exception if they are asked to provide the value but can't contact their backing store. This will crash the build, but there is no choice here.
            • Get credential value only from other providers, on a Jenkins server where this plugin just happens to be installed: Yep, sounds like a bug too. I'm still working out what the behaviour should be here.

            We should also clarify the correct behaviour in the credentials provider standard document, which currently only specifies what should happen when getting the value fails (throw exception).

            Show
            chriskilding Chris Kilding added a comment - - edited From what you've said I think there are 3 broad error cases to look at: Listing credentials: there is indeed a bug here. Instead of throwing the exception it should probably return an uncached empty credential list, and log a warning in the Jenkins logs. Get credential value within this provider: The credentials provider standard document insists that remote providers must throw an exception if they are asked to provide the value but can't contact their backing store. This will crash the build, but there is no choice here. Get credential value only from other providers, on a Jenkins server where this plugin just happens to be installed: Yep, sounds like a bug too. I'm still working out what the behaviour should be here. We should also clarify the correct behaviour in the credentials provider standard document, which currently only specifies what should happen when getting the value fails (throw exception).
            Hide
            chriskilding Chris Kilding added a comment -

            This will take time to work out, so in the meantime if you're wondering how we make this work at our place...

            • We manage our AWS accounts almost entirely with Terraform
            • We use either immutable infrastructure tools (AMI builders, Docker etc plus Jenkins Configuration As Code) or configuration management tools (Puppet etc) to provision servers, so while we try to support manual plugin configuration through the Global Configuration Web UI, it's not a code path we actively use ourselves.
            • We have non-production AWS accounts where we can test all infrastructure changes before rolling them out to production.

            This combination of things probably meant we were able to catch misconfiguration bugs in the credential provider before the bugs ever got near our production Jenkins.

            Show
            chriskilding Chris Kilding added a comment - This will take time to work out, so in the meantime if you're wondering how we make this work at our place... We manage our AWS accounts almost entirely with Terraform We use either immutable infrastructure tools (AMI builders, Docker etc plus Jenkins Configuration As Code) or configuration management tools (Puppet etc) to provision servers, so while we try to support manual plugin configuration through the Global Configuration Web UI, it's not a code path we actively use ourselves. We have non-production AWS accounts where we can test all infrastructure changes before rolling them out to production. This combination of things probably meant we were able to catch misconfiguration bugs in the credential provider before the bugs ever got near our production Jenkins.
            Hide
            imod Dominik Bartholdi added a comment -

            Thanks for the info - I have my setup similar to yours (with CloudFormation, JasC), but until now, the Jenkins Docker Image I create also used to work on my local env (my mac) - but now this unfortunately is not the case anymore and its harder to do some first local testing of the image.

            The most important part for me right now is your first part "Listing credentials" - this effectively hinders my workflow right now.

            I don't know the internals of the credential providers, but from a users point of view: I have a job configured e.g. a pipeline and the only thing to reference credentials are by a string-id. For me as a pipeline admin, I don't care which provider actually holds the credentials. If it is not found by any of the providers, it should throw an exception. So there is no way for me to say that I want this credentials retrieved rather from AWS and not from the internal jenkins provider. Saying this, I don't understand why a provider should throw an exception if it can't resolve credentials for a given ID (unless the credentials framework swallows the exceptions and interprets it as a "NotFound" to hand over to the next provider installed until one can resolve it).

            Show
            imod Dominik Bartholdi added a comment - Thanks for the info - I have my setup similar to yours (with CloudFormation, JasC), but until now, the Jenkins Docker Image I create also used to work on my local env (my mac) - but now this unfortunately is not the case anymore and its harder to do some first local testing of the image. The most important part for me right now is your first part "Listing credentials" - this effectively hinders my workflow right now. I don't know the internals of the credential providers, but from a users point of view: I have a job configured e.g. a pipeline and the only thing to reference credentials are by a string-id. For me as a pipeline admin, I don't care which provider actually holds the credentials. If it is not found by any of the providers, it should throw an exception. So there is no way for me to say that I want this credentials retrieved rather from AWS and not from the internal jenkins provider. Saying this, I don't understand why a provider should throw an exception if it can't resolve credentials for a given ID (unless the credentials framework swallows the exceptions and interprets it as a "NotFound" to hand over to the next provider installed until one can resolve it).
            Hide
            imod Dominik Bartholdi added a comment -

            maybe its worth breaking this issue up into multiple, meaning: fix the low hanging fruits. After all, I think a plugin should never make jenkins unusable just because it is installed but not configured and not used.

            Show
            imod Dominik Bartholdi added a comment - maybe its worth breaking this issue up into multiple, meaning: fix the low hanging fruits. After all, I think a plugin should never make jenkins unusable just because it is installed but not configured and not used.
            Hide
            chriskilding Chris Kilding added a comment -

            Yes, we'll break it up, and address the list views first.

            Show
            chriskilding Chris Kilding added a comment - Yes, we'll break it up, and address the list views first.
            Hide
            chriskilding Chris Kilding added a comment -

            Started work on the pull request to fix the bug.

            Show
            chriskilding Chris Kilding added a comment - Started work on the pull request to fix the bug.

              People

              Assignee:
              chriskilding Chris Kilding
              Reporter:
              imod Dominik Bartholdi
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: