-
Bug
-
Resolution: Unresolved
-
Minor
-
None
-
artifact-manager-s3 1.4 (works on 1.1, fails on 1.2+)
Jenkins 2.164.3-SNAPSHOT
(I was about to file it as a blocker, but just realized actually this seems only to be an issue in the validation page, but enabling the plugin still archive artifacts fine, so filing it still because it's misleading to users but with lower priority – see git bisect log below)
Problem
When opening the /aws page, configuring the plugin and clicking on 'Validate S3 Bucket configuration', we get an error with the following stack trace:
GetBucketLocation failed com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: D910569B825E3D7C; S3 Extended Request ID: 49Hz3b5JOiRPXGCfP+5fySBgjHmp+iUXSPhqqWDdS2eRAqAo3IrZZlaKKCILTzBCkufWMsK1gpM=), S3 Extended Request ID: 49Hz3b5JOiRPXGCfP+5fySBgjHmp+iUXSPhqqWDdS2eRAqAo3IrZZlaKKCILTzBCkufWMsK1gpM= at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1695) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1350) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1101) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:758) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:732) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:714) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:674) at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:656) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:520) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4705) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4652) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4646) at com.amazonaws.services.s3.AmazonS3Client.getBucketLocation(AmazonS3Client.java:989) at com.amazonaws.services.s3.AmazonS3Client.getBucketLocation(AmazonS3Client.java:995) at io.jenkins.plugins.artifact_manager_jclouds.s3.S3BlobStoreConfig.checkGetBucketLocation(S3BlobStoreConfig.java:237) at io.jenkins.plugins.artifact_manager_jclouds.s3.S3BlobStoreConfig.doValidateS3BucketConfig(S3BlobStoreConfig.java:253) at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:627) at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:396) at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:408) at org.kohsuke.stapler.interceptor.RequirePOST$Processor.invoke(RequirePOST.java:77) at org.kohsuke.stapler.PreInvokeInterceptedFunction.invoke(PreInvokeInterceptedFunction.java:26) at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:212) at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:145) at org.kohsuke.stapler.MetaClass$11.doDispatch(MetaClass.java:537) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:739) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:870) at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:282) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:739) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:870) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:668) at org.kohsuke.stapler.Stapler.service(Stapler.java:238) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154) at com.cloudbees.jenkins.support.impl.cloudbees.UnrestrictedApiCallsMonitor$ApiMonitorFilter.doFilter(UnrestrictedApiCallsMonitor.java:120) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151) at org.jenkinsci.plugins.ssegateway.Endpoint$SSEListenChannelFilter.doFilter(Endpoint.java:243) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151) at io.jenkins.blueocean.auth.jwt.impl.JwtAuthenticationFilter.doFilter(JwtAuthenticationFilter.java:61) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151) at com.cloudbees.jenkins.support.slowrequest.SlowRequestFilter.doFilter(SlowRequestFilter.java:37) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151) at io.jenkins.blueocean.ResourceCacheControl.doFilter(ResourceCacheControl.java:134) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151) at jenkins.metrics.impl.MetricsFilter.doFilter(MetricsFilter.java:125) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151) at jenkins.telemetry.impl.UserLanguages$AcceptLanguageFilter.doFilter(UserLanguages.java:128) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:157) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:99) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:135) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249) at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:503) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) at java.lang.Thread.run(Thread.java:748)
To reproduce in short:
- Set up an IAM Instance Profile allowed to do everything on S3 (or with less permissions, your choice)
- Create an EC2 instance, use that Instance Profile
- Instance the plugin and open /aws
- configure and click Validate.
Easy way to set up everything using CloudFormation
Use Evergreen's AWS flavor: https://github.com/jenkins-infra/evergreen/tree/master/distribution/flavors/aws-ec2-cloud
Once provisioned, just connect to the EC2 instance through SSH, and run the WAR manually like java -jar jenkins.war --httpPort=8081, and copy the config from the Evergreen instance (or just get the bucket name from the AWS console, whatever works)
Bisect
git bisect log 7c69b02 git bisect start # bad: [7634ca43ec1ea11ac8c3e00fea234c107317c0b0] [maven-release-plugin] prepare release artifact-manager-s3-1.4 git bisect bad 7634ca43ec1ea11ac8c3e00fea234c107317c0b0 # good: [67a7e3a419214a983e34c7fe5c2c9ad4e9b99284] [maven-release-plugin] prepare release artifact-manager-s3-1.1 git bisect good 67a7e3a419214a983e34c7fe5c2c9ad4e9b99284 # bad: [c9d60bf2f88c300d656a287194c25c4b18e852cd] Merge pull request #82 from jenkinsci/ARC-576 git bisect bad c9d60bf2f88c300d656a287194c25c4b18e852cd # good: [7659f21cfa926eef87f787ace4ed4c52713c1a91] Merge pull request #78 from jenkinsci/metachars-JENKINS-50591-JENKINS-52151 git bisect good 7659f21cfa926eef87f787ace4ed4c52713c1a91 # skip: [f97d65ddb84140ac5e385dbabc5f0579cc68ea18] Merge branch 'master' into GetBucketLocation git bisect skip f97d65ddb84140ac5e385dbabc5f0579cc68ea18 # good: [f1216a60d6df001e3aedcebe3150406cd929c3d7] Missing imports. git bisect good f1216a60d6df001e3aedcebe3150406cd929c3d7 # good: [b08502d2b1da462e0994d8d934c898d46e67d14a] Merge pull request #79 from davidcurrie/ARC-480 git bisect good b08502d2b1da462e0994d8d934c898d46e67d14a # bad: [18ecbe3fe2b5c1466ca16c582385ab8c7c43016e] Check GetBucketLocation on validation git bisect bad 18ecbe3fe2b5c1466ca16c582385ab8c7c43016e # good: [7c69b02ba8b097ef32ee0a509407f5280dcb3af9] Re-enable ignored tests git bisect good 7c69b02ba8b097ef32ee0a509407f5280dcb3af9 # first bad commit: [18ecbe3fe2b5c1466ca16c582385ab8c7c43016e] Check GetBucketLocation on validation