Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-14470

S3 Plugin switches credential profiles

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • s3-plugin
    • None

    Description

      We use the S3 plugin in both a newer Jenkins install (v1.471, waiting for some important stuff to clear before we update it to 1.474) and a few older Hudson builds. The current Jenkins w/S3 (v0.3) works for a while, then all our builds stop uploading to S3 with an "Access Denied" error from Amazon. The builds on the older Hudson/S3 still work to the same bucket, and our credentials still work when I test with s3fs on the same machine. Error from Jenkins build console of one failure (buckets and filenames changed to xxxxx):

      Archiving artifacts
      Publish artifacts to S3 Bucket Using S3 profile: builds2
      Publish artifacts to S3 Bucket bucket=xxxxxxx/store, file=xxxxxxx.latest.apk
      ERROR: Failed to upload files
      java.io.IOException: put Destination [bucketName=xxxxxxxxx, objectName=xxxxxxx/store/xxxxxxxxxxx.latest.apk]: Status Code: 403, AWS Request ID: 85F9CB0156BF7CF6, AWS Error Code: AccessDenied, AWS Error Message: Access Denied, S3 Extended Request ID: swP7ylP4SQNW3o9uP3alhBO/VZthqvuFxWiZ2Pn54mqhSHk2uEtSrIlhPXlJLPUM
      at hudson.plugins.s3.S3Profile.upload(S3Profile.java:82)
      at hudson.plugins.s3.S3BucketPublisher.perform(S3BucketPublisher.java:119)
      at hudson.tasks.BuildStepMonitor$2.perform(BuildStepMonitor.java:27)
      at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:717)
      at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:692)
      at hudson.model.Build$BuildExecution.post2(Build.java:183)
      at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:639)
      at hudson.model.Run.execute(Run.java:1509)
      at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
      at hudson.model.ResourceController.execute(ResourceController.java:88)
      at hudson.model.Executor.run(Executor.java:236)
      Build step 'Publish artifacts to S3 Bucket' changed build result to UNSTABLE

      Attachments

        Activity

          tmack0 Theral Mackey added a comment -

          Restarting (full stop, start) the Jenkins service in windows fixes this problem for a while, but it comes back eventually.

          tmack0 Theral Mackey added a comment - Restarting (full stop, start) the Jenkins service in windows fixes this problem for a while, but it comes back eventually.
          tmack0 Theral Mackey added a comment -

          This appears to be related to our using multiple S3 Profiles. We have ~18 projects on the problematic Jenkins server, most using the S3 uploader. We have 3 profiles setup in the general config area for S3. 2 are identical with the second being setup when the first set appeared to fail (adding credentials seems to temporarily get it working again as well). The other set was added for a different S3 bucket on a different project. We have noticed that when most builds go "unstable" due to S3 uploads failing, this other project still works. After restarting Jenkins to get the other projects working again, this project fails. It looks like Jenkins is switching credential sets between projects.

          tmack0 Theral Mackey added a comment - This appears to be related to our using multiple S3 Profiles. We have ~18 projects on the problematic Jenkins server, most using the S3 uploader. We have 3 profiles setup in the general config area for S3. 2 are identical with the second being setup when the first set appeared to fail (adding credentials seems to temporarily get it working again as well). The other set was added for a different S3 bucket on a different project. We have noticed that when most builds go "unstable" due to S3 uploads failing, this other project still works. After restarting Jenkins to get the other projects working again, this project fails. It looks like Jenkins is switching credential sets between projects.
          mikewatt Michael Watt added a comment -

          Unassigning. I'm not going to be able to pick this up in the near future.

          mikewatt Michael Watt added a comment - Unassigning. I'm not going to be able to pick this up in the near future.

          I'm logging what may be a related bug. I have noticed that the plugin does not like mixed profiles: i.e. some created that rely on an IAM role and others that rely on an Access Key & Secret Key. It will only display all profiles when they are the same. If they are mixed, it shows the first one, but none after that.

          abrookewood Antony Brooke-Wood added a comment - I'm logging what may be a related bug. I have noticed that the plugin does not like mixed profiles: i.e. some created that rely on an IAM role and others that rely on an Access Key & Secret Key. It will only display all profiles when they are the same. If they are mixed, it shows the first one, but none after that.
          hoang Hoang Tran added a comment -

          I've got the same issue using the latest LTS 1.x jenkins and v0.10.1 S3-Plugin.

          My setup has only 2 profiles both using AccessKey/SecretKey.

          We have been restarting jenkins all the time for the use of each profile (once the profile used, it is the one that is OK, the other one will have "Access Denied").

          hoang Hoang Tran added a comment - I've got the same issue using the latest LTS 1.x jenkins and v0.10.1 S3-Plugin. My setup has only 2 profiles both using AccessKey/SecretKey. We have been restarting jenkins all the time for the use of each profile (once the profile used, it is the one that is OK, the other one will have "Access Denied").

          Code changed in jenkins
          User: Akbashev Alexander
          Path:
          src/main/java/hudson/plugins/s3/callable/S3Callable.java
          http://jenkins-ci.org/commit/s3-plugin/4bcdbf30074176a46758e0b14d2c9816769dbd02
          Log:
          JENKINS-14470: S3 Plugin switches credential profiles (#88)

          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Akbashev Alexander Path: src/main/java/hudson/plugins/s3/callable/S3Callable.java http://jenkins-ci.org/commit/s3-plugin/4bcdbf30074176a46758e0b14d2c9816769dbd02 Log: JENKINS-14470 : S3 Plugin switches credential profiles (#88) JENKINS-14470 : S3 Plugin switches credential profiles

          Code changed in jenkins
          User: Alexander Akbashev
          Path:
          src/main/java/hudson/plugins/s3/callable/S3Callable.java
          http://jenkins-ci.org/commit/s3-plugin/8e9d9228bcfbbfaeb0bce23cdbe08240941fba4c
          Log:
          JENKINS-14470: S3 Plugin switches credential profiles

          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Alexander Akbashev Path: src/main/java/hudson/plugins/s3/callable/S3Callable.java http://jenkins-ci.org/commit/s3-plugin/8e9d9228bcfbbfaeb0bce23cdbe08240941fba4c Log: JENKINS-14470 : S3 Plugin switches credential profiles
          jimilian Alexander A added a comment -

          hoang, please, check new version (0.10.3), it should be fixed there.

          jimilian Alexander A added a comment - hoang , please, check new version (0.10.3), it should be fixed there.
          hoang Hoang Tran added a comment - - edited

          jimilian Upgraded, Tested (with latest 1.x stable Jenkins). It works like a charm. Perfect. Thank you ^^

          hoang Hoang Tran added a comment - - edited jimilian Upgraded, Tested (with latest 1.x stable Jenkins). It works like a charm. Perfect. Thank you ^^

          People

            jimilian Alexander A
            tmack0 Theral Mackey
            Votes:
            2 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: