Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-52046

Jenkins build error with aws codepipeline and S3

XMLWordPrintable

      Hi,

      I am building a CI/CD pipeine using Git Enterprise to S3, then to Jenkins for build and AWS CodeDeploy for deployment. All these steps are managed by AWS CodePipeline. I was able to put codes from Github Enterprise to S3 bucket, however, when build jobs triggered, the zip package not making it over to the Jenkins instance that living in the same VPC. I use AWS Codepipeline plugin on Jenkins and the pipeline works fine when I pull repository directly from the public GitHub. It just doesn't work when I pull the repo from S3 bucket. The user and role I use are setup exactly like what's described in this document https://docs.aws.amazon.com/codepipeline/latest/userguide/tutorials-four-stage-pipeline.html#tutorials-four-stage-pipeline-prerequisites Tried to fix that for the last couple of days but no luck and hoping you can provide some guidence.

      The error I am getting showing below:

      ------------ 
Started by an SCM change Building in workspace /var/lib/jenkins/workspace/harrystage1 [AWS CodePipeline Plugin] Job '489208c6-a513-41dc-85a8-cc022cc516b8' received [AWS CodePipeline Plugin] Acknowledged job with ID: 489208c6-a513-41dc-85a8-cc022cc516b8 [AWS CodePipeline Plugin] Clearing workspace '/var/lib/jenkins/workspace/harrystage1' before download [AWS CodePipeline Plugin] Detected compression type: Zip [AWS CodePipeline Plugin] Successfully downloaded artifact from AWS CodePipeline [AWS CodePipeline Plugin] Extracting '/var/lib/jenkins/workspace/harrystage1/7LXLdqC.zip' to '/var/lib/jenkins/workspace/harrystage1' [AWS CodePipeline Plugin] Failed to acquire artifacts: The compressed input file contains files targeting an invalid destination: ./ [AWS CodePipeline Plugin] Stacktrace: [AWS CodePipeline Plugin] com.amazonaws.codepipeline.jenkinsplugin.ExtractionTools.getDestinationFile(ExtractionTools.java:114) [AWS CodePipeline Plugin] com.amazonaws.codepipeline.jenkinsplugin.ExtractionTools.extractZipFile(ExtractionTools.java:70) [AWS CodePipeline Plugin] com.amazonaws.codepipeline.jenkinsplugin.ExtractionTools.extractZip(ExtractionTools.java:46) [AWS CodePipeline Plugin] com.amazonaws.codepipeline.jenkinsplugin.ExtractionTools.decompressFile(ExtractionTools.java:176) [AWS CodePipeline Plugin] com.amazonaws.codepipeline.jenkinsplugin.DownloadCallable.downloadAndExtract(DownloadCallable.java:127) [AWS CodePipeline Plugin] com.amazonaws.codepipeline.jenkinsplugin.DownloadCallable.invoke(DownloadCallable.java:87) [AWS CodePipeline Plugin] com.amazonaws.codepipeline.jenkinsplugin.DownloadCallable.invoke(DownloadCallable.java:36) [AWS CodePipeline Plugin] hudson.FilePath.act(FilePath.java:997) [AWS CodePipeline Plugin] hudson.FilePath.act(FilePath.java:975) [AWS CodePipeline Plugin] com.amazonaws.codepipeline.jenkinsplugin.AWSCodePipelineSCM.checkout(AWSCodePipelineSCM.java:234) [AWS CodePipeline Plugin] hudson.model.AbstractProject.checkout(AbstractProject.java:1212) [AWS CodePipeline Plugin] hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:560) [AWS CodePipeline Plugin] jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) [AWS CodePipeline Plugin] hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:485) [AWS CodePipeline Plugin] hudson.model.Run.execute(Run.java:1735) [AWS CodePipeline Plugin] hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) [AWS CodePipeline Plugin] hudson.model.ResourceController.execute(ResourceController.java:97) [AWS CodePipeline Plugin] hudson.model.Executor.run(Executor.java:415) [AWS CodePipeline Plugin] Build was aborted [AWS CodePipeline Plugin] Publishing artifacts [AWS CodePipeline Plugin] Build failed, calling PutJobFailureResult Build step 'AWS CodePipeline Publisher' marked build as failure Finished: ABORTED

          [JENKINS-52046] Jenkins build error with aws codepipeline and S3

          Harry Lau added a comment -

          This file name threw me off 7LXLdqC.zip

          The filename that in S3 bucket actually different than the one shows in the log.  The filename changed to 7LXLdqC.zip

          When I manually download the zip file from S3 and ran the unzip command, it shows ok but nothing got extracted. See below output:

          root@jenkins:/var/lib/jenkins/workspace/myweb# unzip -t /var/lib/jenkins/workspace/myweb/stage_branch_test.zip
          Archive: /var/lib/jenkins/workspace/myweb/stage_branch_test.zip
          testing: ./ OK
          testing: appspec.yml OK
          testing: index.html OK
          testing: scripts/ OK
          testing: scripts/restart_server OK
          testing: scripts/before_install OK
          No errors detected in compressed data of /var/lib/jenkins/workspace/myweb/bt_harrystage_branch_test.zip.

          When I do a list, nothing got unzipped.
          root@jenkins:/var/lib/jenkins/workspace/myweb# ls -la
          total 12
          drwxr-xr-x 2 jenkins jenkins 4096 Jun 19 21:15 .
          drwxr-xr-x 3 jenkins jenkins 4096 Jun 19 21:14 ..

          Harry Lau added a comment - This file name threw me off  7LXLdqC.zip The filename that in S3 bucket actually different than the one shows in the log.  The filename changed to  7LXLdqC.zip When I manually download the zip file from S3 and ran the unzip command, it shows ok but nothing got extracted. See below output: root@jenkins:/var/lib/jenkins/workspace/myweb# unzip -t /var/lib/jenkins/workspace/myweb/stage_branch_test.zip Archive: /var/lib/jenkins/workspace/myweb/stage_branch_test.zip testing: ./ OK testing: appspec.yml OK testing: index.html OK testing: scripts/ OK testing: scripts/restart_server OK testing: scripts/before_install OK No errors detected in compressed data of /var/lib/jenkins/workspace/myweb/bt_harrystage_branch_test.zip. When I do a list, nothing got unzipped. root@jenkins:/var/lib/jenkins/workspace/myweb# ls -la total 12 drwxr-xr-x 2 jenkins jenkins 4096 Jun 19 21:15 . drwxr-xr-x 3 jenkins jenkins 4096 Jun 19 21:14 ..

          Felipe Almeida added a comment -

          Thanks, that helps. We will investigate the issue post an update to this ticket soon.

           

          > When I do a list, nothing got unzipped.

          that's expected, the -t option won't actually extract zip contents.

           

          Felipe Almeida added a comment - Thanks, that helps. We will investigate the issue post an update to this ticket soon.   > When I do a list, nothing got unzipped. that's expected, the -t option won't actually extract zip contents.  

          Felipe Almeida added a comment -

          We managed to reproduce the error and we're working on a fix.

           

          The problem is caused by the zip entry named ./.

          As a workaround, while we work on the fix, if you have control over the process that's creating the zip file, try modifying it so that ./ is not added to the zip.

          Felipe Almeida added a comment - We managed to reproduce the error and we're working on a fix.   The problem is caused by the zip entry named ./. As a workaround, while we work on the fix, if you have control over the process that's creating the zip file, try modifying it so that ./ is not added to the zip.

          Harry Lau added a comment -

          Are there other workarounds besides modify how the files are zipped?

          What is the ETA on fixing this issue on Jenkins?

          Harry Lau added a comment - Are there other workarounds besides modify how the files are zipped? What is the ETA on fixing this issue on Jenkins?

          Felipe Almeida added a comment -

          Another potential workaround is downgrading to 0.35, but I advise you not to do so, as version 0.36 contains an important fix, please see: https://wiki.jenkins.io/display/JENKINS/AWS+CodePipeline+Plugin

           

          Unfortunately I can't comment on the ETA for the fix

          Felipe Almeida added a comment - Another potential workaround is downgrading to 0.35, but I advise you not to do so, as version 0.36 contains an important fix, please see: https://wiki.jenkins.io/display/JENKINS/AWS+CodePipeline+Plugin   Unfortunately I can't comment on the ETA for the fix

          Harry Lau added a comment -

          Can we downgrade to 0.34?

          Will that be a more stable version to use without the major bugs?

          Harry Lau added a comment - Can we downgrade to 0.34? Will that be a more stable version to use without the major bugs?

          Felipe Almeida added a comment -

          Version 0.34 won't work with newer Jenkins versions

          Felipe Almeida added a comment - Version 0.34 won't work with newer Jenkins versions

          Harry Lau added a comment -

          What is the compatible version of Jenkins that codepipeline-0.34 works with? And where can I obtain the older version of the .hpi file of the plugin?

          We will use the old working version until the bug is fixed.  

          Harry Lau added a comment - What is the compatible version of Jenkins that codepipeline-0.34 works with? And where can I obtain the older version of the .hpi file of the plugin? We will use the old working version until the bug is fixed.  

          Felipe Almeida added a comment -

          Hi,

          the fix is pushed to https://github.com/awslabs/aws-codepipeline-plugin-for-jenkins but the plugin release will still take a few days, as a workaround, you can download the source and build the plugin using the following instructions:

          1. $ git clone https://github.com/awslabs/aws-codepipeline-plugin-for-jenkins.git

          2. $ cd aws-codepipeline-plugin-for-jenkins

          3. $ mvn package

          4. Go to you jenkins instance plugin management page (Manage Jenkins > Manage Plugins)

          5. Click Advanced tab

          6. In Upload Plugin section, click browse and select aws-codepipeline.hpi in aws-codepipeline-plugin-for-jenkins/target

          7. Click upload

          8. Select Restart Jenkins when installation is complete and no jobs are running checkbox

           

          That should fix your issue.

           

          Note: To build the plugin Maven 2 and Java 7 or 8 are required, the build will fail with Java 9+

           

           

           

          Felipe Almeida added a comment - Hi, the fix is pushed to https://github.com/awslabs/aws-codepipeline-plugin-for-jenkins but the plugin release will still take a few days, as a workaround, you can download the source and build the plugin using the following instructions: 1. $ git clone https://github.com/awslabs/aws-codepipeline-plugin-for-jenkins.git 2. $ cd aws-codepipeline-plugin-for-jenkins 3. $ mvn package 4. Go to you jenkins instance plugin management page (Manage Jenkins > Manage Plugins) 5. Click Advanced tab 6. In Upload Plugin section, click browse and select aws-codepipeline.hpi in aws-codepipeline-plugin-for-jenkins/target 7. Click upload 8. Select Restart Jenkins when installation is complete and no jobs are running checkbox   That should fix your issue.   Note: To build the plugin Maven 2 and Java 7 or 8 are required, the build will fail with Java 9+      

          Felipe Almeida added a comment -

          AWS CodePipeline plugin 0.38 was just released, it contains a fix for this issue.

           

          See: https://wiki.jenkins.io/display/JENKINS/AWS+CodePipeline+Plugin

          Felipe Almeida added a comment - AWS CodePipeline plugin 0.38 was just released, it contains a fix for this issue.   See: https://wiki.jenkins.io/display/JENKINS/AWS+CodePipeline+Plugin

            fgalmeida Felipe Almeida
            wah0310 Harry Lau
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: