Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-70252

Artifact Manager on S3 does not delete artifacts from s3 when job is deleted

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • artifact-manager-s3-plugin
    • None
    • Jenkins 2.361.3 LTS

      Jenkins artifacts are not deleted from s3 bucket when a pipeline job get deleted or a branch get deleted in a multi branch pipeline.

       

      How to reproduce

      • Use any git repo to create a jenkins multi branch pipeline named produce_bug using the Jenkinsfile below.
      • Create a branch named testing
      • Build the branch testing twice
      • Assuming the build numbers are #1 and #2, artifacts will be uploaded to s3 to produce_bug/testing/1/ and produce_bug/testing/2/ folders

      First test case: 

      • delete build #1 using the jenkins button Delete build #1 
        • Directory produce_bug/testing/1/ is being deleted from s3
        • Works as expected 

      Second test case: 

      • Delete the branch testing
        • The job for the branch testing is being deleted from s3
        • The directory produce_bug/testing/ is not being deleted from s3
        • Does not work as expected! produce_bug/testing/ should be deleted from s3

      Third test case: 

      • Delete the multi branch pipeline produce_bug
        • The directory produce_bug ** is not being deleted from s3
        • Does not work as expected! produce_bug/ should be deleted

       

       

      // Jenkinsfile
pipeline {
    agent any
    stages {
        stage("demo") {
            steps {
                writeFile file: 'f', text: 'some content here'
                archiveArtifacts 'f'
            }
        }
    }
}

       

       

          [JENKINS-70252] Artifact Manager on S3 does not delete artifacts from s3 when job is deleted

          walid boussafa created issue -
          walid boussafa made changes -
          Description Original: Jenkins artifacts are not deleted from s3 bucket when `Use Transfer Acceleration` is enabled. 
          everything works as expected when `Use Transfer Acceleration` is unchecked.            		New: Jenkins artifacts are not deleted from s3 bucket when `Use Transfer Acceleration` is enabled. 
          everything works as expected when `Use Transfer Acceleration` is unchecked.

           

          How to reproduce:
           * Create two dummy jenkins jobs that save some artifacts
           * Enable `Use Transfer Acceleration`
           * Delete the first jenkins job
           * Check artifacts on s3, THEY ARE NOT DELETED!!!
           * Disable `Use Transfer Acceleration`
           * Delete the second jenkins job
           * Now artifacts are being deleted from s3 as expected. 

          Jesse Glick added a comment -

          Are you setting a system property? The plugin by default does not ever delete anything (since this is expected to be done at the AWS policy level).

          Jesse Glick added a comment - Are you setting a system property? The plugin by default does not ever delete anything (since this is expected to be done at the AWS policy level).

          walid boussafa added a comment -

          jglick yes I set both:

          -Dio.jenkins.plugins.artifact_manager_jclouds.s3.S3BlobStoreConfig.deleteArtifacts=true

           and

          -Dio.jenkins.plugins.artifact_manager_jclouds.s3.S3BlobStoreConfig.deleteStashes=true

          walid boussafa added a comment - jglick yes I set both: -Dio.jenkins.plugins.artifact_manager_jclouds.s3.S3BlobStoreConfig.deleteArtifacts= true  and -Dio.jenkins.plugins.artifact_manager_jclouds.s3.S3BlobStoreConfig.deleteStashes= true
          walid boussafa made changes -
          Summary Original: Artifact Manager on S3 does not delete artifacts from s3 when use Use Transfer Acceleration is enabled New: Artifact Manager on S3 does not delete artifacts from s3 when job is deleted
          walid boussafa made changes -
          Description Original: Jenkins artifacts are not deleted from s3 bucket when `Use Transfer Acceleration` is enabled. 
          everything works as expected when `Use Transfer Acceleration` is unchecked.

           

          How to reproduce:
           * Create two dummy jenkins jobs that save some artifacts
           * Enable `Use Transfer Acceleration`
           * Delete the first jenkins job
           * Check artifacts on s3, THEY ARE NOT DELETED!!!
           * Disable `Use Transfer Acceleration`
           * Delete the second jenkins job
           * Now artifacts are being deleted from s3 as expected.            		New: Jenkins artifacts are not deleted from s3 bucket when a pipeline job get deleted or a branch get deleted in a multi branch pipeline.

           
          h2. How to reproduce
           * Use any git repo to create a jenkins multi branch pipeline named *produce_bug* using the Jenkinsfile below.
           * Create a branch named *testing*
           * Build the branch *testing* twice
           * Assuming the build numbers are *#1* and *#2,* artifacts will be uploaded to s3 to {*}produce_bug/{*}{*}testing/1/{*} and {*}produce_bug/{*}{*}testing/2/{*} folders

          h2. First test case: 
           * delete build *#1* using the jenkins button *Delete build #1* 
           ** Directory {*}produce_bug/{*}{*}testing/1/{*} is being deleted from s3
           ** {color:#57d9a3}Works as expected {color}

          h2. Second test case: 
           * Delete the branch testing

           ** The job for the branch *testing* is being deleted from s3
           ** The directory {*}produce_bug/{*}{*}testing/{*} is not being deleted from s3
           ** {color:#de350b}Does not work as expected! {color}{*}produce_bug/{*}{*}testing/{*} should be deleted from s3

          h2. Third test case: 
           * Delete the multi branch pipeline *produce_bug*

           ** The directory *produce_bug* ** is not being deleted from s3
           ** {color:#de350b}Does not work as expected! {color}{*}produce_bug{*}{*}/{*} should be deleted

           

           
          {code:java}
          // Jenkinsfile
          pipeline {
              agent any
              stages {
                  stage("demo") {
                      steps {
                          writeFile file: 'f', text: 'some content here'
                          archiveArtifacts 'f'
                      }
                  }
              }
          } {code}
           

           

          walid boussafa added a comment -

          jglick my initial testing was not so accurate! I've updated the bug description. 

          walid boussafa added a comment - jglick my initial testing was not so accurate! I've updated the bug description. 
          Alexey made changes -
          Assignee New: Alexey [ alkosenko ]
          Alexey made changes -
          Status Original: Open [ 1 ] New: In Progress [ 3 ]

          Alexey added a comment -

          Hi! We also faced this issue in our Jenkins instance. I found a problem in the deletion logic, the WorkflowJob deletes only files under it and does not call the deletion method for WorkflowRun (extends `hudson.model.Run`), where the correct logic for deleting artifacts is implemented. I fixed the second case in this Pull Request – https://github.com/jenkinsci/workflow-job-plugin/pull/463

          Alexey added a comment - Hi! We also faced this issue in our Jenkins instance. I found a problem in the deletion logic, the WorkflowJob deletes only files under it and does not call the deletion method for WorkflowRun (extends `hudson.model.Run`), where the correct logic for deleting artifacts is implemented. I fixed the second case in this Pull Request – https://github.com/jenkinsci/workflow-job-plugin/pull/463

            alkosenko Alexey
            boussaffawalid walid boussafa
            Votes:
            2 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated: