Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-48588

Artifacts uploaded to S3 does not show up in Blue Ocean

    • Icon: New Feature New Feature
    • Resolution: Unresolved
    • Icon: Major Major
    • s3-plugin
    • None
    • Blue Ocean 1.3.5
      S3 publisher plugin 0.10.12

      I expect files uploaded to S3 via that plugin to be listed on the Artifacts page of the Blue Ocean interface, but they are not.

          [JENKINS-48588] Artifacts uploaded to S3 does not show up in Blue Ocean

          James Dumay added a comment -

          jimilian seems like a feature request rather than a bug. What plugin are you using? Could you please provide a simplified example Jenkinsfile?

          James Dumay added a comment - jimilian seems like a feature request rather than a bug. What plugin are you using? Could you please provide a simplified example Jenkinsfile?

          Ilya Kulakov added a comment -

          jamesdumay See the Environment for the list of my plugins. If it's missing something, please let me know.

          My Jenkinsfile roughly looks like this:

          #!/usr/bin/env groovy
          
          pipeline {
              agent {
                  dockerfile {
                      filename 'Dockerfile.ci'
                  }
              }
          
              stages {
                  stage('Test') {
                      steps {
                          sh 'python setup.py test --pytest-args "--junitxml=junit.xml --cov=.. --cov-report=xml:coverage.xml --cov-branch --ignore=setup.py"'
                          cobertura autoUpdateHealth: false,
                              autoUpdateStability: false,
                              coberturaReportFile: 'coverage.xml',
                              conditionalCoverageTargets: '70, 0, 0',
                              failUnhealthy: true,
                              failUnstable: true,
                              lineCoverageTargets: '80, 0, 0',
                              maxNumberOfBuilds: 0,
                              methodCoverageTargets: '80, 0, 0',
                              sourceEncoding: 'ASCII',
                              zoomCoverageChart: false
                          junit 'junit.xml'
                      }
                  }
                  stage('Publish') {
                      steps {
                          sh 'python setup.py bdist_wheel'
                      }
                  }
              }
              post {
                  success {
                      step([
                          $class: 'S3BucketPublisher',
                          consoleLogLevel: 'INFO',
                          dontWaitForConcurrentBuildCompletion: false,
                          entries: [[
                              bucket: '...',
                              excludedFile: '',
                              flatten: false,
                              gzipFiles: false,
                              keepForever: true,
                              managedArtifacts: true,
                              noUploadOnFailure: true,
                              selectedRegion: 'us-east-1',
                              showDirectlyInBrowser: false,
                              sourceFile: 'requirements.txt, dist/...-*.whl',
                              storageClass: 'STANDARD',
                              uploadFromSlave: true,
                              useServerSideEncryption: false
                          ]],
                          pluginFailureResultConstraint: 'FAILURE',
                          profileName: 'jenkins',
                          userMetadata: []
                      ])
                  }
              }
          }
          

           

          Ilya Kulakov added a comment - jamesdumay See the Environment for the list of my plugins. If it's missing something, please let me know. My Jenkinsfile roughly looks like this: #!/usr/bin/env groovy pipeline { agent { dockerfile { filename 'Dockerfile.ci' } } stages { stage( 'Test' ) { steps { sh 'python setup.py test --pytest-args "--junitxml=junit.xml --cov=.. --cov-report=xml:coverage.xml --cov-branch --ignore=setup.py" ' cobertura autoUpdateHealth: false , autoUpdateStability: false , coberturaReportFile: 'coverage.xml' , conditionalCoverageTargets: '70, 0, 0' , failUnhealthy: true , failUnstable: true , lineCoverageTargets: '80, 0, 0' , maxNumberOfBuilds: 0, methodCoverageTargets: '80, 0, 0' , sourceEncoding: 'ASCII' , zoomCoverageChart: false junit 'junit.xml' } } stage( 'Publish' ) { steps { sh 'python setup.py bdist_wheel' } } } post { success { step([ $class: 'S3BucketPublisher' , consoleLogLevel: 'INFO' , dontWaitForConcurrentBuildCompletion: false , entries: [[ bucket: '...' , excludedFile: '', flatten: false , gzipFiles: false , keepForever: true , managedArtifacts: true , noUploadOnFailure: true , selectedRegion: 'us-east-1' , showDirectlyInBrowser: false , sourceFile: 'requirements.txt, dist/...-*.whl' , storageClass: 'STANDARD' , uploadFromSlave: true , useServerSideEncryption: false ]], pluginFailureResultConstraint: 'FAILURE' , profileName: 'jenkins' , userMetadata: [] ]) } } }  

          Ben Dean added a comment -

          You can have a much more simple example than that. Try this:

          node {
              stage('build'){
                  sh 'echo foo > foo.txt'
                  s3Upload profileName: 'example', entries: [
                          [
                                  bucket          : 'fake-bucket', 
                                  selectedRegion  : 'us-east-1', 
                                  sourceFile      : 'foo.txt',
                                  managedArtifacts: true,
                          ]
                  ]
              }
          }
          

          And the place it's not showing up in Blue Ocean is here:

          Ben Dean added a comment - You can have a much more simple example than that. Try this: node { stage( 'build' ){ sh 'echo foo > foo.txt' s3Upload profileName: 'example' , entries: [ [ bucket : 'fake-bucket' , selectedRegion : 'us-east-1' , sourceFile : 'foo.txt' , managedArtifacts: true , ] ] } } And the place it's not showing up in Blue Ocean is here:

          James Dumay added a comment -

          kentzo I see though I don't think thats the intent of the S3 plugin to display uploaded artifacts. There is an extension point to do that in Blue Ocean (BlueArtifact and BlueArtifactFactory) but it would have to be provided by the S3 plugin.

          James Dumay added a comment - kentzo I see though I don't think thats the intent of the S3 plugin to display uploaded artifacts. There is an extension point to do that in Blue Ocean (BlueArtifact and BlueArtifactFactory) but it would have to be provided by the S3 plugin.

          Ben Dean added a comment -

          jamesdumay, it displays them in the classic UI (although not in the same way as normal archived artifacts). The managedArtifacts option tells the step manage the artifacts in the same way normal artifacts would be managed on the Jenkins master. Here's what the documentation for that option says:

          When enabled, this lets Jenkins fully manage the artifacts, exactly like it does when the artifacts are published to the master.
          In this case, the artifacts are stored in the "jobs/[job]/[build-number]/" path in the selected bucket and prefix path. This means the following features are enabled:

          • artifacts are finger printed and linked to the build
          • artifacts can be downloaded directly from the build page in the S3 Artifact section
          • artifacts are automatically deleted when the build is deleted
          • the S3 Copy Artifact build step can be used to download artifacts from S3 automatically, helping build complex pipelines

          Because of that, I would expect the artifacts to show up in Blue Ocean too (either in the artifacts section or some new S3 Artifacts section like in the classic UI (although, personally I'd rather all the artifacts display in the same place, regardless of where they are archived)). This is not to dispute what you said about the extension points in Blue Ocean, but rather the idea that the S3 plugin doesn't display uploaded artifacts (because it definitely does, just not in Blue Ocean ).

          Ben Dean added a comment - jamesdumay , it displays them in the classic UI (although not in the same way as normal archived artifacts). The managedArtifacts option tells the step manage the artifacts in the same way normal artifacts would be managed on the Jenkins master. Here's what the documentation for that option says: When enabled, this lets Jenkins fully manage the artifacts, exactly like it does when the artifacts are published to the master. In this case, the artifacts are stored in the "jobs/ [job] / [build-number] /" path in the selected bucket and prefix path. This means the following features are enabled: artifacts are finger printed and linked to the build artifacts can be downloaded directly from the build page in the S3 Artifact section artifacts are automatically deleted when the build is deleted the S3 Copy Artifact build step can be used to download artifacts from S3 automatically, helping build complex pipelines Because of that, I would expect the artifacts to show up in Blue Ocean too (either in the artifacts section or some new S3 Artifacts section like in the classic UI (although, personally I'd rather all the artifacts display in the same place, regardless of where they are archived)). This is not to dispute what you said about the extension points in Blue Ocean, but rather the idea that the S3 plugin doesn't display uploaded artifacts (because it definitely does, just not in Blue Ocean ).

          James Dumay added a comment -

          b_dean I stand corrected then We use the standard API for retrieving the artifacts, perhaps the display of the S3 artifacts is custom somehow? Would be worth someone investigating.

          James Dumay added a comment - b_dean I stand corrected then We use the standard API for retrieving the artifacts, perhaps the display of the S3 artifacts is custom somehow? Would be worth someone investigating.

          Ilya Kulakov added a comment -

          jamesdumay

          I see though I don't think thats the intent of the S3 plugin to display uploaded artifacts.

          In the task description I did not make an assumption whose job it is. Only that together these plugins don't play well.

           Would be worth someone investigating.

          What would be the standard API for registering artifacts to be retrievable via the standard API?
          As far as I understand, the plugins generates temporary authorized S3 link dynamically upon each user request. That behavior should be retained.
           

          Ilya Kulakov added a comment - jamesdumay I see though I don't think thats the intent of the S3 plugin to display uploaded artifacts. In the task description I did not make an assumption whose job it is. Only that together these plugins don't play well.  Would be worth someone investigating. What would be the standard API for registering artifacts to be retrievable via the standard API? As far as I understand, the plugins generates temporary authorized S3 link dynamically upon each user request. That behavior should be retained.  

          James Dumay added a comment -

          kentzo we query the run for its artifacts, transform them to a BlueArtifact model and display them.

          In order to get S3 plugin to display, you would need something like what I wrote in PR-113. I won't be implementing this but it should be a start for someone here who wants to.

          James Dumay added a comment - kentzo we query the run for its artifacts, transform them to a BlueArtifact model and display them . In order to get S3 plugin to display, you would need something like what I wrote in PR-113 . I won't be implementing this but it should be a start for someone here who wants to.

            jimilian Alexander A
            kentzo Ilya Kulakov
            Votes:
            4 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated: