Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-48588

Artifacts uploaded to S3 does not show up in Blue Ocean

    XMLWordPrintable

    Details

    • Type: New Feature
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Component/s: s3-plugin
    • Labels:
      None
    • Environment:
      Blue Ocean 1.3.5
      S3 publisher plugin 0.10.12
    • Similar Issues:

      Description

      I expect files uploaded to S3 via that plugin to be listed on the Artifacts page of the Blue Ocean interface, but they are not.

        Attachments

          Activity

          kentzo Ilya Kulakov created issue -
          Hide
          jamesdumay James Dumay added a comment -

          Alexander A seems like a feature request rather than a bug. What plugin are you using? Could you please provide a simplified example Jenkinsfile?

          Show
          jamesdumay James Dumay added a comment - Alexander A seems like a feature request rather than a bug. What plugin are you using? Could you please provide a simplified example Jenkinsfile?
          jamesdumay James Dumay made changes -
          Field Original Value New Value
          Issue Type Bug [ 1 ] New Feature [ 2 ]
          Hide
          kentzo Ilya Kulakov added a comment -

          James Dumay See the Environment for the list of my plugins. If it's missing something, please let me know.

          My Jenkinsfile roughly looks like this:

          #!/usr/bin/env groovy
          
          pipeline {
              agent {
                  dockerfile {
                      filename 'Dockerfile.ci'
                  }
              }
          
              stages {
                  stage('Test') {
                      steps {
                          sh 'python setup.py test --pytest-args "--junitxml=junit.xml --cov=.. --cov-report=xml:coverage.xml --cov-branch --ignore=setup.py"'
                          cobertura autoUpdateHealth: false,
                              autoUpdateStability: false,
                              coberturaReportFile: 'coverage.xml',
                              conditionalCoverageTargets: '70, 0, 0',
                              failUnhealthy: true,
                              failUnstable: true,
                              lineCoverageTargets: '80, 0, 0',
                              maxNumberOfBuilds: 0,
                              methodCoverageTargets: '80, 0, 0',
                              sourceEncoding: 'ASCII',
                              zoomCoverageChart: false
                          junit 'junit.xml'
                      }
                  }
                  stage('Publish') {
                      steps {
                          sh 'python setup.py bdist_wheel'
                      }
                  }
              }
              post {
                  success {
                      step([
                          $class: 'S3BucketPublisher',
                          consoleLogLevel: 'INFO',
                          dontWaitForConcurrentBuildCompletion: false,
                          entries: [[
                              bucket: '...',
                              excludedFile: '',
                              flatten: false,
                              gzipFiles: false,
                              keepForever: true,
                              managedArtifacts: true,
                              noUploadOnFailure: true,
                              selectedRegion: 'us-east-1',
                              showDirectlyInBrowser: false,
                              sourceFile: 'requirements.txt, dist/...-*.whl',
                              storageClass: 'STANDARD',
                              uploadFromSlave: true,
                              useServerSideEncryption: false
                          ]],
                          pluginFailureResultConstraint: 'FAILURE',
                          profileName: 'jenkins',
                          userMetadata: []
                      ])
                  }
              }
          }
          

           

          Show
          kentzo Ilya Kulakov added a comment - James Dumay See the Environment for the list of my plugins. If it's missing something, please let me know. My Jenkinsfile roughly looks like this: #!/usr/bin/env groovy pipeline { agent { dockerfile { filename 'Dockerfile.ci' } } stages { stage( 'Test' ) { steps { sh 'python setup.py test --pytest-args "--junitxml=junit.xml --cov=.. --cov-report=xml:coverage.xml --cov-branch --ignore=setup.py" ' cobertura autoUpdateHealth: false , autoUpdateStability: false , coberturaReportFile: 'coverage.xml' , conditionalCoverageTargets: '70, 0, 0' , failUnhealthy: true , failUnstable: true , lineCoverageTargets: '80, 0, 0' , maxNumberOfBuilds: 0, methodCoverageTargets: '80, 0, 0' , sourceEncoding: 'ASCII' , zoomCoverageChart: false junit 'junit.xml' } } stage( 'Publish' ) { steps { sh 'python setup.py bdist_wheel' } } } post { success { step([ $class: 'S3BucketPublisher' , consoleLogLevel: 'INFO' , dontWaitForConcurrentBuildCompletion: false , entries: [[ bucket: '...' , excludedFile: '', flatten: false , gzipFiles: false , keepForever: true , managedArtifacts: true , noUploadOnFailure: true , selectedRegion: 'us-east-1' , showDirectlyInBrowser: false , sourceFile: 'requirements.txt, dist/...-*.whl' , storageClass: 'STANDARD' , uploadFromSlave: true , useServerSideEncryption: false ]], pluginFailureResultConstraint: 'FAILURE' , profileName: 'jenkins' , userMetadata: [] ]) } } }  
          b_dean Ben Dean made changes -
          Hide
          b_dean Ben Dean added a comment -

          You can have a much more simple example than that. Try this:

          node {
              stage('build'){
                  sh 'echo foo > foo.txt'
                  s3Upload profileName: 'example', entries: [
                          [
                                  bucket          : 'fake-bucket', 
                                  selectedRegion  : 'us-east-1', 
                                  sourceFile      : 'foo.txt',
                                  managedArtifacts: true,
                          ]
                  ]
              }
          }
          

          And the place it's not showing up in Blue Ocean is here:

          Show
          b_dean Ben Dean added a comment - You can have a much more simple example than that. Try this: node { stage( 'build' ){ sh 'echo foo > foo.txt' s3Upload profileName: 'example' , entries: [ [ bucket : 'fake-bucket' , selectedRegion : 'us-east-1' , sourceFile : 'foo.txt' , managedArtifacts: true , ] ] } } And the place it's not showing up in Blue Ocean is here:
          Hide
          jamesdumay James Dumay added a comment -

          Ilya Kulakov I see though I don't think thats the intent of the S3 plugin to display uploaded artifacts. There is an extension point to do that in Blue Ocean (BlueArtifact and BlueArtifactFactory) but it would have to be provided by the S3 plugin.

          Show
          jamesdumay James Dumay added a comment - Ilya Kulakov I see though I don't think thats the intent of the S3 plugin to display uploaded artifacts. There is an extension point to do that in Blue Ocean (BlueArtifact and BlueArtifactFactory) but it would have to be provided by the S3 plugin.
          jamesdumay James Dumay made changes -
          Component/s blueocean-plugin [ 21481 ]
          Hide
          b_dean Ben Dean added a comment -

          James Dumay, it displays them in the classic UI (although not in the same way as normal archived artifacts). The managedArtifacts option tells the step manage the artifacts in the same way normal artifacts would be managed on the Jenkins master. Here's what the documentation for that option says:

          When enabled, this lets Jenkins fully manage the artifacts, exactly like it does when the artifacts are published to the master.
          In this case, the artifacts are stored in the "jobs/[job]/[build-number]/" path in the selected bucket and prefix path. This means the following features are enabled:

          • artifacts are finger printed and linked to the build
          • artifacts can be downloaded directly from the build page in the S3 Artifact section
          • artifacts are automatically deleted when the build is deleted
          • the S3 Copy Artifact build step can be used to download artifacts from S3 automatically, helping build complex pipelines

          Because of that, I would expect the artifacts to show up in Blue Ocean too (either in the artifacts section or some new S3 Artifacts section like in the classic UI (although, personally I'd rather all the artifacts display in the same place, regardless of where they are archived)). This is not to dispute what you said about the extension points in Blue Ocean, but rather the idea that the S3 plugin doesn't display uploaded artifacts (because it definitely does, just not in Blue Ocean ).

          Show
          b_dean Ben Dean added a comment - James Dumay , it displays them in the classic UI (although not in the same way as normal archived artifacts). The managedArtifacts option tells the step manage the artifacts in the same way normal artifacts would be managed on the Jenkins master. Here's what the documentation for that option says: When enabled, this lets Jenkins fully manage the artifacts, exactly like it does when the artifacts are published to the master. In this case, the artifacts are stored in the "jobs/ [job] / [build-number] /" path in the selected bucket and prefix path. This means the following features are enabled: artifacts are finger printed and linked to the build artifacts can be downloaded directly from the build page in the S3 Artifact section artifacts are automatically deleted when the build is deleted the S3 Copy Artifact build step can be used to download artifacts from S3 automatically, helping build complex pipelines Because of that, I would expect the artifacts to show up in Blue Ocean too (either in the artifacts section or some new S3 Artifacts section like in the classic UI (although, personally I'd rather all the artifacts display in the same place, regardless of where they are archived)). This is not to dispute what you said about the extension points in Blue Ocean, but rather the idea that the S3 plugin doesn't display uploaded artifacts (because it definitely does, just not in Blue Ocean ).
          Hide
          jamesdumay James Dumay added a comment -

          Ben Dean I stand corrected then We use the standard API for retrieving the artifacts, perhaps the display of the S3 artifacts is custom somehow? Would be worth someone investigating.

          Show
          jamesdumay James Dumay added a comment - Ben Dean I stand corrected then We use the standard API for retrieving the artifacts, perhaps the display of the S3 artifacts is custom somehow? Would be worth someone investigating.
          Hide
          kentzo Ilya Kulakov added a comment -

          James Dumay

          I see though I don't think thats the intent of the S3 plugin to display uploaded artifacts.

          In the task description I did not make an assumption whose job it is. Only that together these plugins don't play well.

           Would be worth someone investigating.

          What would be the standard API for registering artifacts to be retrievable via the standard API?
          As far as I understand, the plugins generates temporary authorized S3 link dynamically upon each user request. That behavior should be retained.
           

          Show
          kentzo Ilya Kulakov added a comment - James Dumay I see though I don't think thats the intent of the S3 plugin to display uploaded artifacts. In the task description I did not make an assumption whose job it is. Only that together these plugins don't play well.  Would be worth someone investigating. What would be the standard API for registering artifacts to be retrievable via the standard API? As far as I understand, the plugins generates temporary authorized S3 link dynamically upon each user request. That behavior should be retained.  
          Hide
          jamesdumay James Dumay added a comment -

          Ilya Kulakov we query the run for its artifacts, transform them to a BlueArtifact model and display them.

          In order to get S3 plugin to display, you would need something like what I wrote in PR-113. I won't be implementing this but it should be a start for someone here who wants to.

          Show
          jamesdumay James Dumay added a comment - Ilya Kulakov we query the run for its artifacts, transform them to a BlueArtifact model and display them . In order to get S3 plugin to display, you would need something like what I wrote in PR-113 . I won't be implementing this but it should be a start for someone here who wants to.

            People

            Assignee:
            jimilian Alexander A
            Reporter:
            kentzo Ilya Kulakov
            Votes:
            4 Vote for this issue
            Watchers:
            6 Start watching this issue

              Dates

              Created:
              Updated: