Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-61936

AWS Artifact-Manager-S3-plugin uploads in series not parallel

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Looking at  https://github.com/jenkinsci/artifact-manager-s3-plugin and the uploads appear to happen in series

      for  10K, 5KB files upload time is around 20 mins, for a 100MB zip file it is a few second.

      I’ve tried to fix it, but I’m not a Java developer or  have any Jenkins plugin development experience , and have no idea what I’m doing.

      The original line,

      https://github.com/jenkinsci/artifact-manager-s3-plugin/blob/master/src/main/java/io/jenkins/plugins/artifact_manager_jclouds/JCloudsArtifactManager.java#L189

       

        for (Map.Entry<String, URL> entry : artifactUrls.entrySet())

      {                     client.uploadFile(new File(f, entry.getKey()), contentTypes.get(entry.getKey()), entry.getValue(), listener);                 }

       

      My change,

       

        Map.Entry<String, URL> entry : artifactUrls.entrySet().parallelStream().forEach( 

                          client.uploadFile(new File(f, entry.getKey()), contentTypes.get(entry.getKey()), entry.getValue(), listener));

        Attachments

          Activity

          beryl_factset Beryl Snyder created issue -
          hbetts Hutson Betts made changes -
          Field Original Value New Value
          Status Open [ 1 ] In Progress [ 3 ]
          hbetts Hutson Betts made changes -
          Status In Progress [ 3 ] Open [ 1 ]
          hbetts Hutson Betts made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          hbetts Hutson Betts made changes -
          Assignee Beryl Snyder [ beryl_factset ]

            People

            Assignee:
            beryl_factset Beryl Snyder
            Reporter:
            beryl_factset Beryl Snyder
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated: