• Icon: New Feature New Feature
    • Resolution: Unresolved
    • Icon: Minor Minor
    • s3-plugin
    • None

      The S3 plugin currently uploads a file with the key set equal to the base filename that is returned by the matcher.

      I would like to upload files with a key of the path that is returned by the matcher, i.e. if I have a matcher of 'el/**', and several files underneath, I would like to see the following keys in the S3 bucket:

      el/noarch/file1.txt
      el/some/path/file2.c
      el/some/other/path/file3.java
      el/afile
      el/readme

      and so on.

          [JENKINS-16025] Allow full path when uploading a file

          Ray H added a comment -

          IMO it seems that if we want to make this less 'magical', we should allow the user to specify a prefix path, i.e. if I have a file at foo/bar/baz/qux.zip.

          Then I'd specify:

          prefixPath = foo/bar
          search=baz/*

          so that when I upload to `myBucket`, it should uploaded to s3://myBucket/baz/qux.zip` whereas the current behavior: of specifying sourceFile = `foo/bar/baz/*` would upload it to the root directory.

          Ray H added a comment - IMO it seems that if we want to make this less 'magical', we should allow the user to specify a prefix path, i.e. if I have a file at foo/bar/baz/qux.zip. Then I'd specify: prefixPath = foo/bar search=baz/* so that when I upload to `myBucket`, it should uploaded to s3://myBucket/baz/qux.zip` whereas the current behavior: of specifying sourceFile = `foo/bar/baz/*` would upload it to the root directory.

          robmoore added a comment -

          Just noticed this change in version 7: https://github.com/jenkinsci/s3-plugin/commit/0a098fcaa9d42e53f04264b9da1a79770ecdad9e

          Curious if this addresses part of this ticket by offering the ability to 'flatten' the directory?

          From the docs:

          When enabled, Jenkins will ignore the directory structure of the artifacts in
          the source project and copy all matching artifacts directly into the specified
          bucket. By default the artifacts are copied in the same directory structure as
          the source project.

          robmoore added a comment - Just noticed this change in version 7: https://github.com/jenkinsci/s3-plugin/commit/0a098fcaa9d42e53f04264b9da1a79770ecdad9e Curious if this addresses part of this ticket by offering the ability to 'flatten' the directory? From the docs: When enabled, Jenkins will ignore the directory structure of the artifacts in the source project and copy all matching artifacts directly into the specified bucket. By default the artifacts are copied in the same directory structure as the source project.

          Is anyone planning on fixing this? I am seeing the same behaviour as Joshua K 12/June/14. If not, I could have a look at it...

          Justin Santa Barbara added a comment - Is anyone planning on fixing this? I am seeing the same behaviour as Joshua K 12/June/14. If not, I could have a look at it...

          Joshua K added a comment -

          Hey Justin, I found that if I use 'flatten directories' option in 0.7, this piece of code is skipped and it just uses the basename of the file, which is what i wanted anyway. You can add multiple upload configuration blocks if you need to put stuff in S3 sub-buckets.

          Joshua K added a comment - Hey Justin, I found that if I use 'flatten directories' option in 0.7, this piece of code is skipped and it just uses the basename of the file, which is what i wanted anyway. You can add multiple upload configuration blocks if you need to put stuff in S3 sub-buckets.

          Joshua K added a comment -

          ...Not to say that this shouldn't be fixed anyway, but i find this an effective workaround.

          Joshua K added a comment - ...Not to say that this shouldn't be fixed anyway, but i find this an effective workaround.

          It appears that the reported issues on Windows with \'s being passed literally is resolved by this outstanding pull request:
          https://github.com/jenkinsci/s3-plugin/pull/55

          Richard Brooks added a comment - It appears that the reported issues on Windows with \'s being passed literally is resolved by this outstanding pull request: https://github.com/jenkinsci/s3-plugin/pull/55

          Jeff Grimmett added a comment -

          I'm seeing the same issue as Joshua K with regards to truncated file names. Attempting the "flatten directories" option to see if that will see us through, and hoping our build planners don't try to get too clever with file patterns O.o

          Jeff Grimmett added a comment - I'm seeing the same issue as Joshua K with regards to truncated file names. Attempting the "flatten directories" option to see if that will see us through, and hoping our build planners don't try to get too clever with file patterns O.o

          Joshua Spence added a comment -

          +1 on this issue.

          Joshua Spence added a comment - +1 on this issue.

          Kevin R. added a comment -

          for the love of god, bump.

          Kevin R. added a comment - for the love of god, bump.

          For the folks who replied after me or anyone else that comes across this issue, I've been very happy with just using the AWS CLI tool for uploading my artifacts as a post-build step. I have greater faith in Amazon maintaining their own command line tool than some third-party Jenkins plugin. However, be aware that it does introduce a dependency of the command line tool being installed on your slaves as well as the inject-passwords-as-environment-variables plugin (or however else you'd like to provision the command line tool with credentials).

          Here's a gist of what I'm doing to upload the entire contents of a `./artifacts` directory in my job's workspace while preserving the hierarchy of any files within that directory:
          https://gist.github.com/richard-bt/f35723140043fada2a3416d4f5b76b81

          (of course, modify the scripts to artifact files according to your needs - but I prefer to put the logic for how things should look in scripts that arrange the contents of my `./artifacts` directory prior to my post-build artifact step)

          After using Jenkins for a while I've adopted a preference for minimizing the number of dependencies I have on Jenkins plugins for doing things I could otherwise accomplish with a shell script - especially in having to support jobs running on various operating systems.

          Richard Brooks added a comment - For the folks who replied after me or anyone else that comes across this issue, I've been very happy with just using the AWS CLI tool for uploading my artifacts as a post-build step. I have greater faith in Amazon maintaining their own command line tool than some third-party Jenkins plugin. However, be aware that it does introduce a dependency of the command line tool being installed on your slaves as well as the inject-passwords-as-environment-variables plugin (or however else you'd like to provision the command line tool with credentials). Here's a gist of what I'm doing to upload the entire contents of a `./artifacts` directory in my job's workspace while preserving the hierarchy of any files within that directory: https://gist.github.com/richard-bt/f35723140043fada2a3416d4f5b76b81 (of course, modify the scripts to artifact files according to your needs - but I prefer to put the logic for how things should look in scripts that arrange the contents of my `./artifacts` directory prior to my post-build artifact step) After using Jenkins for a while I've adopted a preference for minimizing the number of dependencies I have on Jenkins plugins for doing things I could otherwise accomplish with a shell script - especially in having to support jobs running on various operating systems.

            dmbeer David Beer
            wizard113 wizard113
            Votes:
            8 Vote for this issue
            Watchers:
            17 Start watching this issue

              Created:
              Updated: