Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-42931

Upload spec does not resolve placeholders in target path

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Critical Critical
    • artifactory-plugin
    • None
    • Ubuntu 16.04.1 LTS
      Jenkins version 2.32.1
      artifactory-plugin version 2.9.2

      This works with artifactory-plugin version 2.9.1.

       

      def uploadSpec = """{
       "files": [
         {
           "pattern": "local/${organization}/${moduleName}/(.*?)/(.*?)/(.*?)-SNAPSHOT/(.*?)/(.*?).jar",
           "regexp" : true,
           "target": "${repositoryName}/${organization}/${moduleName}/{1}/{2}/{3}/{4}/{5}.jar"
         }
       ]
      }"""

      with log output:

      For pattern: local/<orga>/<module>/(*)/(*)/(*)-SNAPSHOT/(*)/(*).jar 3 artifacts were found.
      Deploying artifact: http://<artifactory-server>/<repository>/<orga>/<module>/scala_2.10/sbt_0.13/1.0.2/jars/<module>.jar
      Deploying artifact: http://<artifactory-server>/<repository>/<orga>/<module>/scala_2.10/sbt_0.13/1.0.2/jars/<module>-sources.jar
      Deploying artifact: http://<artifactory-server>/<repository>/<orga>/<module>/scala_2.10/sbt_0.13/1.0.2/jars/<module>-javadoc.jar
      

       

      After an upgrade to 2.9.2, the placeholders are no longer resolved. Log shows:

      For pattern: local/<orga>/<module>/(*)/(*)/(*)-SNAPSHOT/(*)/(*).jar 3 artifacts were found.
      Deploying artifact: http://<artifactory-server>/<repository>/<orga>/<module>/{1}/{2}/{3}/{4}/{5}.jar

       

          [JENKINS-42931] Upload spec does not resolve placeholders in target path

          Josh Hasner added a comment -

          This issue seems to have returned in plugin version 2.11.

          The issue I originally reported was fixed as a part of https://www.jfrog.com/jira/browse/HAP-897

          I am not able to try and upgrade to 2.12 to see if the issue persists, and there is no option to downgrade in the Jenkins UI either.

          Can it be confirmed that this is also happening in 2.12?

          Josh Hasner added a comment - This issue seems to have returned in plugin version 2.11. The issue I originally reported was fixed as a part of https://www.jfrog.com/jira/browse/HAP-897 I am not able to try and upgrade to 2.12 to see if the issue persists, and there is no option to downgrade in the Jenkins UI either. Can it be confirmed that this is also happening in 2.12?

          Jan Krag added a comment -

          I have not finished diagnosing, but I seem to have the same problem on

          Jenkins Artifactory Plugin version: 2.12.2.

           

          My spec file (configured inline in a freestyle sandbox job):

          {
           "files": [
               {
                  "pattern": "release_(*)-(*)-(*)-(*)-(*)-(*)-(*).tar.gz", 
                  "target": "Sandbox/fpga/1.0.0-${BUILD_NUMBER}/${3}--${4}.tgz", 
                  "flat": "true"
               }
           ]
          }

           

          It finds the artifact but doesn't replace the placeholders.

          While writing this, my co-worker has found out that it works correctly if we switch to regex mode and rewrite the pattern to:

          {
          "files": [
              {
                "pattern": "release_(\d+)-(\d+)-(\d+)-(\d+)-(\d+)-(\d+)-(\d+).tar.gz",
                "target": "Sandbox/fpga/1.0.0-${BUILD_NUMBER}/{1}--{2}--{3}.tgz",
                "flat": "true",
                "regexp": "true"
              }
            ]
          }

           

          Jan Krag added a comment - I have not finished diagnosing, but I seem to have the same problem on Jenkins Artifactory Plugin version: 2.12.2.   My spec file (configured inline in a freestyle sandbox job): { "files" : [ { "pattern" : "release_(*)-(*)-(*)-(*)-(*)-(*)-(*).tar.gz" , "target" : "Sandbox/fpga/1.0.0-${BUILD_NUMBER}/${3}--${4}.tgz" , "flat" : " true " } ] }   It finds the artifact but doesn't replace the placeholders. While writing this, my co-worker has found out that it works correctly if we switch to regex mode and rewrite the pattern to: { "files" : [ { "pattern" : "release_(\d+)-(\d+)-(\d+)-(\d+)-(\d+)-(\d+)-(\d+).tar.gz" , "target" : "Sandbox/fpga/1.0.0-${BUILD_NUMBER}/{1}--{2}--{3}.tgz" , "flat" : " true " , "regexp" : " true " } ] }  

          jth08527, Sorry for the late response. 

          We have solved the issues above and added tests to prevent this from happening again.

          Our latest release version 2.12.2 supports full path as well as relative paths placeholders. 

          jkrag, Could you try and remove the '$' sign from your first attempt without the regexp?

          Roman Gurevitch added a comment - jth08527 , Sorry for the late response.  We have solved the issues above and added tests to prevent this from happening again. Our latest release version 2.12.2 supports full path as well as relative paths placeholders.  jkrag , Could you try and remove the '$' sign from your first attempt without the regexp?

          Josh Hasner added a comment -

          romang, the issue is not fixed with 2.12.2. The plugin was updated, and Jenkins restarted, here is the error:

          Jenkins Artifactory Plugin version: 2.12.2
          For pattern: /home/centos/ap/jenkins/workspace/silkhp/jobs/build/appcodebuild/BuildArtifacts//_-.zip 1 artifacts were found.
          Deploying artifact: http://<art_url>/generic-test/{1}/{2}/{1}_{2}-{3}.zip
          ERROR: remote file operation failed: /home/centos/ap/jenkins/workspace/silkhp/jobs/deploy/copy to artifactory at hudson.remoting.Channel@37bbdde7:unixaws: java.io.IOException: Failed to deploy file. Status code: 400

          Josh Hasner added a comment - romang , the issue is not fixed with 2.12.2. The plugin was updated, and Jenkins restarted, here is the error: Jenkins Artifactory Plugin version: 2.12.2 For pattern: /home/centos/ap/jenkins/workspace/silkhp/jobs/build/appcodebuild/BuildArtifacts// _ - .zip 1 artifacts were found. Deploying artifact: http://<art_url>/generic-test/{1}/{2}/{1}_{2}-{3}.zip ERROR: remote file operation failed: /home/centos/ap/jenkins/workspace/silkhp/jobs/deploy/copy to artifactory at hudson.remoting.Channel@37bbdde7:unixaws: java.io.IOException: Failed to deploy file. Status code: 400

          Josh,

          I created the below File Specs and local files and got the expected result. It could be the issue you got can be solved by using the below File Specs.

          I created the following 3 files on my file-system:

          eyalb-mac:spec-test eyalb$ ll
          total 0
          drwxr-xr-x   5 eyalb  staff  170 Aug  9 20:18 .
          drwxr-xr-x  29 eyalb  staff  986 Aug  9 20:13 ..
          -rw-r--r--   1 eyalb  staff    0 Aug  9 20:16 release_1-2-3-4-5-6-7.tar.gz
          -rw-r--r--   1 eyalb  staff    0 Aug  9 20:18 release_11-22-33-44-55-66-77.tar.gz
          -rw-r--r--   1 eyalb  staff    0 Aug  9 20:18 release_111-222-333-444-555-668-777.tar.gz

          And used the following File Spec:

          {
          "files": [
          {
          "pattern": "spec-test/release_(*)-(*)-(*)-(*)-(*)-(*)-(*).tar.gz",
          "target": "tests/fpga/1.0.0-${BUILD_NUMBER}/{1}--{2}--{3}.tgz",
          "flat": "true"
          }
          ]
          }

          I got the 3 files uploaded as the following build log shows:

          Building in workspace /Users/eyalb/.jenkins/workspace/generic-3
          Jenkins Artifactory Plugin version: 2.12.2-SNAPSHOT (private-08/09/2017 15:15-eyalb)
          For pattern: spec-test/release_(*)-(*)-(*)-(*)-(*)-(*)-(*).tar.gz 3 artifacts were found.
          Deploying artifact: http://localhost:8081/artifactory/tests/fpga/1.0.0-2/1--2--3.tgz
          Deploying artifact: http://localhost:8081/artifactory/tests/fpga/1.0.0-2/11--22--33.tgz
          Deploying artifact: http://localhost:8081/artifactory/tests/fpga/1.0.0-2/111--222--333.tgz
          Deploying build info to: http://localhost:8081/artifactory/api/build
          Finished: SUCCESS

          I then tried the same with a regular expression with the same 3 files in my file-system:

          {
          "files": [
          {
          "pattern": "spec-test/release_(\d+)-(\d+)-(\d+)-(\d+)-(\d+)-(\d+)-(\d+).tar.gz",
          "target": "tests/fpga/1.0.0-${BUILD_NUMBER}/{1}--{2}--{3}.tgz",
          "flat": "true",
          "regexp": "true"
          }
          ]
          }

          and got the expected result again:

          Started by user admin
          Building in workspace /Users/eyalb/.jenkins/workspace/generic-2
          Jenkins Artifactory Plugin version: 2.12.2-SNAPSHOT (private-08/09/2017 15:15-eyalb)
          For pattern: spec-test/release_(\d+)-(\d+)-(\d+)-(\d+)-(\d+)-(\d+)-(\d+).tar.gz 3 artifacts were found.
          Deploying artifact: http://localhost:8081/artifactory/tests/fpga/1.0.0-3/111--222--333.tgz
          Deploying artifact: http://localhost:8081/artifactory/tests/fpga/1.0.0-3/1--2--3.tgz
          Deploying artifact: http://localhost:8081/artifactory/tests/fpga/1.0.0-3/11--22--33.tgz
          Deploying build info to: http://localhost:8081/artifactory/api/build
          Finished: SUCCESS

           

          Eyal Ben Moshe added a comment - Josh, I created the below File Specs and local files and got the expected result. It could be the issue you got can be solved by using the below File Specs. I created the following 3 files on my file-system: eyalb-mac:spec-test eyalb$ ll total 0 drwxr-xr-x   5 eyalb  staff  170 Aug  9 20:18 . drwxr-xr-x  29 eyalb  staff  986 Aug  9 20:13 .. -rw-r--r--   1 eyalb  staff    0 Aug  9 20:16 release_1-2-3-4-5-6-7.tar.gz -rw-r--r--   1 eyalb  staff    0 Aug  9 20:18 release_11-22-33-44-55-66-77.tar.gz -rw-r--r--   1 eyalb  staff    0 Aug  9 20:18 release_111-222-333-444-555-668-777.tar.gz And used the following File Spec: { "files" : [ { "pattern" : "spec-test/release_(*)-(*)-(*)-(*)-(*)-(*)-(*).tar.gz" , "target" : "tests/fpga/1.0.0-${BUILD_NUMBER}/{1}--{2}--{3}.tgz" , "flat" : " true " } ] } I got the 3 files uploaded as the following build log shows: Building in workspace /Users/eyalb/.jenkins/workspace/ generic -3 Jenkins Artifactory Plugin version: 2.12.2-SNAPSHOT ( private -08/09/2017 15:15-eyalb) For pattern: spec-test/release_(*)-(*)-(*)-(*)-(*)-(*)-(*).tar.gz 3 artifacts were found. Deploying artifact: http: //localhost:8081/artifactory/tests/fpga/1.0.0-2/1--2--3.tgz Deploying artifact: http: //localhost:8081/artifactory/tests/fpga/1.0.0-2/11--22--33.tgz Deploying artifact: http: //localhost:8081/artifactory/tests/fpga/1.0.0-2/111--222--333.tgz Deploying build info to: http: //localhost:8081/artifactory/api/build Finished: SUCCESS I then tried the same with a regular expression with the same 3 files in my file-system: { "files" : [ { "pattern" : "spec-test/release_(\d+)-(\d+)-(\d+)-(\d+)-(\d+)-(\d+)-(\d+).tar.gz" , "target" : "tests/fpga/1.0.0-${BUILD_NUMBER}/{1}--{2}--{3}.tgz" , "flat" : " true " , "regexp" : " true " } ] } and got the expected result again: Started by user admin Building in workspace /Users/eyalb/.jenkins/workspace/ generic -2 Jenkins Artifactory Plugin version: 2.12.2-SNAPSHOT ( private -08/09/2017 15:15-eyalb) For pattern: spec-test/release_(\d+)-(\d+)-(\d+)-(\d+)-(\d+)-(\d+)-(\d+).tar.gz 3 artifacts were found. Deploying artifact: http: //localhost:8081/artifactory/tests/fpga/1.0.0-3/111--222--333.tgz Deploying artifact: http: //localhost:8081/artifactory/tests/fpga/1.0.0-3/1--2--3.tgz Deploying artifact: http: //localhost:8081/artifactory/tests/fpga/1.0.0-3/11--22--33.tgz Deploying build info to: http: //localhost:8081/artifactory/api/build Finished: SUCCESS  

          Josh Hasner added a comment -

          eyalbe, you seem to be using a different build than me? My build output is 2.12.2, while yours is 2.12.2-SNAPSHOT (private-08/09/2017 15:15-eyalb). Would this make a difference? When I updated the plugin, I updated it from the plugin manager UI. Is my issue fixed in this snapshot version you are using?

          Josh Hasner added a comment - eyalbe , you seem to be using a different build than me? My build output is 2.12.2, while yours is 2.12.2-SNAPSHOT (private-08/09/2017 15:15-eyalb). Would this make a difference? When I updated the plugin, I updated it from the plugin manager UI. Is my issue fixed in this snapshot version you are using?

          The plugin version I tested this with is almost similar to 2.12.2. I'll soon run the same test with 2.12.2 and let you know if I get different results.

          Eyal Ben Moshe added a comment - The plugin version I tested this with is almost similar to 2.12.2. I'll soon run the same test with 2.12.2 and let you know if I get different results.

          I've just tested the 2 File Specs with 2.12.2. All deployed as expected.

          Please copy my Specs. I see no reason why it shouldn't work for you as well.

          Eyal Ben Moshe added a comment - I've just tested the 2 File Specs with 2.12.2. All deployed as expected. Please copy my Specs. I see no reason why it shouldn't work for you as well.

          Josh Hasner added a comment -

          eyalbe, I think I may have found my issue. The path for the pattern had a double slash just before the file name. In version 2.9.1, this worked fine. Not sure why it matters now. Regardless, I should be passing a proper path. I fixed this, and now it seems to work. Apologies for making you run around, and thank you for your testing.

          Josh Hasner added a comment - eyalbe , I think I may have found my issue. The path for the pattern had a double slash just before the file name. In version 2.9.1, this worked fine. Not sure why it matters now. Regardless, I should be passing a proper path. I fixed this, and now it seems to work. Apologies for making you run around, and thank you for your testing.

          Great to hear the issue is resolved and I'm happy I could help.

          Eyal Ben Moshe added a comment - Great to hear the issue is resolved and I'm happy I could help.

            eyalbe Eyal Ben Moshe
            dg_skolberg Sascha Kolberg
            Votes:
            2 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated: