Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-45641

Artifactory plugin bug - latest package error

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      There is an option build in File Spec. I am trying to use it for download of latest artifact in our consumer project..

       

      {

        "files": [

          {

            "pattern" or "aql": "[Mandatory]",

            "target": "[Mandatory]",

            "props": "[Optional]",

            "recursive": "[Optional, Default: true]",

            "flat": "[Optional, Default: false]",

            "build": "[Optional]",

            "explode": "[Optional, Default: false]"

          }

        ]

      }

       

      We have found out that build option does not work correctly for “Multi Configuration” Jenkins projects – nothing is downloaded. We have dozens of artifacts in our Artifactory created by one multi configuration project (which has 112 combinations). Each group of 16 packages goes to the same artifactory folder (of course, with different name). We have a download file spec like this in our consumer project:

       

      {

          "files": [

              {

                  "target": "package/",

                  "pattern": "ourproject-snapshot-local/com/ourcompany/ourframework/${Module}/*${VISUAL_STUDIO_VERSION}_${Platform}${BuildType}.tar.gz",

                  "recursive": "false",

                  "flat": "true",

                  "props": "VisualStudioVersion=${VISUAL_STUDIO_VERSION};BuildType=${BuildType};Platform=${Platform};Module=${Module};ClearcaseView=${CLEARCASE_VIEW};SharedLib=${SharedLib}",

                  "build": "Our_Framework :: PF_build_project_windows"

              }

          ]

      }

       

      Please notice implicit LATEST “flag” in this configuration.

       

      This file spec downloads the artifact only in one scenario:

      • when a multi configuration project which is producing those 112 packages is limited by “Combination filter”, so only one Configuration is executed (and one package uploaded)
      • when we have same “Combination filter” in consumer project, file is downloaded.

       

      In all other scenarios (no combination filter), nothing is downloaded.

        Attachments

          Activity

          Hide
          eyalbe Eyal Ben Moshe added a comment -

          The "build" property filters out artifacts found by the specified "pattern" and the "recursive" values. It filter out artifacts, which their checksum is not included the build-info published to Artifactory from Jenkins.

          Do you see the checksums of these artifacts as part of the build-info JSON in Artifactory? You can see the build-info JSON by selecting the right hand tab in the Artifactory Build area.

          Show
          eyalbe Eyal Ben Moshe added a comment - The "build" property filters out artifacts found by the specified "pattern" and the "recursive" values. It filter out artifacts, which their checksum is not included the build-info published to Artifactory from Jenkins. Do you see the checksums of these artifacts as part of the build-info JSON in Artifactory? You can see the build-info JSON by selecting the right hand tab in the Artifactory Build area.
          Hide
          hustoles Roman Malík added a comment -

          Eyal Ben Moshe Yes we see checksums. Let me notice that when we limit the producer build to one configuration only, this mechanism works. Possibly it could be that algorithm firstly check latest build from given project and then it performs pattern check (which does not work for multiconfig project)?

          Show
          hustoles Roman Malík added a comment - Eyal Ben Moshe Yes we see checksums. Let me notice that when we limit the producer build to one configuration only, this mechanism works. Possibly it could be that algorithm firstly check latest build from given project and then it performs pattern check (which does not work for multiconfig project)?
          Hide
          eyalbe Eyal Ben Moshe added a comment -

          When you're not setting a build number, Jenkins tries to first fetch the latest build number from Artifactory, for the specified build name. Then, it builds an AQL query with the build name, build number and the other properties in the File Spec. and sends it to Artifactory to get the artifacts.

          It could be that the artifacts are not deployed as part of the latest build, but as part of one of the other builds in the matrix? You could check the build.name and build.number properties on these artifacts in Artifactory to see if this is the case.

          I guess you'd like to have all the artifacts, from all matrix builds deployed to Artifactory, but in case you don't, you could use the "Multi Configuration Upload Matcher"-->"Combination Matches" field in the Jenkins Multi-Configuration job UI, to specify only one matrix build, for which the artifacts and build-info are deployed, This way the latest build (which is the only build deployed) will include the artifacts.

           

          Show
          eyalbe Eyal Ben Moshe added a comment - When you're not setting a build number, Jenkins tries to first fetch the latest build number from Artifactory, for the specified build name. Then, it builds an AQL query with the build name, build number and the other properties in the File Spec. and sends it to Artifactory to get the artifacts. It could be that the artifacts are not deployed as part of the latest build, but as part of one of the other builds in the matrix? You could check the build.name and build.number properties on these artifacts in Artifactory to see if this is the case. I guess you'd like to have all the artifacts, from all matrix builds deployed to Artifactory, but in case you don't, you could use the "Multi Configuration Upload Matcher"-->"Combination Matches" field in the Jenkins Multi-Configuration job UI, to specify only one matrix build, for which the artifacts and build-info are deployed, This way the latest build (which is the only build deployed) will include the artifacts.  
          Hide
          hustoles Roman Malík added a comment - - edited

          Eyal Ben Moshe

          Thank you Eyal for your response. I just don't see what should I do differently. I will try to explain in detail what is my setup (excluding file spec listed in ticket description) :

           

          I.

          Producer Multi-Configuration project

          1. Has 112 configurations combinations.
          2. Each of this combination produces unique Artifact which is stored in Artifactory with different name. This name is linked to certain configuration.
          3. "Combination Matches" is manually set to have value "true" (I want to upload artifact from each configuration - it has different name so it shall be no problem)
          4. There is no other project producing the same Artifact

           

          II.

          Consumer Multi-Configuration project

          1. Has 112 configurations combinations (totally equal to Producer project)
          2. Each of this combination needs to download just one Artifact which is stored in Artifactory with specific name. The name of the artifact is linked to certain configuration
          3. I have only one of those configurations enabled for build ("Combination filter" is setup in that way)

                - that means just one SPECIFIC artifact shall be downloaded

           

           

          III. Broken scenarion:

          1. Producer project has no "Combination filter" defined.
          2. Producer project is executed. It uploads 112 different artifacts.
          3. Consumer project is executed (Remember the Consumer project is still limited by "Combination filter" to execute just one specific configuration)
          4. Consumer project does not find/download the artifact

           

          IV. Working scenario:

          All steps I done to make "Broken scenario" working

          1. Producer project has "Combination filter". It's value is exactly the same like the one defined in (II.3)
          2. Producer project is executed. It uploads 1 artifact.
          3. Consumer project is executed (Remember the Consumer project has not changed - it is still limited by "Combination filter" to execute just one specific configuration)
          4. Consumer project DOWNLOADS the artifact

           

          • That means that Consumer project has not changed at all.
          • Producer project has changed in just one field - Combination Filter.
          • No one else is uploading the files with the same name - I am the only user of that repository.
          • I am not uploading those files to artifactory manually or by different job.
          • Each artifact (ever) uploaded to this repository has the same build.name
          • build.number is not the same for artifacts uploaded by one Producer project run (as it is not only a number). Example for two different configurations, created with one build:

              1) build.number = "730 :: BuildType=Release,Module=maal,Platform=x64,SharedLib=1,VISUAL_STUDIO_VERSION=win_2013,label=PF_Win_Slave"

              2) build.number = "730 :: BuildType=Debug,Module=maal,Platform=x86,SharedLib=1,VISUAL_STUDIO_VERSION=win_2013,label=PF_Win_Slave"

           

           

          I think it is a bug. If I am doing something wrong, please let me know what should I change in my setup. I am really appreciating your help.

           

           

          Show
          hustoles Roman Malík added a comment - - edited Eyal Ben Moshe Thank you Eyal for your response. I just don't see what should I do differently. I will try to explain in detail what is my setup (excluding file spec listed in ticket description) :   I. Producer Multi-Configuration project Has 112 configurations combinations. Each of this combination produces unique Artifact which is stored in Artifactory with different name. This name is linked to certain configuration. "Combination Matches" is manually set to have value "true" (I want to upload artifact from each configuration - it has different name so it shall be no problem) There is no other project producing the same Artifact   II. Consumer Multi-Configuration project Has 112 configurations combinations (totally equal to Producer project) Each of this combination needs to download just one Artifact which is stored in Artifactory with specific name. The name of the artifact is linked to certain configuration I have only one of those configurations enabled for build ("Combination filter" is setup in that way)       - that means just one SPECIFIC artifact shall be downloaded     III. Broken scenarion: Producer project has no "Combination filter" defined. Producer project is executed. It uploads 112 different artifacts. Consumer project is executed (Remember the Consumer project is still limited by "Combination filter" to execute just one specific configuration) Consumer project does not find/download the artifact   IV. Working scenario: All steps I done to make "Broken scenario" working Producer project has "Combination filter". It's value is exactly the same like the one defined in (II.3) Producer project is executed. It uploads 1 artifact. Consumer project is executed (Remember the Consumer project has not changed - it is still limited by "Combination filter" to execute just one specific configuration) Consumer project DOWNLOADS the artifact   That means that Consumer project has not changed at all. Producer project has changed in just one field - Combination Filter. No one else is uploading the files with the same name - I am the only user of that repository. I am not uploading those files to artifactory manually or by different job. Each artifact (ever) uploaded to this repository has the same build.name build.number is not the same for artifacts uploaded by one Producer project run (as it is not only a number). Example for two different configurations, created with one build:     1) build.number = "730 :: BuildType=Release,Module=maal,Platform=x64,SharedLib=1,VISUAL_STUDIO_VERSION=win_2013,label=PF_Win_Slave"     2) build.number = "730 :: BuildType=Debug,Module=maal,Platform=x86,SharedLib=1,VISUAL_STUDIO_VERSION=win_2013,label=PF_Win_Slave"     I think it is a bug. If I am doing something wrong, please let me know what should I change in my setup. I am really appreciating your help.    
          Hide
          rhadesan Rhades Progman added a comment - - edited

          Eyal Ben Moshe:

          Any news on this topic? It is quite critical bug since it is not possible to use Artifactory plugin at all with Multi Configuration project. We need to test packages uploaded to Artifactory by one job with another job in an automated manner and it is not possible since we cannot choose/download LATEST package.

          Show
          rhadesan Rhades Progman added a comment - - edited Eyal Ben Moshe : Any news on this topic? It is quite critical bug since it is not possible to use Artifactory plugin at all with Multi Configuration project. We need to test packages uploaded to Artifactory by one job with another job in an automated manner and it is not possible since we cannot choose/download LATEST package.

            People

            Assignee:
            eyalbe Eyal Ben Moshe
            Reporter:
            hustoles Roman Malík
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated: