Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-40429

Copy artifacts from multi-branch pipeline project with project name as a build parameter fails with "Unable to find project for artifact copy"

    XMLWordPrintable

Details

    • Bug
    • Status: Open (View Workflow)
    • Blocker
    • Resolution: Unresolved
    • copyartifact-plugin
    • None
    • Master & slave: Debian Jessie 64 bit, Oracle JRE, installed from deb repo
      Jenkins: 2.19.3
      CopyArtifactPlugin: 1.38.1

    Description

      We have a multi-branch pipeline project that produces artifacts, we also have a freeform project that is configured with build parameters to copy artifacts from the pipeline job based on the parameters supplied.

      When I set the project name parameter to something like

      myproject/${BRANCH_NAME} 

      the job fails with

      Unable to find project for artifact copy: myproject/mybranch

      If it set the project name parameter to

      myproject/mybranch

      it works fine.

      We aren't using matrix permissions, logged in users can do anything and Allow anonymous read access is checked. I tried setting the project property CopyArtifactPermissionProperty in the JenkinsFile to

      properties([[$class: 'CopyArtifactPermissionProperty', projectNames: '*']])
      

      but this made no difference.

      Attachments

        Activity

          apikas Anders Pikas added a comment -

          Thanks, haggai_e! I solved this in 30 minutes thanks to you, not a couple of days. I had the same problem as you: the pipeline I was copying from started with "core/" and the pipeline I was running had a name ending in "/core".

          I always specify absolute job/project/pipeline paths as absolute, BUT Jenkins itself gives ${env.JOB_NAME} as a relative path for some reason (maybe backward compatibility?).

          env.JOB_NAME.replaceFirst('^(?!/)', '/')
          
          apikas Anders Pikas added a comment - Thanks, haggai_e ! I solved this in 30 minutes thanks to you, not a couple of days. I had the same problem as you: the pipeline I was copying from started with "core/" and the pipeline I was running had a name ending in "/core". I always specify absolute job/project/pipeline paths as absolute, BUT Jenkins itself gives ${env.JOB_NAME} as a relative path for some reason (maybe backward compatibility?). env.JOB_NAME.replaceFirst( '^(?!/)' , '/' )
          maschall Mark Schall added a comment -

          I was running into the same issues, until I dove into the code of the plugin.

          If you have a multi branch pipeline job, say project/branch and you add the 

          copyArtifactPermission('downstream');

          You need to make it

          copyArtifactPermission('/downstream');

          because the downstream project is not in the "folder" of the project, it's one more level down.

          Then passing in `project/branch` as a parameter should work.

          The root cause, is that permissions are not checked at run time, if you hardcode the project name, but if you set as a variable, they are checked and names don't match what is expected.

          maschall Mark Schall added a comment - I was running into the same issues, until I dove into the code of the plugin. If you have a multi branch pipeline job, say project/branch and you add the  copyArtifactPermission( 'downstream' ); You need to make it copyArtifactPermission( '/downstream' ); because the downstream project is not in the "folder" of the project, it's one more level down. Then passing in `project/branch` as a parameter should work. The root cause, is that permissions are not checked at run time, if you hardcode the project name, but if you set as a variable, they are checked and names don't match what is expected.
          mrw Marc Wäckerlin added a comment - - edited

          First of all, it seems, that I am running into the same problem. But for me, it looks strange that this error has been reported such a long time ago (and why is it not fixed in these years?), but I am sure that it used to work short time ago. I cannot exactly say, how long it used to work for me, but I had successful builds. I ran into it now that I am moving my jenkins to a new server and re-running all builds. I am wondering why the priority should be minor, it's a blocker, the plugin is no more usable.

          My situation:

          Jenkins ver. 2.225
          Tested Copy Artifact 1.43.1 and 1.43

          Global Jenkins environment variable:

          • name: MINGW_ARCH
          • value: x86_64-w64-mingw32 i686-w64-mingw32

          ProjectA:

          • matrix build:
            • name: mingw
            • variable: MINGW_ARCH
          • Artifacts: */usr and }}{{.zip

          ProjectB (built after ProjectA):

          • matrix build:
            • name: mingw
            • variable: MINGW_ARCH
          • Copy Artifacts:
            • project name: ProjectA/mingw=$mingw
          • Artifacts: */usr and }}{{.zip

          This results in:

          ERROR: Unable to find project for artifact copy: ProjectA/mingw=x86_64-w64-mingw32

          Then, if I replace the project name to ProjectA/mingw=x86_64-w64-mingw32, that means hard coded, no variable, then it works.

          As you see from the error log, the variable substitution is 100% correct! But still there is somehow a failure with regard to the variable.

          The projects configurations (ProjectA is mrw-cxx.win and ProjectB is libxml-cxx.win):

          ProjectA:

          <?xml version="1.1" encoding="UTF-8" standalone="no"?><matrix-project plugin="matrix-project@1.14">
          <actions/>
          <description/>
          <keepDependencies>false</keepDependencies>
          <properties>

          </properties>
          <scm class="hudson.plugins.git.GitSCM" plugin="git@4.2.2">
          <configVersion>2</configVersion>
          <userRemoteConfigs>
          <hudson.plugins.git.UserRemoteConfig>
          <url>https://mrw.dev/libraries/mrw-cxx</url>
          </hudson.plugins.git.UserRemoteConfig>
          </userRemoteConfigs>
          <branches>
          <hudson.plugins.git.BranchSpec>
          <name>*/master</name>
          </hudson.plugins.git.BranchSpec>
          </branches>
          <doGenerateSubmoduleConfigurations>false</doGenerateSubmoduleConfigurations>
          <submoduleCfg class="list"/>
          <extensions/>
          </scm>
          <quietPeriod>5</quietPeriod>
          <scmCheckoutRetryCount>10</scmCheckoutRetryCount>
          <canRoam>true</canRoam>
          <disabled>false</disabled>
          <blockBuildWhenDownstreamBuilding>true</blockBuildWhenDownstreamBuilding>
          <blockBuildWhenUpstreamBuilding>true</blockBuildWhenUpstreamBuilding>
          <triggers>
          <hudson.triggers.TimerTrigger>
          <spec>@monthly</spec>
          </hudson.triggers.TimerTrigger>
          <hudson.triggers.SCMTrigger>
          <spec>H/5 * * * *</spec>
          <ignorePostCommitHooks>false</ignorePostCommitHooks>
          </hudson.triggers.SCMTrigger>
          </triggers>
          <concurrentBuild>false</concurrentBuild>
          <axes>
          <ca.silvermaplesolutions.jenkins.plugins.daxis.DynamicAxis plugin="dynamic-axis@1.0.3">
          <name>mingw</name>
          <values>
          <string>MINGW_ARCHS</string>
          </values>
          <varName>MINGW_ARCHS</varName>
          <axisValues>
          <string>default</string>
          </axisValues>
          </ca.silvermaplesolutions.jenkins.plugins.daxis.DynamicAxis>
          </axes>
          <builders>
          <hudson.tasks.Shell>
          <command>docker pull mwaeckerlin/mingw
          docker run --rm -v $(pwd):/workdir -u $(id -u) -e MINGW=$mingw -e BUILD_NUMBER=$BUILD_NUMBER mwaeckerlin/mingw /build.sh -z

          FILE=mrw-c++-*_${mingw%-w64-mingw32}.zip
          TARGET=~/repository/windows/${mingw%-w64-mingw32}/${JOB_NAME%.win/*}
          test -d ${TARGET} || mkdir -p ${TARGET}
          cp ${FILE} ${TARGET}/
          </command>
          </hudson.tasks.Shell>
          </builders>
          <publishers>
          <hudson.tasks.ArtifactArchiver>
          <artifacts>usr/**, *.zip</artifacts>
          <allowEmptyArchive>false</allowEmptyArchive>
          <onlyIfSuccessful>false</onlyIfSuccessful>
          <fingerprint>false</fingerprint>
          <defaultExcludes>true</defaultExcludes>
          <caseSensitive>true</caseSensitive>
          </hudson.tasks.ArtifactArchiver>
          </publishers>
          <buildWrappers>
          <hudson.plugins.ws__cleanup.PreBuildCleanup plugin="ws-cleanup@0.38">
          <deleteDirs>false</deleteDirs>
          <cleanupParameter/>
          <externalDelete/>
          <disableDeferredWipeout>false</disableDeferredWipeout>
          </hudson.plugins.ws__cleanup.PreBuildCleanup>
          </buildWrappers>
          <executionStrategy class="hudson.matrix.DefaultMatrixExecutionStrategyImpl">
          <runSequentially>false</runSequentially>
          </executionStrategy>
          </matrix-project>

          ProjectB:

          <?xml version="1.1" encoding="UTF-8" standalone="no"?><matrix-project plugin="matrix-project@1.14">
          <actions/>
          <description/>
          <keepDependencies>false</keepDependencies>
          <properties>

          </properties>
          <scm class="hudson.plugins.git.GitSCM" plugin="git@4.2.2">
          <configVersion>2</configVersion>
          <userRemoteConfigs>
          <hudson.plugins.git.UserRemoteConfig>
          <url>https://mrw.dev/libraries/libxml-cxx</url>
          </hudson.plugins.git.UserRemoteConfig>
          </userRemoteConfigs>
          <branches>
          <hudson.plugins.git.BranchSpec>
          <name>*/master</name>
          </hudson.plugins.git.BranchSpec>
          </branches>
          <doGenerateSubmoduleConfigurations>false</doGenerateSubmoduleConfigurations>
          <submoduleCfg class="list"/>
          <extensions>
          <hudson.plugins.git.extensions.impl.WipeWorkspace/>
          </extensions>
          </scm>
          <quietPeriod>5</quietPeriod>
          <scmCheckoutRetryCount>10</scmCheckoutRetryCount>
          <canRoam>true</canRoam>
          <disabled>false</disabled>
          <blockBuildWhenDownstreamBuilding>true</blockBuildWhenDownstreamBuilding>
          <blockBuildWhenUpstreamBuilding>true</blockBuildWhenUpstreamBuilding>
          <triggers>
          <jenkins.triggers.ReverseBuildTrigger>
          <spec/>
          <upstreamProjects>mrw-cxx.win</upstreamProjects>
          <threshold>
          <name>SUCCESS</name>
          <ordinal>0</ordinal>
          <color>BLUE</color>
          <completeBuild>true</completeBuild>
          </threshold>
          </jenkins.triggers.ReverseBuildTrigger>
          <hudson.triggers.TimerTrigger>
          <spec>@monthly</spec>
          </hudson.triggers.TimerTrigger>
          <hudson.triggers.SCMTrigger>
          <spec>H/5 * * * *</spec>
          <ignorePostCommitHooks>false</ignorePostCommitHooks>
          </hudson.triggers.SCMTrigger>
          </triggers>
          <concurrentBuild>false</concurrentBuild>
          <axes>
          <ca.silvermaplesolutions.jenkins.plugins.daxis.DynamicAxis plugin="dynamic-axis@1.0.3">
          <name>mingw</name>
          <values>
          <string>MINGW_ARCHS</string>
          </values>
          <varName>MINGW_ARCHS</varName>
          <axisValues>
          <string>default</string>
          </axisValues>
          </ca.silvermaplesolutions.jenkins.plugins.daxis.DynamicAxis>
          </axes>
          <builders>
          <hudson.plugins.copyartifact.CopyArtifact plugin="copyartifact@1.43">
          <project>mrw-cxx.win/mingw=$mingw</project>
          <filter/>
          <target/>
          <excludes/>
          <selector class="hudson.plugins.copyartifact.StatusBuildSelector"/>
          <doNotFingerprintArtifacts>false</doNotFingerprintArtifacts>
          </hudson.plugins.copyartifact.CopyArtifact>
          <hudson.tasks.Shell>
          <command>docker pull mwaeckerlin/mingw
          docker run --rm -v $(pwd):/workdir -u $(id -u) -e MINGW=$mingw -e BUILD_NUMBER=$BUILD_NUMBER mwaeckerlin/mingw /build.sh -z

          FILE=${JOB_NAME%.win/}-_${mingw%-w64-mingw32}.zip
          TARGET=~/repository/windows/${mingw%-w64-mingw32}/${JOB_NAME%.win/*}
          test -d ${TARGET} || mkdir -p ${TARGET}
          cp ${FILE} ${TARGET}/
          </command>
          </hudson.tasks.Shell>
          </builders>
          <publishers>
          <hudson.tasks.ArtifactArchiver>
          <artifacts>usr/**, *.zip</artifacts>
          <allowEmptyArchive>false</allowEmptyArchive>
          <onlyIfSuccessful>false</onlyIfSuccessful>
          <fingerprint>false</fingerprint>
          <defaultExcludes>true</defaultExcludes>
          <caseSensitive>true</caseSensitive>
          </hudson.tasks.ArtifactArchiver>
          </publishers>
          <buildWrappers>
          <hudson.plugins.ws__cleanup.PreBuildCleanup plugin="ws-cleanup@0.38">
          <deleteDirs>false</deleteDirs>
          <cleanupParameter/>
          <externalDelete/>
          <disableDeferredWipeout>false</disableDeferredWipeout>
          </hudson.plugins.ws__cleanup.PreBuildCleanup>
          </buildWrappers>
          <executionStrategy class="hudson.matrix.DefaultMatrixExecutionStrategyImpl">
          <runSequentially>false</runSequentially>
          </executionStrategy>
          </matrix-project>

          mrw Marc Wäckerlin added a comment - - edited First of all, it seems, that I am running into the same problem. But for me, it looks strange that this error has been reported such a long time ago (and why is it not fixed in these years?), but I am sure that it used to work short time ago. I cannot exactly say, how long it used to work for me, but I had successful builds. I ran into it now that I am moving my jenkins to a new server and re-running all builds. I am wondering why the priority should be minor, it's a blocker, the plugin is no more usable. My situation: Jenkins ver. 2.225 Tested Copy Artifact 1.43.1 and 1.43 Global Jenkins environment variable: name: MINGW_ARCH value: x86_64-w64-mingw32 i686-w64-mingw32 ProjectA: matrix build: name: mingw variable: MINGW_ARCH Artifacts: * /usr and }} {{.zip ProjectB (built after ProjectA): matrix build: name: mingw variable: MINGW_ARCH Copy Artifacts: project name: ProjectA/mingw=$mingw Artifacts: * /usr and }} {{.zip This results in: ERROR: Unable to find project for artifact copy: ProjectA/mingw=x86_64-w64-mingw32 Then, if I replace the project name to ProjectA/mingw=x86_64-w64-mingw32 , that means hard coded, no variable, then it works. As you see from the error log, the variable substitution is 100% correct! But still there is somehow a failure with regard to the variable. The projects configurations ( ProjectA is mrw-cxx.win and ProjectB is libxml-cxx.win ): ProjectA: <?xml version="1.1" encoding="UTF-8" standalone="no"?><matrix-project plugin="matrix-project@1.14"> <actions/> <description/> <keepDependencies>false</keepDependencies> <properties> </properties> <scm class="hudson.plugins.git.GitSCM" plugin="git@4.2.2"> <configVersion>2</configVersion> <userRemoteConfigs> <hudson.plugins.git.UserRemoteConfig> <url> https://mrw.dev/libraries/mrw-cxx </url> </hudson.plugins.git.UserRemoteConfig> </userRemoteConfigs> <branches> <hudson.plugins.git.BranchSpec> <name>*/master</name> </hudson.plugins.git.BranchSpec> </branches> <doGenerateSubmoduleConfigurations>false</doGenerateSubmoduleConfigurations> <submoduleCfg class="list"/> <extensions/> </scm> <quietPeriod>5</quietPeriod> <scmCheckoutRetryCount>10</scmCheckoutRetryCount> <canRoam>true</canRoam> <disabled>false</disabled> <blockBuildWhenDownstreamBuilding>true</blockBuildWhenDownstreamBuilding> <blockBuildWhenUpstreamBuilding>true</blockBuildWhenUpstreamBuilding> <triggers> <hudson.triggers.TimerTrigger> <spec>@monthly</spec> </hudson.triggers.TimerTrigger> <hudson.triggers.SCMTrigger> <spec>H/5 * * * *</spec> <ignorePostCommitHooks>false</ignorePostCommitHooks> </hudson.triggers.SCMTrigger> </triggers> <concurrentBuild>false</concurrentBuild> <axes> <ca.silvermaplesolutions.jenkins.plugins.daxis.DynamicAxis plugin="dynamic-axis@1.0.3"> <name>mingw</name> <values> <string>MINGW_ARCHS</string> </values> <varName>MINGW_ARCHS</varName> <axisValues> <string>default</string> </axisValues> </ca.silvermaplesolutions.jenkins.plugins.daxis.DynamicAxis> </axes> <builders> <hudson.tasks.Shell> <command>docker pull mwaeckerlin/mingw docker run --rm -v $(pwd):/workdir -u $(id -u) -e MINGW=$mingw -e BUILD_NUMBER=$BUILD_NUMBER mwaeckerlin/mingw /build.sh -z FILE=mrw-c++-*_${mingw%-w64-mingw32}.zip TARGET=~/repository/windows/${mingw%-w64-mingw32}/${JOB_NAME%.win/*} test -d ${TARGET} || mkdir -p ${TARGET} cp ${FILE} ${TARGET}/ </command> </hudson.tasks.Shell> </builders> <publishers> <hudson.tasks.ArtifactArchiver> <artifacts>usr/**, *.zip</artifacts> <allowEmptyArchive>false</allowEmptyArchive> <onlyIfSuccessful>false</onlyIfSuccessful> <fingerprint>false</fingerprint> <defaultExcludes>true</defaultExcludes> <caseSensitive>true</caseSensitive> </hudson.tasks.ArtifactArchiver> </publishers> <buildWrappers> <hudson.plugins.ws__cleanup.PreBuildCleanup plugin="ws-cleanup@0.38"> <deleteDirs>false</deleteDirs> <cleanupParameter/> <externalDelete/> <disableDeferredWipeout>false</disableDeferredWipeout> </hudson.plugins.ws__cleanup.PreBuildCleanup> </buildWrappers> <executionStrategy class="hudson.matrix.DefaultMatrixExecutionStrategyImpl"> <runSequentially>false</runSequentially> </executionStrategy> </matrix-project> ProjectB: <?xml version="1.1" encoding="UTF-8" standalone="no"?><matrix-project plugin="matrix-project@1.14"> <actions/> <description/> <keepDependencies>false</keepDependencies> <properties> </properties> <scm class="hudson.plugins.git.GitSCM" plugin="git@4.2.2"> <configVersion>2</configVersion> <userRemoteConfigs> <hudson.plugins.git.UserRemoteConfig> <url> https://mrw.dev/libraries/libxml-cxx </url> </hudson.plugins.git.UserRemoteConfig> </userRemoteConfigs> <branches> <hudson.plugins.git.BranchSpec> <name>*/master</name> </hudson.plugins.git.BranchSpec> </branches> <doGenerateSubmoduleConfigurations>false</doGenerateSubmoduleConfigurations> <submoduleCfg class="list"/> <extensions> <hudson.plugins.git.extensions.impl.WipeWorkspace/> </extensions> </scm> <quietPeriod>5</quietPeriod> <scmCheckoutRetryCount>10</scmCheckoutRetryCount> <canRoam>true</canRoam> <disabled>false</disabled> <blockBuildWhenDownstreamBuilding>true</blockBuildWhenDownstreamBuilding> <blockBuildWhenUpstreamBuilding>true</blockBuildWhenUpstreamBuilding> <triggers> <jenkins.triggers.ReverseBuildTrigger> <spec/> <upstreamProjects>mrw-cxx.win</upstreamProjects> <threshold> <name>SUCCESS</name> <ordinal>0</ordinal> <color>BLUE</color> <completeBuild>true</completeBuild> </threshold> </jenkins.triggers.ReverseBuildTrigger> <hudson.triggers.TimerTrigger> <spec>@monthly</spec> </hudson.triggers.TimerTrigger> <hudson.triggers.SCMTrigger> <spec>H/5 * * * *</spec> <ignorePostCommitHooks>false</ignorePostCommitHooks> </hudson.triggers.SCMTrigger> </triggers> <concurrentBuild>false</concurrentBuild> <axes> <ca.silvermaplesolutions.jenkins.plugins.daxis.DynamicAxis plugin="dynamic-axis@1.0.3"> <name>mingw</name> <values> <string>MINGW_ARCHS</string> </values> <varName>MINGW_ARCHS</varName> <axisValues> <string>default</string> </axisValues> </ca.silvermaplesolutions.jenkins.plugins.daxis.DynamicAxis> </axes> <builders> <hudson.plugins.copyartifact.CopyArtifact plugin="copyartifact@1.43"> <project>mrw-cxx.win/mingw=$mingw</project> <filter/> <target/> <excludes/> <selector class="hudson.plugins.copyartifact.StatusBuildSelector"/> <doNotFingerprintArtifacts>false</doNotFingerprintArtifacts> </hudson.plugins.copyartifact.CopyArtifact> <hudson.tasks.Shell> <command>docker pull mwaeckerlin/mingw docker run --rm -v $(pwd):/workdir -u $(id -u) -e MINGW=$mingw -e BUILD_NUMBER=$BUILD_NUMBER mwaeckerlin/mingw /build.sh -z FILE=${JOB_NAME%.win/ }- _${mingw%-w64-mingw32}.zip TARGET=~/repository/windows/${mingw%-w64-mingw32}/${JOB_NAME%.win/*} test -d ${TARGET} || mkdir -p ${TARGET} cp ${FILE} ${TARGET}/ </command> </hudson.tasks.Shell> </builders> <publishers> <hudson.tasks.ArtifactArchiver> <artifacts>usr/**, *.zip</artifacts> <allowEmptyArchive>false</allowEmptyArchive> <onlyIfSuccessful>false</onlyIfSuccessful> <fingerprint>false</fingerprint> <defaultExcludes>true</defaultExcludes> <caseSensitive>true</caseSensitive> </hudson.tasks.ArtifactArchiver> </publishers> <buildWrappers> <hudson.plugins.ws__cleanup.PreBuildCleanup plugin="ws-cleanup@0.38"> <deleteDirs>false</deleteDirs> <cleanupParameter/> <externalDelete/> <disableDeferredWipeout>false</disableDeferredWipeout> </hudson.plugins.ws__cleanup.PreBuildCleanup> </buildWrappers> <executionStrategy class="hudson.matrix.DefaultMatrixExecutionStrategyImpl"> <runSequentially>false</runSequentially> </executionStrategy> </matrix-project>

          My personal Workaround:

          I removed the artifact copy plugin and replaced the copy artifact build step by a freeform shell script as follows, which does the same as this plugin, just set the name of the source project in FROM:

          FROM=mrw-cxx.win

          STABLE=$(sed -n 's/^lastSuccessfulBuild //p' ~/jobs/${FROM}/builds/permalinks)
          SOURCE=~/jobs/${FROM}/configurations/axis-mingw/${mingw}/builds/${STABLE}/archive/usr

          cp -a ${SOURCE} .

          mrw Marc Wäckerlin added a comment - My personal Workaround: I removed the artifact copy plugin and replaced the copy artifact build step by a freeform shell script as follows, which does the same as this plugin, just set the name of the source project in FROM : FROM=mrw-cxx.win STABLE=$(sed -n 's/^lastSuccessfulBuild //p' ~/jobs/${FROM}/builds/permalinks) SOURCE=~/jobs/${FROM}/configurations/axis-mingw/${mingw}/builds/${STABLE}/archive/usr cp -a ${SOURCE} .
          harrygg Harry G. added a comment - - edited

          I have the same issue here.
          I think it is also the same like JENKINS-14999, which I believe was never fixed.

          Definitely CopyArtifacts behaves differently with a parameterized job name and with a hardcoded job name.
          This should not be the case.

          IMHO a parameterized job does not use the migration mode and needs an explicit copy artifact permission set although migration mode is turned on an no permission should be needed.

          harrygg Harry G. added a comment - - edited I have the same issue here. I think it is also the same like JENKINS-14999 , which I believe was never fixed. Definitely CopyArtifacts behaves differently with a parameterized job name and with a hardcoded job name. This should not be the case. IMHO a parameterized job does not use the migration mode and needs an explicit copy artifact permission set although migration mode is turned on an no permission should be needed.

          People

            Unassigned Unassigned
            matty666 Matt Evans
            Votes:
            11 Vote for this issue
            Watchers:
            22 Start watching this issue

            Dates

              Created:
              Updated: