Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-58716

Failure to check out specified tag for incremental version

      I tried to run the PCT against an incremental release of a plugin, and it failed:

      Created plugin checkout dir : .../pct-work/jsch
      Checking out from SCM connection URL : scm:git:git://github.com/jenkinsci/jsch-plugin.git (jsch-0.1.55.1-rc41.4943eb07c811)
      [INFO] Executing: /bin/sh -c cd '.../pct-work' && 'git' 'clone' 'git://github.com/jenkinsci/jsch-plugin.git' '.../pct-work/jsch'
      [INFO] Working directory: .../pct-work
      [INFO] Executing: /bin/sh -c cd '.../pct-work/jsch' && 'git' 'fetch' 'git://github.com/jenkinsci/jsch-plugin.git'
      [INFO] Working directory: .../pct-work/jsch
      [INFO] Executing: /bin/sh -c cd '.../pct-work/jsch' && 'git' 'checkout' 'jsch-0.1.55.1-rc41.4943eb07c811'
      [INFO] Working directory: .../pct-work/jsch
      Error : The git-checkout command failed. || Cloning into '.../pct-work/jsch'...
      From git://github.com/jenkinsci/jsch-plugin
       * branch            HEAD       -> FETCH_HEAD
      error: pathspec 'jsch-0.1.55.1-rc41.4943eb07c811' did not match any file(s) known to git
      

      That is because this code improperly assumes that ${project.artifactId}-${project.version} is going to be a valid tag. Maven makes no such guarantee. Rather, this code should be looking for a /project/scm/tag and honoring it if present. In this case it would have found

          <tag>4943eb07c81131909f1d3b16bf18dec8a8b91a1b</tag>
      

      which is, in fact, the correct hash to check out.

          [JENKINS-58716] Failure to check out specified tag for incremental version

          Jesse Glick created issue -
          Jesse Glick made changes -
          Epic Link New: JENKINS-50686 [ 189770 ]
          Jesse Glick made changes -
          Link New: This issue is blocking JENKINS-47498 [ JENKINS-47498 ]
          Basil Crow made changes -
          Remote Link New: This issue links to "PR #181 (Web Link)" [ 23321 ]

          Basil Crow added a comment -

          Rather, this code should be looking for a /project/scm/tag and honoring it if present

          As explained in jenkinsci/plugin-compat-tester#181 (comment), this is necessary but not sufficient in order to solve the problem.

          Basil Crow added a comment - Rather, this code should be looking for a /project/scm/tag and honoring it if present As explained in jenkinsci/plugin-compat-tester#181 (comment) , this is necessary but not sufficient in order to solve the problem.

          Basil Crow added a comment -

          this is necessary but not sufficient in order to solve the problem

          Here's a sketch of a possible solution. Introduce a new property for the GitHub organization, say scmOrganization. Then use it from <connection>, <developerConnection>, and <url>:

          <properties>
              <scmOrganization>jenkinsci</scmOrganization>
          </properties>
          
          <scm>
              <connection>scm:git:git://github.com/${scmOrganization}/${project.artifactId}-plugin.git</connection>
              <developerConnection>scm:git:git@github.com:${scmOrganization}/${project.artifactId}-plugin.git</developerConnection>
              <url>https://github.com/${scmOrganization}/${project.artifactId}-plugin</url>
              [...]
          </scm>
          

          (To implement this, the archetype would need to be updated and IncrementalifyMojo#update would need to evolve to be able to update an existing POM with this change.)

          With this in place, the git-changelist-maven-extension in incrementals-tools could be updated to set this property for incrementals builds. In particular, we should be able to look up the Git remote in Main#afterSessionStart. From there, we can parse out the organization and set scmOrganization to the appropriate value for incrementals builds.

          Basil Crow added a comment - this is necessary but not sufficient in order to solve the problem Here's a sketch of a possible solution. Introduce a new property for the GitHub organization, say scmOrganization . Then use it from <connection> , <developerConnection> , and <url> : <properties> <scmOrganization>jenkinsci</scmOrganization> </properties> <scm> <connection>scm:git:git://github.com/${scmOrganization}/${project.artifactId}-plugin.git</connection> <developerConnection>scm:git:git@github.com:${scmOrganization}/${project.artifactId}-plugin.git</developerConnection> <url>https://github.com/${scmOrganization}/${project.artifactId}-plugin</url> [...] </scm> (To implement this, the archetype would need to be updated and IncrementalifyMojo#update would need to evolve to be able to update an existing POM with this change.) With this in place, the git-changelist-maven-extension in incrementals-tools could be updated to set this property for incrementals builds. In particular, we should be able to look up the Git remote in Main#afterSessionStart . From there, we can parse out the organization and set scmOrganization to the appropriate value for incrementals builds.

          Jesse Glick added a comment -

          we should be able to look up the Git remote

          I am not so sure. git remote -v in a forked PR build shows https://github.com/jenkinsci/XXX-plugin.git. This is because Jenkins first clones the origin repo at master, then merges in the PR head.

          As to the property, I think you want to introduce a property for the entire repository location, so

          <properties>
              <gitHubRepo>jenkinsci/${project.artifactId}-plugin</gitHubRepo>
          </properties>
          <scm>
              <connection>scm:git:git://github.com/${gitHubRepo}.git</connection>
              <developerConnection>scm:git:git@github.com:${gitHubRepo}.git</developerConnection>
              <url>https://github.com/${gitHubRepo}</url>
              <tag>${scmTag}</tag>
          </scm>
          

          since a developer’s fork may not have the same name as the origin repository. In particular, if the developer already has some unrelated “sources” repo or fork named foo-plugin, then jenkinsci/foo-plugin will be forked to jqhacker/foo-plugin-1 or similar.

          You also would likely want to

          • Make incrementals:incrementalify do this transformation.
          • Adjust jenkinsci/archetypes to encourage new plugins to do this.

          Jesse Glick added a comment - we should be able to look up the Git remote I am not so sure. git remote -v in a forked PR build shows https: //github.com/jenkinsci/XXX-plugin.git . This is because Jenkins first clones the origin repo at master , then merges in the PR head. As to the property, I think you want to introduce a property for the entire repository location, so <properties> <gitHubRepo> jenkinsci/${project.artifactId}-plugin </gitHubRepo> </properties> <scm> <connection> scm:git:git://github.com/${gitHubRepo}.git </connection> <developerConnection> scm:git:git@github.com:${gitHubRepo}.git </developerConnection> <url> https://github.com/${gitHubRepo} </url> <tag> ${scmTag} </tag> </scm> since a developer’s fork may not have the same name as the origin repository. In particular, if the developer already has some unrelated “sources” repo or fork named foo-plugin , then jenkinsci/foo-plugin will be forked to jqhacker/foo-plugin-1 or similar. You also would likely want to Make incrementals:incrementalify do this transformation. Adjust jenkinsci/archetypes to encourage new plugins to do this.
          Jesse Glick made changes -
          Component/s New: incrementals-tools [ 23725 ]

          Basil Crow added a comment -

          I am not so sure. git remote -v in a forked PR build shows https://github.com/jenkinsci/XXX-plugin.git. This is because Jenkins first clones the origin repo at master, then merges in the PR head.

          Ah, that's unfortunate. However, the information we need is exposed by the CHANGE_FORK environment variable, although today it only consists of the username and not necessarily the repository name. However, JENKINS-58450 seeks to rectify that by setting CHANGE_FORK to $user/$repo when the repository name of the fork differs from the upstream repository name.

          Given the above, I think we could check CHANGE_FORK from git-changelist-maven-extension. If it's just a username, we can update githubRepo to the username from CHANGE_FORK and the repository name from the original value of githubRepo. If it's a username and a repository (post-JENKINS-58450), we can update gitHubRepo to the value of CHANGE_FORK verbatim.

          What do you think?

          Basil Crow added a comment - I am not so sure. git remote -v in a forked PR build shows https://github.com/jenkinsci/XXX-plugin.git . This is because Jenkins first clones the origin repo at master , then merges in the PR head. Ah, that's unfortunate. However, the information we need is exposed by the CHANGE_FORK environment variable, although today it only consists of the username and not necessarily the repository name. However, JENKINS-58450 seeks to rectify that by setting CHANGE_FORK to $user/$repo when the repository name of the fork differs from the upstream repository name. Given the above, I think we could check CHANGE_FORK from git-changelist-maven-extension . If it's just a username, we can update githubRepo to the username from CHANGE_FORK and the repository name from the original value of githubRepo . If it's a username and a repository (post- JENKINS-58450 ), we can update gitHubRepo to the value of CHANGE_FORK verbatim. What do you think?
          Basil Crow made changes -
          Link New: This issue depends on JENKINS-58450 [ JENKINS-58450 ]

            jglick Jesse Glick
            jglick Jesse Glick
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: