Wrong component.

Works if you have workflow-job 2.7+.

How is this bug resolved? 

I'm using Jenkins 2.73.2, pipeline job and it still doesn't work.

Is there a workaround for this?

The workaround that worked was to uncheck  the 'lightweight checkout'

See comment here:

https://issues.jenkins-ci.org/browse/JENKINS-28447?focusedCommentId=308351&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-308351

And also JENKINS-48431

I am still seeing this as well. 

I am also seeing this with Jenkins version 2.176.1 (from the Debian package repo provided by the Jenkins project).

We're seeing this same behavior today running Pipeline: Job (workflow-job) 2.32 (on Jenkins 2.176.1 LTS), with the "Lightweight checkout" checkbox unchecked. Is there a possibility that the system is still choosing a lightweight checkout essentially "on the fly", and if so, is there a way to force a heavyweight checkout?

Aha – for us the issue was that we were reading a file from the workspace before the start of the the pipeline block, using a command like `def common = evaluate readTrusted('./Common.groovy')`. This seems to have bypassed the variable resolution. Removing this line caused the expected behavior to start again.

We have still a problem with Pipeline: Job 2.32 (even after upgrade to 2.36) on Jenkins 2.179. "Lightweight checkout" checkbox unchecked. Parameter Expansion does not work if parameter comes from "Properties File Path", It works, however, with parameter from "Properties Content". Too bad.

BTW, our properties file is correct because it previously worked with a freestyle job.

I created new issue JENKINS-60250.

I could confirm that using `readTrusted` causes such error.

We have same problem, but in our case we don't have readTrusted
But we are using kubernetes with kubernetes-plugin, and we were using yaml file for pod configuration

  agent {
    kubernetes {
      yamlFile "pod.yaml"
    }
  }  

seems that underhood it calls readTrusted
we tried to use

  agent {
    kubernetes {
      yaml readFile("./pod.yaml")
    }
  }

but this also didn't work
only inline pod yaml helped to solve this issue

agent {
    kubernetes {
      yaml """
apiVersion: v1
kind: Pod
spec:
  containers:
  - name: jnlp
    image: imagename
    imagePullPolicy: Always

"""
    }
  }    

The issue still persists. I identified 3 modules that need to be fixed. The 

scm-api plugin does not forward the Run<?,?> to SCMFileSystem builder

git plugin would need to implement a new builder that would use the Run<?,?>

workflow-cps plugin needs to send the Run<?,?> to the scm-api in the CpsScmFlowDefinition.java to SCMFileSystem.of (which needs to be extended)

I have implemented a fix on my local machine, I have a draft opened on github although I would need some feedback, if this seems OK for the devs https://github.com/jenkinsci/scm-api-plugin/pull/160 if I am on right track
Also I got on my PC fix on the other 2 repos so It was working, although this will require some magic to make it production quality

I wondered whether I should file a request to make BitbucketSCMFileSystem in the Bitbucket Server Integration plugin implement the method that was added to SCM API. However, it seems this bug is not currently reproducible with Bitbucket Server Integration because it does not yet support lightweight checkout for this scenario. Therefore, I'm not filing such a request.

I tried to reproduce this issue with Jenkins 2.346.2, Bitbucket Server Integration plugin 3.2.2, Git plugin 4.11.4, SCM API plugin 620.v0a_5b_1f8054c0, and Pipeline: Groovy plugin 2759.v87459c4eea_ca_. Jenkins logged "Lightweight checkout support not available, falling back to full checkout", created a Git repository on the Jenkins controller, fetched refs/heads/* from the remote, resolved the commit ID of ${PIPELINE_BRANCH}, checked out that commit, and read the pipeline from the worktree. If I understand correctly, a lightweight checkout using the Bitbucket Server Integration plugin would have requested the file content from the Bitbucket Server REST API instead of creating a Git repository on the Jenkins controller, and a lightweight checkout using the Git plugin would have streamed the blob from the Git repository instead of checking out all the files.

martink89 first of all: fantastic job on getting this fixed. Works perfectly for me for regular pipeline jobs after updating to:

1. SCM API 621.vda_a_b_055e58f7
2. Pipeline: Groovy Plugin 3601.v9b_36a_d99e1cc
3. Git Plugin 4.14.1

Your efforts are greatly appreciated!

I had hoped that this would also automatically fix the same issue for the Kubernetes plugin https://plugins.jenkins.io/kubernetes/ but no such luck unfortunately.

I have a simple pipeline job with "Pipeline script from SCM" (Git), the contents of which looks something like this:

#!/usr/bin/env groovy
pipeline {
  agent {
    kubernetes {
      yamlFile 'pod-spec-of-generic-jnlp-agent-nothing-special.yml'
    }
  }
  parameters {
    string(name: 'GIT_REF', description: 'A git tag, branch or commit to check-out', trim: true)
  }
  stages {
    stage('Build') {
      steps {
        echo "Hi there!"
      }
    }
  }
}

Under the "Branches to build" settings, I have: "Branch Specifier (blank for 'any')": ${GIT_REF}

Unfortunately, when building with any GIT_REF build-parameter value, the result is:

hudson.plugins.git.GitException: Command "git fetch --tags --force --progress --prune -- origin +refs/heads/${GIT_REF}:refs/remotes/origin/${GIT_REF}" returned status code 128:
stderr: fatal: couldn't find remote ref refs/heads/${GIT_REF} 

It doesn't matter if I choose "Lightweight checkout" or not. The result is the same. The all too familiar lack of variable substitution.

When using a regular (non-kubernetes) agent, it all works perfectly. Including lightweight checkout thanks to you.

Using a regular branch expression instead of a param/env var also works fine with the kubernetes agent obviously.

I'm not sure if you are familiar with the Kubernetes plugin or not. Should I inform the authors of your work on this issue so that they might implement it as well?

jglick thank you for your reply.

I've taken your advice to heart and did some more testing. Turns out that if I put the pod spec in-line it does work.

#!/usr/bin/env groovy
pipeline {
  agent {
    kubernetes {
      yaml '''
apiVersion: v1
kind: Pod
spec:
  containers:
    - name: jnlp
      image: registry.corp/my-project/my-agent:latest
'''
    }
  }
  parameters {
    string(name: 'GIT_REF', description: 'A git tag, branch or commit to check-out', trim: true)
  }
  stages {
    stage('Build') {
      steps {
        echo "Hi there!"
      }
    }
  }
}

Results in (with GIT_REF == my-awesome-branch as build-parameter):

> git fetch --tags --force --progress -- ssh://git@bitbucket.corp:7999/my-project/my-repo.git +refs/heads/*:refs/remotes/origin/* # timeout=10
> git rev-parse refs/remotes/origin/my-awesome-branch^{commit} # timeout=10
> git rev-parse my-awesome-branch^{commit} # timeout=10
Checking out Revision 889bcecd6fa2e5d30214455360030eade3fe4b82 (refs/remotes/origin/my-awesome-branch) 

But I'm not quite sure which conclusion to draw from this.

edit: turns out I missed kirk_gm 's reply, in which he describes the exact same behavior. But no solution as of yet.

edit 2: after digging through the kubernetes plugin code, I can't find anything remotely related to this behavior either.

Ah, yamlFile specifically. That runs the readTrusted step (reference) and I suppose https://github.com/jenkinsci/workflow-multibranch-plugin/blob/e0a02cbc6c0b72cd12fbcf04a5223481a9d098e3/src/main/java/org/jenkinsci/plugins/workflow/multibranch/SCMBinder.java#L104 and (more to the point) https://github.com/jenkinsci/workflow-multibranch-plugin/blob/e0a02cbc6c0b72cd12fbcf04a5223481a9d098e3/src/main/java/org/jenkinsci/plugins/workflow/multibranch/ReadTrustedStep.java#L106 need to use the new overload.

Thanks for the pointer to the line that needs the fix jglick ! That will resolve the issue reported in JENKINS-70155 by codesuki.

paybas would you like to submit the pull request to fix the problem?

Thanks jglick that makes sense!

markewaite I haven't really touched any Java code in years, and I'm not really familiar with Jenkins plugin development. Only some trivial plugins, never the fundamental plugins.

So yes: I could give it a shot. But it will take time (which I won't have for a couple of weeks). Especially since it will need unit tests.

The code change (src/main/java/) is likely trivial: adding something like , build in a couple of spots, to thread this information along to the git plugin. But the POM would need to be updated to pick up the new dependencies, and writing a test case can be tricky, so this is not something I would recommend for someone just getting started with plugin development unless you want to get practice!

Or martink89 are you interested in taking this on?

I am not sure whether JENKINS-70155 and/or JENKINS-60697 are related. Those seem to discuss exotic refspecs for some use case I am not familiar with. This is specifically about expansion of a build parameter.

Does this resolution only work for the branch field? I am trying to use a parameter for the 'Repository URL' field as well but it does not expand. It is working for branch though.

This is also still a problem for the Repository URL (but not the branch) for me as well:
Jenkins version 2.401.3
git plugin 5.2.0

• Jenkins/Manage System/Global Properties:
  • "SHARED_REPO" = "ssh://git@server:23/path/project.git"
  • "SHARED_BRANCH" = "my_test_branch"
• In Pipeline specification:
  • Pipeline script from SCM
  • SCM = Git
  • Repository URL: "${SHARED_REPO}"
  • Branch Specifier (blank for 'any'): "*/${SHARED_BRANCH}"
  • Lightweight checkout: CHECKED

output:

hudson.plugins.git.GitException: Command "C:\Program Files\Git\bin\git.exe fetch --tags --force --progress --prune -- origin +refs/heads/my_test_branch:refs/remotes/origin/my_test_branch" returned status code 128:
stdout: 
stderr: fatal: '${SHARED_REPO}' does not appear to be a git repository

So, from the output, the plugin is parsing the environment property in the branch field, but not in the Repository URL.

jglick do you want me to create a new ticket specifically for https://issues.jenkins.io/browse/JENKINS-42971?focusedId=433866&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-433866 ?

I have the feeling that this will never be resolved as long as it's just a comment thread on a "Resolved" ticket.

edit: Nevermind. Sorry for the "mention". https://issues.jenkins.io/browse/JENKINS-70155 was buried below the fold so I missed it.