Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-43681

Method calls in environment block are always null

      I am attempting to calculate a value to be used in a few different places in my pipeline. The definition is in a method (in a global library, but I have identical behavior with an inline method).

      environment {
        MY_VAR = calculate()
      }

      In this scenario, MY_VAR is always null--even if the RHS is the GString {{ "${1+2}" }}! In fact, credentials() is the only non-compile-time-constant expression that I can get to evaluate to non-null.

      I expected the values in the environment block to be evaluated on the master when the pipeline is started up and then injected appropriately.

          [JENKINS-43681] Method calls in environment block are always null

          Andrew Bayer added a comment -

          So this should work fine with a global library (not a library loaded via @Library or the libraries directive, though) - see https://github.com/jenkinsci/pipeline-model-definition-plugin/blob/master/pipeline-model-definition/src/test/resources/nonLiteralEnvironment.groovy for an example (returnAThing(...) is defined in the global library). This should get reworked for @Library and libraries to work, but an inline method I wouldn't expect to ever work.

          Andrew Bayer added a comment - So this should work fine with a global library (not a library loaded via @Library or the libraries directive, though) - see https://github.com/jenkinsci/pipeline-model-definition-plugin/blob/master/pipeline-model-definition/src/test/resources/nonLiteralEnvironment.groovy  for an example ( returnAThing(...) is defined in the global library). This should get reworked for @Library and libraries to work, but an inline method I wouldn't expect to ever work.

          Basically, the rules for the transform here are startlingly opaque, especially given the amount of treatment that the credentials() feature gets in the documentation. I can see a number of arbitrary breakpoints that could be defended as "this amount of foo is supported, but no more", but the limits and rationale need to be documented, and the proper handling of unsupported whatevers should be an error instead of a silent null. The lack of support for a constant-resulting GString is particularly unintuitive.

          Christopher Smith added a comment - Basically, the rules for the transform here are startlingly opaque, especially given the amount of treatment that the credentials() feature gets in the documentation. I can see a number of arbitrary breakpoints that could be defended as "this amount of foo is supported, but no more", but the limits and rationale need to be documented, and the proper handling of unsupported whatevers should be an error instead of a silent null. The lack of support for a constant-resulting GString is particularly unintuitive.

          Andrew Bayer added a comment -

          You are not wrong. That said, I could have sworn I had tests for 1+2 sort of case...

          Andrew Bayer added a comment - You are not wrong. That said, I could have sworn I had tests for 1+2 sort of case...

          Andrew Bayer added a comment -

          Ah, yeah, we basically do - see https://github.com/jenkinsci/pipeline-model-definition-plugin/blob/master/pipeline-model-definition/src/test/resources/nonLiteralEnvironment.groovy again, specifically the ANOTHER_ENV declaration. What version are you on?

          Andrew Bayer added a comment - Ah, yeah, we basically do - see https://github.com/jenkinsci/pipeline-model-definition-plugin/blob/master/pipeline-model-definition/src/test/resources/nonLiteralEnvironment.groovy again, specifically the ANOTHER_ENV declaration. What version are you on?

          Andrew Bayer added a comment -

          Andrew Bayer added a comment - fwiw, adding a test for this specifically in https://github.com/jenkinsci/pipeline-model-definition-plugin/pull/154

          Looks like 1.1.2 for everything, and I'm seeing an update to 1.1.3, and looking at the repo that seems that the feature expansion happened on the step between 1.1.2 and 1.1.3!

          Christopher Smith added a comment - Looks like 1.1.2 for everything, and I'm seeing an update to 1.1.3, and looking at the repo that seems that the feature expansion happened on the step between 1.1.2 and 1.1.3!

          Upgraded to 1.1.3, and it looks like the value is being added to the environment. This works as expected:

          environment {
            FIVE = "${2 + 3}"
            TWENTY_FIVE = "2${FIVE}"
          }
          

          However, method calls within the GString, like this:

          environment {
            MY_VAR = "${formatDate()}-${BUILD_ID}"
          }
          

          where formatDate is a var in a local plugin, silently leaves MY_VAR undefined (causing various sorts of errors later on). Trying to use

          MY_VAR = formatDate() + "-${BUILD_ID}"
          

          resulted in "Environment variable values to be concatenated together must be single or double quoted". Using just

          MY_VAR = formatDate()
          

          works as expected, so it seems that the var bindings aren't getting applied to the GString evaluation context (and evaluation exceptions are getting swallowed silently). This is something I can work around a bit clumsily (export the string concat to a different method).

          Christopher Smith added a comment - Upgraded to 1.1.3, and it looks like the value is being added to the environment. This works as expected: environment { FIVE = "${2 + 3}" TWENTY_FIVE = "2${FIVE}" } However, method calls within the GString , like this: environment { MY_VAR = "${formatDate()}-${BUILD_ID}" } where formatDate is a var in a local plugin, silently leaves MY_VAR undefined (causing various sorts of errors later on). Trying to use MY_VAR = formatDate() + "-${BUILD_ID}" resulted in "Environment variable values to be concatenated together must be single or double quoted". Using just MY_VAR = formatDate() works as expected, so it seems that the var bindings aren't getting applied to the GString evaluation context (and evaluation exceptions are getting swallowed silently). This is something I can work around a bit clumsily (export the string concat to a different method).

          (As a side note, my apologies if I was rude in the initial exchange; I was at the end of a marathon trying to get the pipeline in place and had the social module turned off. The Declarative Pipeline is a major improvement in CI generally, and I'm glad to be adopting it.)

          Christopher Smith added a comment - (As a side note, my apologies if I was rude in the initial exchange; I was at the end of a marathon trying to get the pipeline in place and had the social module turned off. The Declarative Pipeline is a major improvement in CI generally, and I'm glad to be adopting it.)

          By the way, I don't know what the design decision here was, so I'm not sure whether/how to open a new issue on it, but the identifier MY_VAR in that case isn't defined in the steps themselves, so while I can say

          echo "$MY_VAR"
          

          I can't say

          doStep MY_VAR
          

          but have to use env.MY_VAR. It would be nice for the former to work, but I can imagine clashes that led to a decision not to, in which case it would be helpful to have a thorough explanation of why env is necessary in the docs.

          Christopher Smith added a comment - By the way, I don't know what the design decision here was, so I'm not sure whether/how to open a new issue on it, but the identifier MY_VAR in that case isn't defined in the steps themselves, so while I can say echo "$MY_VAR" I can't say doStep MY_VAR but have to use env.MY_VAR . It would be nice for the former to work, but I can imagine clashes that led to a decision not to, in which case it would be helpful to have a thorough explanation of why env is necessary in the docs.

          Andrew Bayer added a comment -

          Weeeeird - I honestly didn't think the changes in 1.1.3 would have made a difference with this, but hey, I'll take it. =)

          Of those environment cases - yeaaaaah. There's a lot of hoop-jumping and evaluation at different points, so what's happening here is that the method is getting evaluated without the full context. I'll think on it and see if I can come up with a solution.

          Re: the MY_VAR thing - what exactly is the error you're getting?

          Andrew Bayer added a comment - Weeeeird - I honestly didn't think the changes in 1.1.3 would have made a difference with this, but hey, I'll take it. =) Of those environment cases - yeaaaaah. There's a lot of hoop-jumping and evaluation at different points, so what's happening here is that the method is getting evaluated without the full context. I'll think on it and see if I can come up with a solution. Re: the MY_VAR thing - what exactly is the error you're getting?

          Code changed in jenkins
          User: Andrew Bayer
          Path:
          pipeline-model-definition/src/test/java/org/jenkinsci/plugins/pipeline/modeldefinition/EnvironmentTest.java
          pipeline-model-definition/src/test/resources/nonLiteralEnvironment.groovy
          http://jenkins-ci.org/commit/pipeline-model-definition-plugin/e2001b6a7907bec8298faeafa16a88c7bcaa1ff4
          Log:
          JENKINS-43681 Just verify that a simple expression works as well

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Andrew Bayer Path: pipeline-model-definition/src/test/java/org/jenkinsci/plugins/pipeline/modeldefinition/EnvironmentTest.java pipeline-model-definition/src/test/resources/nonLiteralEnvironment.groovy http://jenkins-ci.org/commit/pipeline-model-definition-plugin/e2001b6a7907bec8298faeafa16a88c7bcaa1ff4 Log: JENKINS-43681 Just verify that a simple expression works as well

          Code changed in jenkins
          User: Andrew Bayer
          Path:
          pipeline-model-definition/src/test/java/org/jenkinsci/plugins/pipeline/modeldefinition/EnvironmentTest.java
          pipeline-model-definition/src/test/resources/nonLiteralEnvironment.groovy
          http://jenkins-ci.org/commit/pipeline-model-definition-plugin/595bf3379ed67a962902084149e1bf81cb437056
          Log:
          Merge pull request #154 from abayer/jenkins-43681-verify-test

          JENKINS-43681 Just verify that a simple expression works as well

          Compare: https://github.com/jenkinsci/pipeline-model-definition-plugin/compare/a185431a9658...595bf3379ed6

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Andrew Bayer Path: pipeline-model-definition/src/test/java/org/jenkinsci/plugins/pipeline/modeldefinition/EnvironmentTest.java pipeline-model-definition/src/test/resources/nonLiteralEnvironment.groovy http://jenkins-ci.org/commit/pipeline-model-definition-plugin/595bf3379ed67a962902084149e1bf81cb437056 Log: Merge pull request #154 from abayer/jenkins-43681-verify-test JENKINS-43681 Just verify that a simple expression works as well Compare: https://github.com/jenkinsci/pipeline-model-definition-plugin/compare/a185431a9658...595bf3379ed6

          Andrew Bayer added a comment -

          chrylis Ping - I'm trying to clean up tickets. Is this still a problem for you?

          Andrew Bayer added a comment - chrylis Ping - I'm trying to clean up tickets. Is this still a problem for you?

          Andrew Bayer added a comment -

          This definitely works with the JENKINS-42753 PR in any form I could come up with, even if it isn't working in Declarative 1.1.x. So worst case, it's fixed in the upcoming Declarative 1.2 release.

          Andrew Bayer added a comment - This definitely works with the JENKINS-42753 PR in any form I could come up with, even if it isn't working in Declarative 1.1.x. So worst case, it's fixed in the upcoming Declarative 1.2 release.

          Liam Newman added a comment -

          Bulk closing resolved issues.

          Liam Newman added a comment - Bulk closing resolved issues.

            abayer Andrew Bayer
            chrylis Christopher Smith
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: