• Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Critical Critical
    • core
    • Jenkins 2.241

      SimpleBuildStep.perform needs to be given an EnvVars argument. Otherwise there is no way for an implementation to observe any local environment variable settings.

          [JENKINS-29144] SimpleBuildStep to receive EnvVars

          Jesse Glick added a comment -

          jeremym CHANGE_URL is defined in branch-api. GIT_URL etc. are map keys in the return value of git / checkout, done in JENKINS-26100.

          Jesse Glick added a comment - jeremym CHANGE_URL is defined in branch-api . GIT_URL etc. are map keys in the return value of git / checkout , done in JENKINS-26100 .

          Everywhere it's "you must implement Step instead".

          However, from the looks of things, implementing Step does not make it available in a Freestyle project.
          It seems completely ridiculous to have to duplicate all the processing (and UI jellies) just to be able to access the context environment in a pipeline.

          It also means any SimpleBuildSteps using the Launcher are potentially broken, as they will not honor any PATH set (whether via tools/environment/withEnv/...).

          My initial thought would be "can't the pipeline set the environment on the Run object for the duration of the step execution?"; but I can see how there might be issues with that (especially with async steps in the mix).

          But is there anything problematic about the following approach?

          1. add a new method to SimpleBuildStep
              default void perform(@Nonnull Run<?,?> run, @Nonnull FilePath workspace, @Nonnull EnvVars env, @Nonnull Launcher launcher,
                             @Nonnull TaskListener listener) throws InterruptedException, IOException {
                // by default, ignore the environment
                this.perform(run, workspace, launcher, listener);
              }
              
          2. adjust BuildStepCompatibilityLayer to use it:
              ...
                   if (this instanceof SimpleBuildStep) {
                        // delegate to the overloaded version defined in SimpleBuildStep
                        FilePath workspace = build.getWorkspace();
                        if (workspace == null) {
                            throw new AbortException("no workspace for " + build);
                        }
                        ((SimpleBuildStep) this).perform(build, workspace, build.getEnvironment(), launcher, listener);
                        return true;
                    }
              ...
              
          3. so far, that fully supports existing code, there's just an additional call that has its return value ignored
          4. extend CoreStep's descriptor to require EnvVars.class too, and adjust its Execution to pass it:
                    @Override protected Void run() throws Exception {
                        FilePath workspace = getContext().get(FilePath.class);
                        workspace.mkdirs();
                        delegate.perform(getContext().get(Run.class), workspace, getContext().get(EnvVars.class), getContext().get(Launcher.class), getContext().get(TaskListener.class));
                        return null;
                    }
              

          I mean, it would also be possible to add SimpleBuildStepWithEnvironment, special-case it in BuildStepCompatibilityLayer, and add a CoreStepWithEnvironment that duplicates most of CoreStep.
          That would keep all current code completely unchanged, with plugins able to switch interface to opt-in to receiving the environment too.

          Tim Van Holder added a comment - Everywhere it's "you must implement Step instead". However, from the looks of things, implementing Step does not make it available in a Freestyle project. It seems completely ridiculous to have to duplicate all the processing (and UI jellies) just to be able to access the context environment in a pipeline. It also means any SimpleBuildSteps using the Launcher are potentially broken, as they will not honor any PATH set (whether via tools/environment/withEnv/...). My initial thought would be "can't the pipeline set the environment on the Run object for the duration of the step execution?"; but I can see how there might be issues with that (especially with async steps in the mix). But is there anything problematic about the following approach? add a new method to SimpleBuildStep default void perform(@Nonnull Run<?,?> run, @Nonnull FilePath workspace, @Nonnull EnvVars env, @Nonnull Launcher launcher, @Nonnull TaskListener listener) throws InterruptedException, IOException { // by default , ignore the environment this .perform(run, workspace, launcher, listener); } adjust BuildStepCompatibilityLayer to use it: ... if ( this instanceof SimpleBuildStep) { // delegate to the overloaded version defined in SimpleBuildStep FilePath workspace = build.getWorkspace(); if (workspace == null ) { throw new AbortException( "no workspace for " + build); } ((SimpleBuildStep) this ).perform(build, workspace, build.getEnvironment(), launcher, listener); return true ; } ... so far, that fully supports existing code, there's just an additional call that has its return value ignored extend CoreStep 's descriptor to require EnvVars.class too, and adjust its Execution to pass it: @Override protected Void run() throws Exception { FilePath workspace = getContext().get(FilePath.class); workspace.mkdirs(); delegate.perform(getContext().get(Run.class), workspace, getContext().get(EnvVars.class), getContext().get(Launcher.class), getContext().get(TaskListener.class)); return null ; } I mean, it would also be possible to add SimpleBuildStepWithEnvironment, special-case it in BuildStepCompatibilityLayer , and add a CoreStepWithEnvironment that duplicates most of CoreStep . That would keep all current code completely unchanged, with plugins able to switch interface to opt-in to receiving the environment too.

          Jesse Glick added a comment -

          zastai If you have an API proposal you believe would address the gap without breaking backwards compatibility, it is easiest to discuss details in PRs:

          • a PR in jenkins with
            • a patch to SimpleBuildStep and BuldStepCompatibilityLayer
            • a patch to something in test/src/main/java/ proving that a sample step (@TestExtension) can implement the new method and receive actual build environment variables (e.g., values from ParametersAction)
          • a (draft) PR in workflow-basic-steps with jenkins.version set to the incremental version produced by the first PR, with
            • a patch to CoreStep to consume the new API
            • a patch to CoreStepTest using @TestExtension to define a sample step like the one in core, but this time tested in a Pipeline project inside a withEnv block demonstrating access to contextual environment variables
          • optionally also a (draft) PR in some OSS plugin which consumes the new API to improve an existing SimpleBuildStep

          As to the proposed API, a default method in SimpleBuildStep can be made to work but it is awkward since we would prefer to mark the old overload @Deprecated and a new implementation would want to only implement the new overload, so you would have to make the original method also default and do some tricks with Util.isOverridden and AbstractMethodError. We do this kind of dance in various places. Introducing a SimpleBuildStep2 is also a possibility though the constructor of CoreStep would pose a bit of an obstacle.

          Jesse Glick added a comment - zastai If you have an API proposal you believe would address the gap without breaking backwards compatibility, it is easiest to discuss details in PRs: a PR in jenkins with a patch to SimpleBuildStep and BuldStepCompatibilityLayer a patch to something in test/src/main/java/ proving that a sample step ( @TestExtension ) can implement the new method and receive actual build environment variables (e.g., values from ParametersAction ) a (draft) PR in workflow-basic-steps with jenkins.version set to the incremental version produced by the first PR, with a patch to CoreStep to consume the new API a patch to CoreStepTest using @TestExtension to define a sample step like the one in core, but this time tested in a Pipeline project inside a withEnv block demonstrating access to contextual environment variables optionally also a (draft) PR in some OSS plugin which consumes the new API to improve an existing SimpleBuildStep As to the proposed API, a default method in SimpleBuildStep can be made to work but it is awkward since we would prefer to mark the old overload @Deprecated and a new implementation would want to only implement the new overload, so you would have to make the original method also default and do some tricks with Util.isOverridden and AbstractMethodError . We do this kind of dance in various places. Introducing a SimpleBuildStep2 is also a possibility though the constructor of CoreStep would pose a bit of an obstacle.

          Tim Van Holder added a comment - - edited

          Part of the trickiness of doing this with a PR is that two repositories are involved.

          But I'm glad to hear that there isn't a more basic problem with changing CoreStep to require envvars while it did not before (are there failure conditions from such changes?)

           

          I think what I'll try is to make a SimpleBuildStepWithEnv and CoreStepWithEnv (names TBD) locally in my plugin. If that works, I'll see what I can do about PRs. I might look at 2 or three alternative approaches and make PRs for each.

          Tim Van Holder added a comment - - edited Part of the trickiness of doing this with a PR is that two repositories are involved. But I'm glad to hear that there isn't a more basic problem with changing CoreStep to require envvars while it did not before (are there failure conditions from such changes?)   I think what I'll try is to make a SimpleBuildStepWithEnv and CoreStepWithEnv (names TBD) locally in my plugin. If that works, I'll see what I can do about PRs. I might look at 2 or three alternative approaches and make PRs for each.

          Jesse Glick added a comment -

          Part of the trickiness of doing this with a PR is that two repositories are involved.

          This is pretty routine actually. Dev guide coming soon in https://github.com/jenkins-infra/jenkins.io/pull/3343 but https://github.com/jenkinsci/incrementals-tools has some more technical docs already.

          there isn't a more basic problem with changing CoreStep to require envvars while it did not before

          Not that I can think of offhand. But that is why you file the PRs—to see if CI says there are test failures you did not anticipate!

          Introducing CoreStepWithEnv is technically possible but sounds like a mess from a user perspective, and might not interoperate with some plugins which analyze the build graph. I would definitely recommend some approach that involves updating the existing CoreStep.

          That said, there is absolutely nothing wrong with filing alternative PRs and letting reviewers compare the impact, so long as you make the situation very clear in the PR description. (Remember to refer to one PR from another by pasting in a naked URL, not a Markdown link, so that GitHub displays a hover tip and more importantly creates an automatic backlink.) Also recommended:

          • assign Jira to yourself, mark In Progress while working and In Review when you believe you have something ready to merge (not counting draft status due to an incremental version in a dependency—up to repository maintainers to deal with that)
          • link to Jira from each PR (we do not currently have a bot to do this, shame on us)
          • link to PRs from Jira (ditto)

          Jesse Glick added a comment - Part of the trickiness of doing this with a PR is that two repositories are involved. This is pretty routine actually. Dev guide coming soon in https://github.com/jenkins-infra/jenkins.io/pull/3343 but https://github.com/jenkinsci/incrementals-tools has some more technical docs already. there isn't a more basic problem with changing CoreStep to require envvars while it did not before Not that I can think of offhand. But that is why you file the PRs—to see if CI says there are test failures you did not anticipate! Introducing CoreStepWithEnv is technically possible but sounds like a mess from a user perspective, and might not interoperate with some plugins which analyze the build graph. I would definitely recommend some approach that involves updating the existing CoreStep . That said, there is absolutely nothing wrong with filing alternative PRs and letting reviewers compare the impact, so long as you make the situation very clear in the PR description. (Remember to refer to one PR from another by pasting in a naked URL, not a Markdown link, so that GitHub displays a hover tip and more importantly creates an automatic backlink.) Also recommended: assign Jira to yourself, mark In Progress while working and In Review when you believe you have something ready to merge (not counting draft status due to an incremental version in a dependency—up to repository maintainers to deal with that) link to Jira from each PR (we do not currently have a bot to do this, shame on us) link to PRs from Jira (ditto)

          Turns out that for my immediate purpose in my plugin, it was actually very easy to add support for getting environment variables by implementing a metastep (all my builders share a base class that implements the perform() anyway, so no SimpleBuildStep needed at all).

          The trick is to know that metasteps exist, which I didn't until I read the javadoc for isMetaStep() while going over what CoreStep does.
           

          Anyway, I've assigned this to myself and will look at setting up clones of jenkins and workflow-basic-steps-plugin to see what I can make work without breaking tests.

          And thanks for the pointer to the incrementals; that's useful info.

          Tim Van Holder added a comment - Turns out that for my immediate purpose in my plugin, it was actually very easy to add support for getting environment variables by implementing a metastep (all my builders share a base class that implements the perform() anyway, so no SimpleBuildStep needed at all). The trick is to know that metasteps exist, which I didn't until I read the javadoc for isMetaStep() while going over what CoreStep does.   Anyway, I've assigned this to myself and will look at setting up clones of jenkins and workflow-basic-steps-plugin to see what I can make work without breaking tests. And thanks for the pointer to the incrementals; that's useful info.

          Jesse Glick added a comment -

          The metastep system was not intended for use by domain-specific plugins, i.e. for anything other than core, wrap, and in the future perhaps checkout.

          Jesse Glick added a comment - The metastep system was not intended for use by domain-specific plugins, i.e. for anything other than core , wrap , and in the future perhaps checkout .

          Add link to draft plugin PR.

          Tim Van Holder added a comment - Add link to draft plugin PR.

          Oleg Nenashev added a comment -

          The Jenkins core patch was released in Jenkins 2.241

          Oleg Nenashev added a comment - The Jenkins core patch was released in Jenkins 2.241

          Technically this issue won't be solved until workflow-basic-steps is updated too.

          Tim Van Holder added a comment - Technically this issue won't be solved until workflow-basic-steps is updated too.

            zastai Tim Van Holder
            jglick Jesse Glick
            Votes:
            3 Vote for this issue
            Watchers:
            14 Start watching this issue

              Created:
              Updated:
              Resolved: