Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-53649

Strings from the "echo" step are suppressed in BlueOcean UI if they contain values found in an environment variable

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Minor Minor
    • workflow-cps-plugin
    • Jenkins: 2.138.1
      BlueOcean plugin: 1.8.2
      A clean install on Windows 7 with just a standard set of plugins
      Reproed with Chrome on Windows and Firefox on a Linux desktop

      When a string is displayed in the BlueOcean UI from an "echo" step, the string "Print Message" is displayed IF the string being echoed contains a substring that matches the contents of any environment variable. 

      Repro steps:

      Create a pipeline with this code:

      pipeline {
          agent any
          stages {
              stage('test') {
                  environment {
                      THING = 'foobarbuzz'
                  }
                  steps {
                      echo "blah"
                      echo "foobarbuzz"
                      echo "foobarZZZbuzz"
                      echo "This is a $THING here"
                  }
              }
          }
      }

      Run the pipeline via Blue Ocean

      Observe that there are four step labels displayed in the Blue Ocean UI. (see attached screenshot)  The first and third steps display the strings from "echo" as expected.  But, the second and fourth steps display only "Print Message" and don't show the expected string.  The user needs to click the step to expand it to see the string.  I would expect that all strings from all echo statements should show up in the main Blue Ocean UI and would not require some to be manually expended.

       

          [JENKINS-53649] Strings from the "echo" step are suppressed in BlueOcean UI if they contain values found in an environment variable

          mcw Can you specify the versions you tested those? I guess we can just write a script instruction to not echo anything and place the actual String in the label... have you tried that?

          Marcello de Sales added a comment - mcw Can you specify the versions you tested those? I guess we can just write a script instruction to not echo anything and place the actual String in the label... have you tried that?

          Brian J Murrell added a comment - - edited

          mcw Even using {{label:}}s produces inconsistent results:

          Those two steps in the red box are both of the form:

          sh label: <label_name>,
             script: '''...'''
          

          The first one is in the steps portion of the stage and looks like:

          sh label: env.STAGE_NAME,
             script: '''...'''
          

          and the second one is in the post->success portion of the stage and looks like:

           sh label: "Collect artifacts",
               script: '''...'''
          

          But as you can see, they are displayed quite differently.

          I've opened JENKINS-59918 about this, FWIW.

          Brian J Murrell added a comment - - edited mcw Even using {{label:}}s produces inconsistent results: Those two steps in the red box are both of the form: sh label: <label_name>, script: '''...''' The first one is in the steps portion of the stage and looks like: sh label: env.STAGE_NAME, script: '''...''' and the second one is in the post -> success portion of the stage and looks like: sh label: "Collect artifacts", script: '''...''' But as you can see, they are displayed quite differently. I've opened  JENKINS-59918 about this, FWIW.

          It's been a year and this hasn't been fixed.  Perhaps the standard/static Jenkins community is starting to die out a little bit in favor of things like Jenkins-X.  

          Shaun McDonnell added a comment - It's been a year and this hasn't been fixed.  Perhaps the standard/static Jenkins community is starting to die out a little bit in favor of things like Jenkins-X.  

          I would also be very happy about this issue being resolved (in order to nicely show a parameter value) and actually would have hoped that this is easy for someone like bitwiseman (of course I am not just trying to challenge him in any way )...

          The workaround with sh step using label attribute also requires a node wrapping context and thus is quite some runtime overhead too.
          JENKINS-55612 could also be considered some overhead with respect to written pipeline code.
          I am quite confused/concerned/surprised about the mentioned/found inconsistencies and whether or not this is actually on purpose (cf. parameter and env values) and why?
          Another workaround is adding a non-printable anywhere in the middle, e.g. for an URL parameter: echo "URL: '${params.URL.replace('.', '.\u007F')}'"

          Reinhold Füreder added a comment - I would also be very happy about this issue being resolved (in order to nicely show a parameter value) and actually would have hoped that this is easy for someone like bitwiseman (of course I am not just trying to challenge him in any way )... The workaround with sh step using label attribute also requires a node wrapping context and thus is quite some runtime overhead too. JENKINS-55612 could also be considered some overhead with respect to written pipeline code. I am quite confused/concerned/surprised about the mentioned/found inconsistencies and whether or not this is actually on purpose (cf. parameter and env values) and why? Another workaround is adding a non-printable anywhere in the middle, e.g. for an URL parameter: echo "URL: '${params.URL.replace('.', '.\u007F')}'"

          Andreas Meyer added a comment -

          I believe this is one of the little bugs that are not really serious but still would make many people happy when fixed, because it is so annoying.

          I've found a generic workaround that doesn't use a shell step, so it's much more efficient. The trick is to temporarily clear the EnvVars for the duration of the "echo" call.

          DISCLAIMER: This workaround disables the masking of secrets during the echo call, so use it at your own risk and especially don't use it within "withCredentials" scope!

          If anyone has an idea to prevent this issue, please reply. We could propably call "getContext" to detect any "withCredentials" context and fallback to plain "echo" in this case. I could live with an occassional "Print Message" in "withCredentials" scopes. Unfortunately "getContext" prints a message on its own, so it's not very useful .

          Sample code:

          pipeline{
              agent any
          
              environment {
                  MY_ENV1 = 'foobarbaz'
              }
              
              parameters { 
                  string(name: 'MY_PARAM', defaultValue: 'bazbarfoo') 
              }
              
              stages {
                  stage('test') {
                      steps {
                          script {
                              env.MY_ENV2 = 'foobazbar'
                              
                              withEnv(['MY_ENV3=bazfoobar']) {
          
                                  // Any of the resolved variable values trigger the "Print message" of plain echo
                                  def msg = "MY_ENV1=$MY_ENV1, MY_ENV2=$MY_ENV2, MY_ENV3=$MY_ENV3, MY_PARAM=$MY_PARAM"
          
                                  echo "echo: $msg"        // Displays "Print Message" as the step label
          
                                  myecho "myecho: $msg"    // Displays the message as the step label, as expected
                              }
                          }
                      }
                  }
              }
          }
          
          // Workaround for "Print message" that is displayed by original echo step if the message
          // contains a value of an environment variable or a pipeline parameter.
          //
          // WARNING: This disables masking of secrets in the output, so don't call it within witCredentials{} scope!
          
          void myecho( String msg ) {
              withContext( new MyEnvClearer() ) {
                  echo msg
              }
          }
          
          // Clears all environment variables, to be used from withContext{}.
          class MyEnvClearer extends org.jenkinsci.plugins.workflow.steps.EnvironmentExpander {
              @NonCPS
              void expand( hudson.EnvVars env ) throws IOException, InterruptedException {
                  env.clear()    
              }    
          }
          

          Andreas Meyer added a comment - I believe this is one of the little bugs that are not really serious but still would make many people happy when fixed, because it is so annoying. I've found a generic workaround that doesn't use a shell step, so it's much more efficient. The trick is to temporarily clear the EnvVars for the duration of the "echo" call. DISCLAIMER: This workaround disables the masking of secrets during the echo call, so use it at your own risk and especially don't use it within "withCredentials" scope! If anyone has an idea to prevent this issue, please reply. We could propably call "getContext" to detect any "withCredentials" context and fallback to plain "echo" in this case. I could live with an occassional "Print Message" in "withCredentials" scopes. Unfortunately "getContext" prints a message on its own, so it's not very useful . Sample code: pipeline{ agent any environment { MY_ENV1 = 'foobarbaz' } parameters { string(name: 'MY_PARAM' , defaultValue: 'bazbarfoo' ) } stages { stage( 'test' ) { steps { script { env.MY_ENV2 = 'foobazbar' withEnv([ 'MY_ENV3=bazfoobar' ]) { // Any of the resolved variable values trigger the "Print message" of plain echo def msg = "MY_ENV1=$MY_ENV1, MY_ENV2=$MY_ENV2, MY_ENV3=$MY_ENV3, MY_PARAM=$MY_PARAM" echo "echo: $msg" // Displays "Print Message" as the step label myecho "myecho: $msg" // Displays the message as the step label, as expected } } } } } } // Workaround for "Print message" that is displayed by original echo step if the message // contains a value of an environment variable or a pipeline parameter. // // WARNING: This disables masking of secrets in the output, so don't call it within witCredentials{} scope! void myecho( String msg ) { withContext( new MyEnvClearer() ) { echo msg } } // Clears all environment variables, to be used from withContext{}. class MyEnvClearer extends org.jenkinsci.plugins.workflow.steps.EnvironmentExpander { @NonCPS void expand( hudson.EnvVars env ) throws IOException, InterruptedException { env.clear() } }

          CHRISTOPHER KELLY added a comment - - edited

          In the same vein as zett42 I created a groovy library to wrap his method for easy use in my pipelines.

           

          I call the file unsafeEcho.groovy and call it via 

          unsafeEcho "SHORT_BUILD_VERSION: $SHORT_BUILD_VERSION"
          
          #!/usr/bin/env groovy
          
          // Workaround for "Print message" that is displayed by original echo step if the message
          // contains a value of an environment variable or a pipeline parameter.
          //
          // WARNING: This disables masking of secrets in the output, so don't call it 
          
          void call( String msg ) { 
              withContext( new unsafeEchoClearer() ) { 
                  echo msg 
              } 
          }
          
          
          // Clears all environment variables, to be used from withContext{}.
          class unsafeEchoClearer extends org.jenkinsci.plugins.workflow.steps.EnvironmentExpander {
              @NonCPS
              void expand( hudson.EnvVars env ) throws IOException, InterruptedException {
                  env.clear()    
              }    
          }
          

           

          CHRISTOPHER KELLY added a comment - - edited In the same vein as zett42 I created a groovy library to wrap his method for easy use in my pipelines.   I call the file unsafeEcho.groovy and call it via  unsafeEcho "SHORT_BUILD_VERSION: $SHORT_BUILD_VERSION" #!/usr/bin/env groovy // Workaround for "Print message" that is displayed by original echo step if the message // contains a value of an environment variable or a pipeline parameter. // // WARNING: This disables masking of secrets in the output, so don't call it void call( String msg ) { withContext( new unsafeEchoClearer() ) { echo msg } } // Clears all environment variables, to be used from withContext{}. class unsafeEchoClearer extends org.jenkinsci.plugins.workflow.steps.EnvironmentExpander { @NonCPS void expand( hudson.EnvVars env ) throws IOException, InterruptedException { env.clear() } }  

          Harry Weppner added a comment -

          Been affected by this issue as well and debugged further. Based on my findings the "flaw" is in the attempt of locating environment variables that might be credentials bindings.

           https://github.com/jenkinsci/workflow-cps-plugin/blame/master/src/main/java/org/jenkinsci/plugins/workflow/cps/DSL.java#L240

          My "slightly more than minimal failing" example after configuring a secret text with the id hello is:

          withEnv(['FOO=bar']) {
              echo "Testing bar world"
              withCredentials([string(credentialsId: 'hello', variable: 'MY_SECRET')]) {
                  echo "Testing bar world"
                  withEnv(['BAR=baz']) {
                      echo "Testing bar world"
                      withCredentials([string(credentialsId: 'hello', variable: 'MY_OTHER_SECRET')]) {
                          echo "Testing bar world"
                      }
                  }
              }
          }

          The issue is that we can currently not cleanly separate credentials from other env variables. As a result, all env vars are effectively treated as credentials and their output is masked. That's also why I don't think this is an issue with Blue Ocean.

          I think the system would function as intended if only env vars used as credentials were passed in as constructors to ArgumentsActionImpl at https://github.com/jenkinsci/workflow-cps-plugin/blame/master/src/main/java/org/jenkinsci/plugins/workflow/cps/DSL.java#L246.

          jglick what would be a clean way to only determine env vars that were added as credentials?

           

          Harry Weppner added a comment - Been affected by this issue as well and debugged further. Based on my findings the "flaw" is in the attempt of locating environment variables that might  be credentials bindings.   https://github.com/jenkinsci/workflow-cps-plugin/blame/master/src/main/java/org/jenkinsci/plugins/workflow/cps/DSL.java#L240 My "slightly more than minimal failing" example after configuring a secret text with the id hello is: withEnv([ 'FOO=bar' ]) { echo "Testing bar world" withCredentials([string(credentialsId: 'hello' , variable: 'MY_SECRET' )]) { echo "Testing bar world" withEnv([ 'BAR=baz' ]) { echo "Testing bar world" withCredentials([string(credentialsId: 'hello' , variable: 'MY_OTHER_SECRET' )]) { echo "Testing bar world" } } } } The issue is that we can currently not cleanly separate credentials from other env variables. As a result, all env vars are effectively treated as credentials and their output is masked. That's also why I don't think this is an issue with Blue Ocean. I think the system would function as intended if only  env vars used as credentials were passed in as constructors to ArgumentsActionImpl at  https://github.com/jenkinsci/workflow-cps-plugin/blame/master/src/main/java/org/jenkinsci/plugins/workflow/cps/DSL.java#L246. jglick what would be a clean way to only determine env vars that were added as credentials?  

          Jesse Glick added a comment -

          Well known limitation of the current design. Essentially the same problem as what is described in JENKINS-47101, though with different symptoms. APIs to solve this have been discussed recently, but no promises.

          Jesse Glick added a comment - Well known limitation of the current design. Essentially the same problem as what is described in JENKINS-47101 , though with different symptoms. APIs to solve this have been discussed recently, but no promises.

          Harry Weppner added a comment -

          jglick JENKINS-47101 seems to be the conflicting requirement to suppress credentials in a step, which is different from this issue where non-credentials are suppressed.

          Where are these discussions taking place? Would be happy to help!

          What if `EnvironmentExpander` had a method to yield only env vars that represent a `Secret`. With that the env vars passed into the sanitization could only retain those and argument masking should work as intended!?

          Harry Weppner added a comment - jglick JENKINS-47101 seems to be the conflicting requirement to suppress credentials in a step, which is different from this issue where non-credentials are suppressed. Where are these discussions taking place? Would be happy to help! What if `EnvironmentExpander` had a method to yield only env vars that represent a `Secret`. With that the env vars passed into the sanitization could only retain those and argument masking should work as intended!?

          Jesse Glick added a comment -

          JENKINS-47101 seems to be the conflicting requirement

          Yes, as I said the symptoms are different. The underlying problem is the lack of an API to determine whether environment variables are sensitive, something like AbstractBuild.getSensitiveBuildVariables but suited to Pipeline’s needs; in lieu of that, workflow-cps assumes most variables are sensitive, but makes exemptions for a fixed list of common system variables. And yes EnvironmentExpander is the likely locus of such a new API, such as Set<String> getSensitiveVariables().

          Jesse Glick added a comment - JENKINS-47101 seems to be the conflicting requirement Yes, as I said the symptoms are different. The underlying problem is the lack of an API to determine whether environment variables are sensitive, something like AbstractBuild.getSensitiveBuildVariables but suited to Pipeline’s needs; in lieu of that, workflow-cps assumes most variables are sensitive, but makes exemptions for a fixed list of common system variables. And yes EnvironmentExpander is the likely locus of such a new API, such as Set<String> getSensitiveVariables() .

            Unassigned Unassigned
            sirgnip S Nelson
            Votes:
            25 Vote for this issue
            Watchers:
            29 Start watching this issue

              Created:
              Updated:
              Resolved: