Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-54133

No ANSI coloring on slave agents in pipeline

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Minor
    • Resolution: Fixed

    Description

      Pipeline:

      node('master'){
          ansiColor('xterm') {
              sh 'echo -e "\033[31mRed\033[0m"'
          }
          wrap([$class: 'AnsiColorBuildWrapper', 'colorMapName': 'XTerm']) {
              sh 'echo -e "\033[31mRed\033[0m"'
          }
      }
      node('ansible'){
          ansiColor('xterm') {
              sh 'echo -e "\033[31mRed\033[0m"'
          }
          wrap([$class: 'AnsiColorBuildWrapper', 'colorMapName': 'XTerm']) {
              sh 'echo -e "\033[31mRed\033[0m"'
          }
      }
      

      Output:

       

       

       

      I've verified I see ANSI coloring on a freestyle project restricted to the same 'ansible' label

      Shell build step

       

      echo -e "\033[31mRed\033[0m"
      

      Output

       

       

      I've verified this happens on multiple Jenkins instances running 2.138.2. This started happening after the following plugin updates:

      -rw-r--r--. 1 jenkins jenkins  560808 Oct 12 16:54 workflow-cps.jpi
      -rw-r--r--. 1 jenkins jenkins  111622 Oct 12 16:54 workflow-job.jpi
      -rw-r--r--. 1 jenkins jenkins  311324 Oct 12 16:54 ws-cleanup.jpi
      -rw-r--r--. 1 jenkins jenkins  360909 Oct 12 16:54 workflow-support.jpi
      

       

       

      Attachments

        Issue Links

          Activity

            vivek Vivek Pandey added a comment -

            jglick ptal to rule out possible regression. Thanks.

            vivek Vivek Pandey added a comment - jglick ptal to rule out possible regression. Thanks.
            jglick Jesse Glick added a comment - - edited

            Likely similar cause to JENKINS-54081—console notes generated in the agent JVM are not trusted and thus ignored during rendering. The best fix would likely be analogous to what I did for JENKINS-48344: deprecate the build wrapper at least for Pipeline, and replace the contextual ConsoleLogFilter with a global ConsoleAnnotatorFactory that is able to render ANSI escape sequences in the HTML view rather than embedding SimpleHtmlNote objects. This would have the same efficiency benefit as for JENKINS-48344 (get rid of a vast volume of junk in log), as well as improving Blue Ocean compatibility with a major sore point with this plugin today: you can choose to either enable the wrapper and thus render colors in classic UI but see plain text in BO; or do nothing and thus render colors in BO but see raw escape sequences in classic UI.

            jglick Jesse Glick added a comment - - edited Likely similar cause to JENKINS-54081 —console notes generated in the agent JVM are not trusted and thus ignored during rendering. The best fix would likely be analogous to what I did for JENKINS-48344 : deprecate the build wrapper at least for Pipeline, and replace the contextual ConsoleLogFilter with a global ConsoleAnnotatorFactory that is able to render ANSI escape sequences in the HTML view rather than embedding SimpleHtmlNote objects. This would have the same efficiency benefit as for JENKINS-48344 (get rid of a vast volume of junk in log ), as well as improving Blue Ocean compatibility with a major sore point with this plugin today: you can choose to either enable the wrapper and thus render colors in classic UI but see plain text in BO; or do nothing and thus render colors in BO but see raw escape sequences in classic UI.
            jglick Jesse Glick added a comment -

            I forgot to mention that as in JENKINS-54081 it is likely possible to work around this (have not yet tried it) using the JVM option -Dorg.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep.USE_WATCHING=false, or (do not try this at home!) by disabling the SECURITY-382 defense.

            jglick Jesse Glick added a comment - I forgot to mention that as in JENKINS-54081 it is likely possible to work around this (have not yet tried it) using the JVM option -Dorg.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep.USE_WATCHING=false , or (do not try this at home!) by disabling the SECURITY-382 defense.
            jglick Jesse Glick added a comment -

            I am afraid there are several more cases where a plugin offers a ConsoleLogFilter (whether directly via BodyInvoker.mergeConsoleLogFilters, or indirectly via CoreWrapperStep calling SimpleBuildWrapper.createLoggerDecorator) that is Serializable on the master (so, can be tracked in program.dat) yet is not safely remotable, typically because it creates a ConsoleNote which is not (or reasonably could not be) pregenerated on the master side and transmitted to the agent in encoded form. This is one example in the ant plugin; the pipeline-maven plugin similarly does colorizing; the logstash plugin has a call which would not work remotely, though I am not much concerned since this use case is addressed directly by JEP-210; pipeline-utility-steps has a tee step but this does work. So I need to think about whether there is a general way to either detect filters which would add a ConsoleNote and process them master-side when using the default log storage; or securely pregenerate the note; or rework all of these to support remote filtering.

            jglick Jesse Glick added a comment - I am afraid there are several more cases where a plugin offers a ConsoleLogFilter (whether directly via BodyInvoker.mergeConsoleLogFilters , or indirectly via CoreWrapperStep calling SimpleBuildWrapper.createLoggerDecorator ) that is Serializable on the master (so, can be tracked in program.dat ) yet is not safely remotable, typically because it creates a ConsoleNote which is not (or reasonably could not be) pregenerated on the master side and transmitted to the agent in encoded form. This is one example in the ant plugin; the pipeline-maven plugin similarly does colorizing ; the logstash plugin has a call which would not work remotely , though I am not much concerned since this use case is addressed directly by JEP-210; pipeline-utility-steps has a tee step but this does work . So I need to think about whether there is a general way to either detect filters which would add a ConsoleNote and process them master-side when using the default log storage; or securely pregenerate the note; or rework all of these to support remote filtering.
            jglick Jesse Glick added a comment -

            Have a functional test demonstrating the essential aspect of the problem.

            jglick Jesse Glick added a comment - Have a functional test demonstrating the essential aspect of the problem.
            jglick Jesse Glick added a comment -

            Fix for the ant plugin was easy at least.

            jglick Jesse Glick added a comment - Fix for the ant plugin was easy at least.
            jglick Jesse Glick added a comment -

            I have not yet tried it, but studying the pipeline-maven code it looks like that should be easy to fix as well, by pregenerating notes.

            The ansicolor case is harder because pregenerating notes looks complex. SimpleHtmlNote would be the most convenient possible vector for a stored-XSS attack were it not for SECURITY-382. There is a range of element names apparently passed to AnsiAttributeElement, which would not be a major problem, but for some types there is also a range of colors passed in, and for FGBG there is a pair of colors, thus 8 × 8 = 64 possibilities that would need to be pregenerated.

            Again creating a ConsoleAnnotatorFactory would be by far the preferable solution in general, even if there were no ConsoleNote compatibility issue—you pay the cost of scanning only during HTML rendering in the classic UI, if even ever shown; the log file is not bloated by large encrypted notes; and you are not in a Catch-22 w.r.t. Blue Ocean—but the extensive logic in AnsiHtmlOutputStream + AnsiOutputStream would have to be reworked significantly to follow this API, which is not structured as a filter on a stream but as a hook able to insert markup.

            jglick Jesse Glick added a comment - I have not yet tried it, but studying the pipeline-maven code it looks like that should be easy to fix as well, by pregenerating notes. The ansicolor case is harder because pregenerating notes looks complex. SimpleHtmlNote would be the most convenient possible vector for a stored-XSS attack were it not for SECURITY-382. There is a range of element names apparently passed to AnsiAttributeElement , which would not be a major problem, but for some types there is also a range of colors passed in, and for FGBG there is a pair of colors, thus 8 × 8 = 64 possibilities that would need to be pregenerated. Again creating a ConsoleAnnotatorFactory would be by far the preferable solution in general, even if there were no ConsoleNote compatibility issue—you pay the cost of scanning only during HTML rendering in the classic UI, if even ever shown; the log file is not bloated by large encrypted notes; and you are not in a Catch-22 w.r.t. Blue Ocean—but the extensive logic in AnsiHtmlOutputStream + AnsiOutputStream would have to be reworked significantly to follow this API, which is not structured as a filter on a stream but as a hook able to insert markup.
            jglick Jesse Glick added a comment -

            Since I see that I have not commented here recently: I do plan to offer a fix for this, I am just trying to deal with higher-priority issues first (JENKINS-54073, then JENKINS-54078, then JENKINS-54081, then this).

            jglick Jesse Glick added a comment - Since I see that I have not commented here recently: I do plan to offer a fix for this, I am just trying to deal with higher-priority issues first ( JENKINS-54073 , then JENKINS-54078 , then JENKINS-54081 , then this).
            jglick Jesse Glick added a comment -

            Just found this code suggesting that note pregeneration is not possible in general (though the common cases could be handled).

            jglick Jesse Glick added a comment - Just found this code suggesting that note pregeneration is not possible in general (though the common cases could be handled).
            jglick Jesse Glick added a comment -

            Filed a PR with the sketch of a fix that currently just handles escape sequences that set a (regular-intensity) foreground color from the colormap, as in the example and the existing test coverage.

            The repository is currently set up to build on Travis CI. If .travis.yml were replaced or supplemented with a Jenkinsfile

            buildPlugin()
            

            then I could mvn incrementals:incrementalify and so PR builds on ci.jenkins.io would automatically publish experimental releases for people to evaluate, via JEP-305. In the meantime, anyone wanting to try this would have to build from sources I think.

            jglick Jesse Glick added a comment - Filed a PR with the sketch of a fix that currently just handles escape sequences that set a (regular-intensity) foreground color from the colormap, as in the example and the existing test coverage. The repository is currently set up to build on Travis CI. If .travis.yml were replaced or supplemented with a Jenkinsfile buildPlugin() then I could mvn incrementals:incrementalify and so PR builds on ci.jenkins.io would automatically publish experimental releases for people to evaluate, via JEP-305 . In the meantime, anyone wanting to try this would have to build from sources I think.
            jglick Jesse Glick added a comment -

            I seem to have succeeded in using ConsoleAnnotatorFactory after all.

            jglick Jesse Glick added a comment - I seem to have succeeded in using ConsoleAnnotatorFactory after all.
            jglick Jesse Glick added a comment -

            All associated PRs merged.

            jglick Jesse Glick added a comment - All associated PRs merged.

            Delivered in pipeline-maven-plugin 3.6.0

            cleclerc Cyrille Le Clerc added a comment - Delivered in pipeline-maven-plugin 3.6.0
            jglick Jesse Glick added a comment -

            Thus the fixes for ant and pipeline-maven have been released; as well as the hotfix of this issue as originally filed for ansicolor (PRs 128 & 130), though the full fix through rewrite (PR 132) which is merged but not yet released by dnusbaum (and which also requires a post-LTS version of Jenkins core).

            jglick Jesse Glick added a comment - Thus the fixes for ant and pipeline-maven have been released; as well as the hotfix of this issue as originally filed for ansicolor (PRs 128 & 130), though the full fix through rewrite (PR 132) which is merged but not yet released by dnusbaum (and which also requires a post-LTS version of Jenkins core).
            dnusbaum Devin Nusbaum added a comment -

            jglick I just released the full fix for ansicolor in version 0.6.0 of the plugin.

            dnusbaum Devin Nusbaum added a comment - jglick I just released the full fix for ansicolor in version 0.6.0 of the plugin.
            toisen Konstantin Berndikov added a comment - - edited

            I have reproduced the issue with Jenkins 2.332.1 and AnsiColor 1.0.1. The nodes are started with bash and they are using WebSocket connection if that matters. The node executes jar-file which prints color coded messages in log. Everything is fine on master, but no colors or color escape sequences in the output for nodes.

            Jenkins version: 2.332.1
            JDK: openjdk-11+28
            OS: Oracle Linux Server 7.9

            toisen Konstantin Berndikov added a comment - - edited I have reproduced the issue with  Jenkins 2.332.1  and AnsiColor 1.0.1. The nodes are started with bash and they are using WebSocket connection if that matters. The node executes jar-file which prints color coded messages in log. Everything is fine on master, but no colors or color escape sequences in the output for nodes. Jenkins version: 2.332.1 JDK: openjdk-11+28 OS: Oracle Linux Server 7.9
            jglick Jesse Glick added a comment -

            toisen best to file a fresh issue linked to this one with steps to reproduce from scratch.

            jglick Jesse Glick added a comment - toisen best to file a fresh issue linked to this one with steps to reproduce from scratch.

            People

              jglick Jesse Glick
              bspecht Ben Specht
              Votes:
              3 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: