Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-65564

The 'sh' action in pipeline does not correctly execute shell command

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      There's probably some rational for how shell commands are passed from the pipeline sh action into the actual shell, but I've no clue what it is. I've read a lot of responses to people with similar issues, and I still can't get this command.to work...

      docker images -a | grep '<none>' | awk '{print $3}' | xargs docker rmi

      It should not be this hard. I've tried encasing in double quotes and escaping single quotes, encasing in single quotes and escaping double quotes, escaping the dollar sign, hours worth of combinations. There should be a simple 'take this string as literal, really really, as literal' syntax.

      Someone who knows how this works want to tell me how to put this command, which deletes untagged docker images, into the sh() action in a way which will actually do that?

      All help appreciated.

        Attachments

          Activity

          Hide
          mpokorny Martin added a comment -

          ,docker image prune, Remove unused images

          Show
          mpokorny Martin added a comment - ,docker image prune, Remove unused images
          Hide
          mpokorny Martin added a comment - - edited

          I am not similar with shell scripts. But in my case I will get the list with sh (script : 'docker images -a', returnStdout: true).trim() then can split it in a List<String> and then sh 'docker rmi ' for each item.

          2. Way. Try if the shell script works from some terminal. When it works you need to escape specific groovy characters like $, ' or \

          Show
          mpokorny Martin added a comment - - edited I am not similar with shell scripts. But in my case I will get the list with sh (script : 'docker images -a', returnStdout: true).trim() then can split it in a List<String> and then sh 'docker rmi ' for each item. 2. Way. Try if the shell script works from some terminal. When it works you need to escape specific groovy characters like $, ' or \
          Hide
          dberkman David added a comment -

          The shell script works from a terminal.

          If I don't escape | the command is not sent as a single line.

           

          If I escape all chars  think might be problematic...

          docker images -a | grep '<none>' | awk '{print \$3}' | xargs docker rmi

          ...I get...

          WorkflowScript: 25: unexpected char: '\' @ line 25, column 30.
          sh "docker images -a | grep '\<none\>' | awk '

          {print \$3\}' \| xargs docker rmi"   If I double escape the | ... docker images -a \\| grep '<none>' \\| awk '\{print \$3}

          '
          | xargs docker rmi

          ...I get...

          + docker images -a | grep <none> | awk {print $3} | xargs docker rmi "docker images" requires at most 1 argument.

          ...which is missing the single quotes.

           

          If I also double escape the single quotes...

          docker images -a
          | grep \\'<none>
          '
          | awk
          '{print \$3}
          '
          | xargs docker rmi

          ...I get...

          cannot open none: No such file + docker images -a | grep ' | awk '{print }' | xargs docker rmi

           

          If add double escape for < and >...

          docker images -a
          | grep \\'\\<none\\>
          '
          | awk
          '{print \$3}
          '
          | xargs docker rmi

          ...I get...

          + docker images -a | grep '<none>' | awk '{print }' | xargs docker rmi "docker images" requires at most 1 argument.

           

          If I add a double escape for $...

          docker images -a
          | grep \\'\\<none\\>
          '
          | awk
          '{print \\$3}
          '
          | xargs docker rmi

          ...I get...

          WorkflowScript: 25: illegal string body character after dollar sign;
          solution: either escape a literal dollar sign "\$5" or bracket the value expression "${5}" @ line 25, column 9.
          sh "docker images -a
          | grep \\'\\<none\\>
          '
          | awk
          '{print \\$3}
          '
          | xargs docker rmi"

           

          If I triple escape $...

          docker images -a
          | grep \\'\\<none\\>
          '
          | awk
          '{print \\\$3}
          '
          | xargs docker rmi

          ...I get...

          + docker images -a | grep '<none>' | awk '{print $3}' | xargs docker rmi "docker images" requires at most 1 argument.

           

          And that now looks correct in the log but whatever is actually being passed to the shell is clearly not correct. Whoever designed this clearly did not try it themselves, didn't think about what people do with shell commands, and didn't take into account Groovy and Jenkins var substitution in any way that would make this useable.

           

          What I want is a way to Jenkins should do var substitutions into these clearly marked things, and all the rest of this should be takes as is, as literal, and passed as typed into the shell. And in this case I don't even need Jenkins vars. I have gone through this same pointless exercide starting with single quotes for sh '...', with no better results.

          Show
          dberkman David added a comment - The shell script works from a terminal. If I don't escape | the command is not sent as a single line.   If I escape all chars  think might be problematic... docker images -a | grep '<none>' | awk '{print \$3}' | xargs docker rmi ...I get... WorkflowScript: 25: unexpected char: '\' @ line 25, column 30. sh "docker images -a | grep '\<none\>' | awk ' {print \$3\}' \| xargs docker rmi"   If I double escape the | ... docker images -a \\| grep '<none>' \\| awk '\{print \$3} ' | xargs docker rmi ...I get... + docker images -a | grep <none> | awk {print $3} | xargs docker rmi "docker images" requires at most 1 argument. ...which is missing the single quotes.   If I also double escape the single quotes... docker images -a | grep \\'<none> ' | awk '{print \$3} ' | xargs docker rmi ...I get... cannot open none: No such file + docker images -a | grep ' | awk '{print }' | xargs docker rmi   If add double escape for < and >... docker images -a | grep \\'\\<none\\> ' | awk '{print \$3} ' | xargs docker rmi ...I get... + docker images -a | grep '<none>' | awk '{print }' | xargs docker rmi "docker images" requires at most 1 argument.   If I add a double escape for $... docker images -a | grep \\'\\<none\\> ' | awk '{print \\$3} ' | xargs docker rmi ...I get... WorkflowScript: 25: illegal string body character after dollar sign; solution: either escape a literal dollar sign "\$5" or bracket the value expression "${5}" @ line 25, column 9. sh "docker images -a | grep \\'\\<none\\> ' | awk '{print \\$3} ' | xargs docker rmi"   If I triple escape $... docker images -a | grep \\'\\<none\\> ' | awk '{print \\\$3} ' | xargs docker rmi ...I get... + docker images -a | grep '<none>' | awk '{print $3}' | xargs docker rmi "docker images" requires at most 1 argument.   And that now looks correct in the log but whatever is actually being passed to the shell is clearly not correct. Whoever designed this clearly did not try it themselves, didn't think about what people do with shell commands, and didn't take into account Groovy and Jenkins var substitution in any way that would make this useable.   What I want is a way to Jenkins should do var substitutions into these clearly marked things, and all the rest of this should be takes as is, as literal, and passed as typed into the shell. And in this case I don't even need Jenkins vars. I have gone through this same pointless exercide starting with single quotes for sh '...', with no better results.
          Hide
          mpokorny Martin added a comment -

          Thi has nothing with jenkins or implementation in jenkins. This is just groovy / java syntax.
          https://groovy-lang.org/syntax.html#all-strings

          Show
          mpokorny Martin added a comment - Thi has nothing with jenkins or implementation in jenkins. This is just groovy / java syntax. https://groovy-lang.org/syntax.html#all-strings
          Hide
          dberkman David added a comment -

          I can believe that, but I tried encasing the whole thing in single quotes and it still didn't work. What class in the open source Jenkins project implements the sh() action?

          Show
          dberkman David added a comment - I can believe that, but I tried encasing the whole thing in single quotes and it still didn't work. What class in the open source Jenkins project implements the sh() action?

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            dberkman David
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated: