• Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • workflow-cps-plugin
    • None

      As with JENKINS-26481 the eachLine-method on String only works for the first iteration.

      Using version 3.39

      We use a cleanup-pipeline to find all running docker-containers without a corresponding feature-branch (deleted after merge).

      stage('clean') {
       def branches = []
       // extract available branches from git
       sh (returnStdout: true, script: "ssh-agent bash -c 'ssh-add /var/lib/jenkins/.ssh/id_rsa &>/dev/null; git ls-remote --heads --refs ssh://git@myrepo/project.git' | cut -f 2")
        .eachLine { branches << it }
       // extract all containers (including stopped)
       def containers = []
       sh (returnStdout: true, script: "docker ps -a --format '{{.Names}}' --filter name=project")
        .trim()
        .eachLine { containers << it }
       println(containers) // <---- only prints first container
       println(branches) // <---- only prints first branch
      
       //stop containers for non existing branches...
       containers.each{ containername ->
        if(branches.findAll({branch -> branch.contains(containername)}).isEmpty()){
         println("trying to stop ${containername}")
         //sh ("docker stop ${containername} || true") // container might already be stopped
         println("removing ${containername}")
         //sh ("docker rm ${containername}")
        }
       }
      }

      As a workaround replace eachLine with split('\n').each { ... }

          [JENKINS-46988] String.eachLine only reads first line

          I'd like to affirm the previous comment.  I just did exactly that – I spent a couple of hours trying to figure out why my code wasn't working, and ended up here when I realized it was a bug in the Jenkins groovy interpreter for scripted pipelines.

          Andrew Lawrence added a comment - I'd like to affirm the previous comment.  I just did exactly that – I spent a couple of hours trying to figure out why my code wasn't working, and ended up here when I realized it was a bug in the Jenkins groovy interpreter for scripted pipelines.

          Likewise. Just burned a bit of time on this. Moving to the workaround.

          Dominique Thornton added a comment - Likewise. Just burned a bit of time on this. Moving to the workaround.

          I've just stumbled over this as well.

          Piotr Kożuchowski added a comment - I've just stumbled over this as well.

          Tony Hoyle added a comment -

          Same here.  What's worse is the error message links to a page https://jenkins.io/redirect/pipeline-cps-method-mismatches/ which has absolutely no hints on what the error could be, and assumes you know the internal workings of the groovy compiler.  No, I just want a simple and obvious construct to work..

           

           

           

          Tony Hoyle added a comment - Same here.  What's worse is the error message links to a page https://jenkins.io/redirect/pipeline-cps-method-mismatches/  which has absolutely no hints on what the error could be, and assumes you know the internal workings of the groovy compiler.  No, I just want a simple and obvious construct to work..      

          Affects us as well

          Nancy Robertson added a comment - Affects us as well

          Also wasted time one this issue, but thanks to this bug report, quickly switched to workaround.

          Dmitry Bigunyak added a comment - Also wasted time one this issue, but thanks to this bug report, quickly switched to workaround.

          Brian Minton added a comment - - edited

          Confirming that this is still happening as of Jenkins 2.319.1.  I agree with tonyhoylerps that the CPS mismatch link is not helpful.

          Brian Minton added a comment - - edited Confirming that this is still happening as of Jenkins 2.319.1.  I agree with tonyhoylerps that the CPS mismatch link is not helpful.

          Mark Waite added a comment -

          The Jenkins project recommends that these types of complex operations be performed with an sh step that calls the program of your choice. You can use groovy in that sh step or whatever other language you prefer. By placing the complex operation in an sh step, it can be executed on an agent instead of on the controller. By placing the complex operation in an sh step, it can use whatever language and version you prefer.

          Mark Waite added a comment - The Jenkins project recommends that these types of complex operations be performed with an sh step that calls the program of your choice. You can use groovy in that sh step or whatever other language you prefer. By placing the complex operation in an sh step, it can be executed on an agent instead of on the controller. By placing the complex operation in an sh step, it can use whatever language and version you prefer.

          Jo Rhett added a comment -

          The Jenkins project recommends that these types of complex operations be performed with an sh step that calls the program of your choice.

          Thank you for an utterly irrelevant copypasta response. The example shown is an sh step that calls the program of their choice, Mark. So your post just wasted everyone's time because it's not relevant or helpful in any way.

          Jo Rhett added a comment - The Jenkins project recommends that these types of complex operations be performed with an sh step that calls the program of your choice. Thank you for an utterly irrelevant copypasta response. The example shown is an sh step that calls the program of their choice, Mark. So your post just wasted everyone's time because it's not relevant or helpful in any way.

          Mark Waite added a comment -

          jorhett I don't think you understood the message that I was trying to convey.

          The script provided in the bug description uses the Jenkins agent to perform an sh call to list the branches and uses the Jenkins agent to perform an sh call to list the containers. It then uses the Jenkins controller to iterate over the list of containers and the list of branches to decide which containers should be stopped. Then it calls sh to use the Jenkins agent to stop the containers and remove them.

          I think it is simpler to develop, diagnose, and debug if that entire sequence of operations (list branches, list containers, choose containers to stop, stop containers) is performed on the agent inside an sh step that calls a script that runs entirely on the agent. The script that I wrote to do that type of operation in Python looks like this:

          #!/usr/bin/python3
          
          import subprocess
          import sys
          
          #-----------------------------------------------------------------------
          
          def get_branches():
              ls_remote_lines = subprocess.check_output(['git', 'ls-remote', '--heads', '--refs', '--quiet']).decode('utf-8').split('\n')
              branches = []
              for line in ls_remote_lines:
                  if 'refs/' in line:
                      sha1, branch_name = line.split('\t')
                      branches.append(branch_name.replace('refs/heads/', ''))
              return branches
          
          #-----------------------------------------------------------------------
          
          def get_containers():
              containers_lines = subprocess.check_output(['docker', 'ps', '-a', '--format', '{{.Names}}']).decode('utf-8').split('\n')
              return containers_lines[:-1] # discard trailing empty line
          
          #-----------------------------------------------------------------------
          
          def stop_container(container):
              print("docker stop " + container)
          
          #-----------------------------------------------------------------------
          
          def cleanup_pipeline(args = []):
              branches = get_branches()
              # print('==== Branches ====')
              # for branch in branches:
              #     print(branch)
          
              containers = get_containers()
              # print('==== Containers ====')
              # for container in containers:
              #     print(container)
          
              # Stop containers for non-existent branches
              for container in containers:
                  found = False
                  for branch in branches:
                      if branch in container:
                          found = True
                  if not found:
                      stop_container(container)
          
          #-----------------------------------------------------------------------
          
          if __name__ == "__main__": cleanup_pipeline(sys.argv[1:])
          

          The Jenkins project recommends keeping Pipeline scripts simple and pushing as much logic as is feasible into scripts that run on the agent. See the "How to use Jenkins less" talk from Jesse Glick and the "Use just enough pipeline" blog post by Darin Pope and me.

          I truly was trying to provide useful information with my comment. My apologies if it was not received as such.

          Mark Waite added a comment - jorhett I don't think you understood the message that I was trying to convey. The script provided in the bug description uses the Jenkins agent to perform an sh call to list the branches and uses the Jenkins agent to perform an sh call to list the containers. It then uses the Jenkins controller to iterate over the list of containers and the list of branches to decide which containers should be stopped. Then it calls sh to use the Jenkins agent to stop the containers and remove them. I think it is simpler to develop, diagnose, and debug if that entire sequence of operations (list branches, list containers, choose containers to stop, stop containers) is performed on the agent inside an sh step that calls a script that runs entirely on the agent. The script that I wrote to do that type of operation in Python looks like this: #!/usr/bin/python3 import subprocess import sys #----------------------------------------------------------------------- def get_branches(): ls_remote_lines = subprocess.check_output(['git', 'ls-remote', '--heads', '--refs', '--quiet']).decode('utf-8').split('\n') branches = [] for line in ls_remote_lines: if 'refs/' in line: sha1, branch_name = line.split('\t') branches.append(branch_name.replace('refs/heads/', '')) return branches #----------------------------------------------------------------------- def get_containers(): containers_lines = subprocess.check_output(['docker', 'ps', '-a', '--format', '{{.Names}}']).decode('utf-8').split('\n') return containers_lines[:-1] # discard trailing empty line #----------------------------------------------------------------------- def stop_container(container): print("docker stop " + container) #----------------------------------------------------------------------- def cleanup_pipeline(args = []): branches = get_branches() # print('==== Branches ====') # for branch in branches: # print(branch) containers = get_containers() # print('==== Containers ====') # for container in containers: # print(container) # Stop containers for non-existent branches for container in containers: found = False for branch in branches: if branch in container: found = True if not found: stop_container(container) #----------------------------------------------------------------------- if __name__ == "__main__": cleanup_pipeline(sys.argv[1:]) The Jenkins project recommends keeping Pipeline scripts simple and pushing as much logic as is feasible into scripts that run on the agent. See the "How to use Jenkins less" talk from Jesse Glick and the "Use just enough pipeline" blog post by Darin Pope and me. I truly was trying to provide useful information with my comment. My apologies if it was not received as such.

            Unassigned Unassigned
            lostiniceland Marc Schlegel
            Votes:
            32 Vote for this issue
            Watchers:
            26 Start watching this issue

              Created:
              Updated: