Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-54346

tee keeps file open

    XMLWordPrintable

Details

    Description

      Using the below pipeline script fails in the second stage and in any subsequent runs until the master or slave isn't restarted.

       

      The error is:
      ERROR: Cannot delete workspace...
       

      The script:

      stage('first') {
        node() {
          cleanWs()

          tee('output.log')

      Unknown macro: {      echo 'test'    }

        }
      }

      stage('second') {
        node()

      Unknown macro: {    cleanWs()     echo 'cleaned'  }

      }

      Marked as critical due to the fact that slave/master restart is required to resolve the issue and allow further runs to wipe the workspace (until tee is used again).

       

      Attachments

        Issue Links

          Activity

            I created a pull request that solves this problem: https://github.com/jenkinsci/pipeline-utility-steps-plugin/pull/56

            roel0 roel postelmans added a comment - I created a pull request that solves this problem: https://github.com/jenkinsci/pipeline-utility-steps-plugin/pull/56

            Probably related to this issue: I had a "too many open files" error in linux, and the lsof command revealed thousands of open file descriptors resulting from Jenkins pipeline tee command.

            Note that tee is executed in parallel tasks in my case.

            A restart of the Jenkins service has temporarily solved the problem.

            dmel Dimitris Meletiou added a comment - Probably related to this issue: I had a "too many open files" error in linux, and the lsof command revealed thousands of open file descriptors resulting from Jenkins pipeline tee command. Note that tee  is executed in parallel tasks in my case. A restart of the Jenkins service has temporarily solved the problem.
            tskrainar Tom Skrainar added a comment -

            We just started using tee() in a workflow, and in less than 24h, one agent leaked enough descriptors to hit the per-process limit.  Use of tee() in this workflow was fundamental to the workflow itself, so this is a huge blocker for us (considering an agent restart is required to resolve).

             

            jglick any update on https://github.com/jenkinsci/pipeline-utility-steps-plugin/pull/62 ? 

            tskrainar Tom Skrainar added a comment - We just started using tee() in a workflow, and in less than 24h, one agent leaked enough descriptors to hit the per-process limit.  Use of tee() in this workflow was fundamental to the workflow itself, so this is a huge blocker for us (considering an agent restart is required to resolve).   jglick any update on https://github.com/jenkinsci/pipeline-utility-steps-plugin/pull/62  ? 
            jglick Jesse Glick added a comment -

            I am neither the author of the patch, nor the maintainer of the plugin.

            There is a trivial workaround for some use cases: replace

            tee('output.txt') {
              sh 'some command'
            }
            

            with

            sh 'some command >output.txt 2>&1'
            

            Of course this is not possible if you were redirecting output from multiple nested steps that could not be coalesced.

            jglick Jesse Glick added a comment - I am neither the author of the patch, nor the maintainer of the plugin. There is a trivial workaround for some use cases: replace tee( 'output.txt' ) { sh 'some command' } with sh 'some command >output.txt 2>&1' Of course this is not possible if you were redirecting output from multiple nested steps that could not be coalesced.

            In our installation(2.176.4) tee on windows nodes works good with workflow-job plugin <=2.25. After upgrade workflow-job  plugin to 2.26+ version job with

             

            node('windows')
            {
            tee {bat ... }
            cleanWs()
            }
            

             

            drops with

            ERROR: Cannot delete workspace: Unable to delete '...'. Tried 3 times...

            alexanderu Alexander Ukhanov added a comment - In our installation(2.176.4) tee on windows nodes works good with workflow-job plugin <=2.25. After upgrade workflow-job  plugin to 2.26+ version job with   node( 'windows' ) { tee {bat ... } cleanWs() }   drops with ERROR: Cannot delete workspace: Unable to delete '...'. Tried 3 times...
            kansasmann Glenn Herbert added a comment -

            Just running into this today. For I workaround i switched to use powershell as follows:

            bat '''
               powershell "cmd /c '.\your.bat' | tee output.txt"
            '''
            kansasmann Glenn Herbert added a comment - Just running into this today. For I workaround i switched to use powershell as follows: bat ''' powershell "cmd /c '.\your.bat' | tee output.txt" '''

            People

              szabi_tolnai Szabolcs Tolnai
              szabi_tolnai Szabolcs Tolnai
              Votes:
              15 Vote for this issue
              Watchers:
              20 Start watching this issue

              Dates

                Created:
                Updated: