Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-48344

Log files generated by Jenkins pipeline scripts are bloated

XMLWordPrintable

    • 1.9

      We recently had one of our production Jenkins masters nearly run out of disk space due to an over-allocation of log data by just a couple of jobs running on our farm. In fact, 2 such builds consumed nearly a half a terabyte for their build logs alone! Further, closer examination of those log files revealed that the data therein was being excessively bloated by what appears to be some sort of markup or metadata attached to each line in the build log that looks something like this:

       

      ^[[8mha:////4OsXDMICFQEJVGu5QN07bZyJAnhssncpc0tH8m8uvnrSAAAAaB+LCAAAAAAAAP9b85aBtbiIwTG/KF0vKzUvOzOvODlTryCnNB3I0ivPL8pOy8kv18vKT9JLzs8rzs9J1QuHCgaV5jlDhPzyS1IZIICRiYGhoohBKqM0pTg/D64Hh8ICAFt0h+h/AAAA^[[0m[Pipeline] node

       

      To illustrate the problem I created a super trivial pipeline script as follows:

      node {
  stage('first') {
    echo "hello world"
  }
}

      This simple example produced 5kb worth of build logs! To make matters worse, it appears as though the raw text (ie: excluding the markup) is also duplicated among several other .log files in the same folder as the main build log, causing even further bloat.

      I am creating this issue in the hopes that (a) someone can explain what this extra log metadata is in the main build log of a pipeline build and (b) someone can suggest some way to either eliminate this bloat and superfluous duplication, or to at least offer some way to detect large build logs and perhaps truncate, purge or even prevent them by causing the build to fail. Really anything other than filling up the Jenkins home folder would be preferable.

        1. log
          3 kB
          Kevin Phillips

            jglick Jesse Glick
            leedega Kevin Phillips
            Votes:
            11 Vote for this issue
            Watchers:
            31 Start watching this issue

              Created:
              Updated:
              Resolved: