Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-46120

OOM: MultiJob and adding value to end of environment variable, eg. path

XMLWordPrintable

      Multijob fails to handle (global) environment variable that is adding value to end of variable eg. for path.

      Finally the issue seems to be "minor" but I lefted it open because Multijob was only "project model" to fail on this.

      Global properties in system preference:

      LD_LIBRARY_PATH
      /opt/oracle/instantclient:$LD_LIBRARY_PATH

       

      This is from end to change build name which also fails for OOM heap.

      Set build name.
13:36:46 FATAL: Java heap space
13:36:46 java.lang.OutOfMemoryError: Java heap space
13:36:46 	at java.util.Arrays.copyOf(Arrays.java:3332)
13:36:46 	at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
13:36:46 	at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448)
13:36:46 	at java.lang.StringBuilder.append(StringBuilder.java:136)
13:36:46 	at hudson.Util.replaceMacro(Util.java:180)
13:36:46 	at hudson.Util.replaceMacro(Util.java:146)
13:36:46 	at hudson.EnvVars.resolve(EnvVars.java:343)
13:36:46 	at hudson.model.AbstractBuild.getEnvironment(AbstractBuild.java:886)
13:36:46 	at org.jenkinsci.plugins.tokenmacro.TokenMacro.expandAll(TokenMacro.java:225)
13:36:46 	at org.jenkinsci.plugins.tokenmacro.TokenMacro.expandAll(TokenMacro.java:216)
13:36:46 	at org.jenkinsci.plugins.tokenmacro.TokenMacro.expandAll(TokenMacro.java:208)
13:36:46 	at org.jenkinsci.plugins.buildnamesetter.BuildNameSetter.setDisplayName(BuildNameSetter.java:74)
13:36:46 	at org.jenkinsci.plugins.buildnamesetter.BuildNameSetter.access$000(BuildNameSetter.java:31)
13:36:46 	at org.jenkinsci.plugins.buildnamesetter.BuildNameSetter$1.tearDown(BuildNameSetter.java:64)
13:36:46 	at hudson.model.Build$BuildExecution.doRun(Build.java:174)
13:36:46 	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:490)
13:36:46 	at com.tikal.jenkins.plugins.multijob.MultiJobBuild$MultiJobRunnerImpl.run(MultiJobBuild.java:136)
13:36:46 	at hudson.model.Run.execute(Run.java:1737)
13:36:46 	at com.tikal.jenkins.plugins.multijob.MultiJobBuild.run(MultiJobBuild.java:73)
13:36:46 	at hudson.model.ResourceController.execute(ResourceController.java:97)
13:36:46 	at hudson.model.Executor.run(Executor.java:419)
13:36:59 Notifying upstream projects of job completion

        1. Biggest objects 2,2GB for MultijobBuild.png
          Biggest objects 2,2GB for MultijobBuild.png
          315 kB
        2. Class based heap usage.png
          Class based heap usage.png
          175 kB
        3. Reachability table of heap dump.png
          Reachability table of heap dump.png
          410 kB
        4. job2.xml
          24 kB
        5. job.xml
          31 kB

          [JENKINS-46120] OOM: MultiJob and adding value to end of environment variable, eg. path

          Heikki Simperi created issue -
          Heikki Simperi made changes -
          Attachment New: job.xml [ 39237 ]
          Heikki Simperi made changes -
          Description Original: Updated jenkins 2.55-1.1 -> 2.73-1.1 and plugins to newest.

           

          Running simple multijob project stop working to OOM Heap. Adding java mx from 5G -> 8G did not help.

           

          The multijob has several build and install jobs which are light and executed fine.

          The next phase is just to call empty jobs to mark up version and it breaks down to OOM heap.
          *13:33:30* [MultiJob] - [ERROR] - Problems occurs on injecting env vars as a build step: Java heap space
           

          This is from end to change build name which also fails for OOM heap.
          {code:java}
          Set build name.
          13:36:46 FATAL: Java heap space
          13:36:46 java.lang.OutOfMemoryError: Java heap space
          13:36:46 at java.util.Arrays.copyOf(Arrays.java:3332)
          13:36:46 at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
          13:36:46 at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448)
          13:36:46 at java.lang.StringBuilder.append(StringBuilder.java:136)
          13:36:46 at hudson.Util.replaceMacro(Util.java:180)
          13:36:46 at hudson.Util.replaceMacro(Util.java:146)
          13:36:46 at hudson.EnvVars.resolve(EnvVars.java:343)
          13:36:46 at hudson.model.AbstractBuild.getEnvironment(AbstractBuild.java:886)
          13:36:46 at org.jenkinsci.plugins.tokenmacro.TokenMacro.expandAll(TokenMacro.java:225)
          13:36:46 at org.jenkinsci.plugins.tokenmacro.TokenMacro.expandAll(TokenMacro.java:216)
          13:36:46 at org.jenkinsci.plugins.tokenmacro.TokenMacro.expandAll(TokenMacro.java:208)
          13:36:46 at org.jenkinsci.plugins.buildnamesetter.BuildNameSetter.setDisplayName(BuildNameSetter.java:74)
          13:36:46 at org.jenkinsci.plugins.buildnamesetter.BuildNameSetter.access$000(BuildNameSetter.java:31)
          13:36:46 at org.jenkinsci.plugins.buildnamesetter.BuildNameSetter$1.tearDown(BuildNameSetter.java:64)
          13:36:46 at hudson.model.Build$BuildExecution.doRun(Build.java:174)
          13:36:46 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:490)
          13:36:46 at com.tikal.jenkins.plugins.multijob.MultiJobBuild$MultiJobRunnerImpl.run(MultiJobBuild.java:136)
          13:36:46 at hudson.model.Run.execute(Run.java:1737)
          13:36:46 at com.tikal.jenkins.plugins.multijob.MultiJobBuild.run(MultiJobBuild.java:73)
          13:36:46 at hudson.model.ResourceController.execute(ResourceController.java:97)
          13:36:46 at hudson.model.Executor.run(Executor.java:419){code}           		New: Updated jenkins 2.55-1.1 -> 2.73-1.1 and plugins to newest.

           

          Running simple multijob project stop working to OOM Heap. Adding java mx from 5G -> 8G did not help.

           

          The multijob has several build and install jobs which are light and executed fine.

          The next phase is just to call empty jobs to mark up version and it breaks down to OOM heap.
           *13:33:30* [MultiJob] - [ERROR] - Problems occurs on injecting env vars as a build step: Java heap space
            

          This is from end to change build name which also fails for OOM heap.
          {code:java}
          Set build name.
          13:36:46 FATAL: Java heap space
          13:36:46 java.lang.OutOfMemoryError: Java heap space
          13:36:46 at java.util.Arrays.copyOf(Arrays.java:3332)
          13:36:46 at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
          13:36:46 at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448)
          13:36:46 at java.lang.StringBuilder.append(StringBuilder.java:136)
          13:36:46 at hudson.Util.replaceMacro(Util.java:180)
          13:36:46 at hudson.Util.replaceMacro(Util.java:146)
          13:36:46 at hudson.EnvVars.resolve(EnvVars.java:343)
          13:36:46 at hudson.model.AbstractBuild.getEnvironment(AbstractBuild.java:886)
          13:36:46 at org.jenkinsci.plugins.tokenmacro.TokenMacro.expandAll(TokenMacro.java:225)
          13:36:46 at org.jenkinsci.plugins.tokenmacro.TokenMacro.expandAll(TokenMacro.java:216)
          13:36:46 at org.jenkinsci.plugins.tokenmacro.TokenMacro.expandAll(TokenMacro.java:208)
          13:36:46 at org.jenkinsci.plugins.buildnamesetter.BuildNameSetter.setDisplayName(BuildNameSetter.java:74)
          13:36:46 at org.jenkinsci.plugins.buildnamesetter.BuildNameSetter.access$000(BuildNameSetter.java:31)
          13:36:46 at org.jenkinsci.plugins.buildnamesetter.BuildNameSetter$1.tearDown(BuildNameSetter.java:64)
          13:36:46 at hudson.model.Build$BuildExecution.doRun(Build.java:174)
          13:36:46 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:490)
          13:36:46 at com.tikal.jenkins.plugins.multijob.MultiJobBuild$MultiJobRunnerImpl.run(MultiJobBuild.java:136)
          13:36:46 at hudson.model.Run.execute(Run.java:1737)
          13:36:46 at com.tikal.jenkins.plugins.multijob.MultiJobBuild.run(MultiJobBuild.java:73)
          13:36:46 at hudson.model.ResourceController.execute(ResourceController.java:97)
          13:36:46 at hudson.model.Executor.run(Executor.java:419)
          13:36:59 Notifying upstream projects of job completion
          {code}
          Heikki Simperi made changes -
          Summary Original: Problems occurs on injecting env vars as a build step: Java heap space New: OOM: Problems occurs on injecting env vars as a build step: Java heap space
          Heikki Simperi made changes -
          Summary Original: OOM: Problems occurs on injecting env vars as a build step: Java heap space New: OOM: MultiJob Problems occurs on injecting env vars as a build step: Java heap space
          Heikki Simperi made changes -
          Attachment New: job2.xml [ 39245 ]

          Heikki Simperi added a comment - - edited

          Added as attachment second plain multijob project xml file with same result. This is console printout:

           

          12:56:55 Started by user Name
12:56:55 [EnvInject] - Loading node environment variables.
12:56:55 Building remotely on hara mini (REMOTE_MINI_JOBS JENKINS_MINI) in workspace /mnt/jenkins/mini/workspace/31-PROJ-CUST-deploy-all
12:56:55 [WS-CLEANUP] Deleting project workspace...
12:56:55 [WS-CLEANUP] Done
12:56:55 Copied 2 artifacts from "00-PROJ-build" build number 7957
12:56:55 [EnvInject] - Injecting environment variables from a build step.
12:56:55 [EnvInject] - Injecting as environment variables the properties file path 'version.properties'
12:56:55 [EnvInject] - Variables injected successfully.
12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
12:56:55 Starting build job 31-PROJ-CUST-deploy.
12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
12:56:55 Starting build job 31-PROJ-CUST-deploy.
12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
12:56:55 Starting build job 31-PROJ-CUST-deploy.
12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
12:56:55 Starting build job 31-PROJ-CUST-deploy.
12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
12:56:55 Starting build job 31-PROJ-CUST-deploy.
12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
12:56:55 Starting build job 31-PROJ-CUST-deploy.
12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
12:56:55 Starting build job 31-PROJ-CUST-deploy.
12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
12:56:55 Starting build job 31-PROJ-CUST-deploy.
12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
12:56:55 Starting build job 31-PROJ-CUST-deploy.
12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
12:56:56 Starting build job 31-PROJ-CUST-deploy.
12:56:56 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
12:56:57 Starting build job 31-PROJ-CUST-deploy.
12:56:57 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
12:57:00 Starting build job 31-PROJ-CUST-deploy.
12:57:01 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
12:57:10 Starting build job 31-PROJ-CUST-deploy.
12:57:11 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
12:57:40 Starting build job 31-PROJ-CUST-deploy.
12:57:44 >> Job status: Warning: [31-PROJ-CUST-deploy] subjob in the phase configuration is disabled.
12:59:17 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
12:59:29 [MultiJob] - [ERROR] - Problems occurs on injecting env vars as a build step: Java heap space
12:59:29 Starting build job 31-PROJ-CUST-deploy.
12:59:39 FATAL: Java heap space
12:59:39 java.lang.OutOfMemoryError: Java heap space
12:59:40 Notifying upstream projects of job completion
12:59:40 Finished: FAILURE

          Heikki Simperi added a comment - - edited Added as attachment second plain multijob project xml file with same result. This is console printout:   12:56:55 Started by user Name 12:56:55 [EnvInject] - Loading node environment variables. 12:56:55 Building remotely on hara mini (REMOTE_MINI_JOBS JENKINS_MINI) in workspace /mnt/jenkins/mini/workspace/31-PROJ-CUST-deploy-all 12:56:55 [WS-CLEANUP] Deleting project workspace... 12:56:55 [WS-CLEANUP] Done 12:56:55 Copied 2 artifacts from "00-PROJ-build" build number 7957 12:56:55 [EnvInject] - Injecting environment variables from a build step. 12:56:55 [EnvInject] - Injecting as environment variables the properties file path 'version.properties' 12:56:55 [EnvInject] - Variables injected successfully. 12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:56:55 Starting build job 31-PROJ-CUST-deploy. 12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:56:55 Starting build job 31-PROJ-CUST-deploy. 12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:56:55 Starting build job 31-PROJ-CUST-deploy. 12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:56:55 Starting build job 31-PROJ-CUST-deploy. 12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:56:55 Starting build job 31-PROJ-CUST-deploy. 12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:56:55 Starting build job 31-PROJ-CUST-deploy. 12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:56:55 Starting build job 31-PROJ-CUST-deploy. 12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:56:55 Starting build job 31-PROJ-CUST-deploy. 12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:56:55 Starting build job 31-PROJ-CUST-deploy. 12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:56:56 Starting build job 31-PROJ-CUST-deploy. 12:56:56 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:56:57 Starting build job 31-PROJ-CUST-deploy. 12:56:57 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:57:00 Starting build job 31-PROJ-CUST-deploy. 12:57:01 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:57:10 Starting build job 31-PROJ-CUST-deploy. 12:57:11 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:57:40 Starting build job 31-PROJ-CUST-deploy. 12:57:44 >> Job status: Warning: [31-PROJ-CUST-deploy] subjob in the phase configuration is disabled. 12:59:17 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:59:29 [MultiJob] - [ERROR] - Problems occurs on injecting env vars as a build step: Java heap space 12:59:29 Starting build job 31-PROJ-CUST-deploy. 12:59:39 FATAL: Java heap space 12:59:39 java.lang.OutOfMemoryError: Java heap space 12:59:40 Notifying upstream projects of job completion 12:59:40 Finished: FAILURE
          Heikki Simperi made changes -
          Component/s New: envinject-plugin [ 15893 ]
          Heikki Simperi made changes -
          Link New: This issue relates to JENKINS-38850 [ JENKINS-38850 ]
          Heikki Simperi made changes -
          Description Original: Updated jenkins 2.55-1.1 -> 2.73-1.1 and plugins to newest.

           

          Running simple multijob project stop working to OOM Heap. Adding java mx from 5G -> 8G did not help.

           

          The multijob has several build and install jobs which are light and executed fine.

          The next phase is just to call empty jobs to mark up version and it breaks down to OOM heap.
           *13:33:30* [MultiJob] - [ERROR] - Problems occurs on injecting env vars as a build step: Java heap space
            

          This is from end to change build name which also fails for OOM heap.
          {code:java}
          Set build name.
          13:36:46 FATAL: Java heap space
          13:36:46 java.lang.OutOfMemoryError: Java heap space
          13:36:46 at java.util.Arrays.copyOf(Arrays.java:3332)
          13:36:46 at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
          13:36:46 at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448)
          13:36:46 at java.lang.StringBuilder.append(StringBuilder.java:136)
          13:36:46 at hudson.Util.replaceMacro(Util.java:180)
          13:36:46 at hudson.Util.replaceMacro(Util.java:146)
          13:36:46 at hudson.EnvVars.resolve(EnvVars.java:343)
          13:36:46 at hudson.model.AbstractBuild.getEnvironment(AbstractBuild.java:886)
          13:36:46 at org.jenkinsci.plugins.tokenmacro.TokenMacro.expandAll(TokenMacro.java:225)
          13:36:46 at org.jenkinsci.plugins.tokenmacro.TokenMacro.expandAll(TokenMacro.java:216)
          13:36:46 at org.jenkinsci.plugins.tokenmacro.TokenMacro.expandAll(TokenMacro.java:208)
          13:36:46 at org.jenkinsci.plugins.buildnamesetter.BuildNameSetter.setDisplayName(BuildNameSetter.java:74)
          13:36:46 at org.jenkinsci.plugins.buildnamesetter.BuildNameSetter.access$000(BuildNameSetter.java:31)
          13:36:46 at org.jenkinsci.plugins.buildnamesetter.BuildNameSetter$1.tearDown(BuildNameSetter.java:64)
          13:36:46 at hudson.model.Build$BuildExecution.doRun(Build.java:174)
          13:36:46 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:490)
          13:36:46 at com.tikal.jenkins.plugins.multijob.MultiJobBuild$MultiJobRunnerImpl.run(MultiJobBuild.java:136)
          13:36:46 at hudson.model.Run.execute(Run.java:1737)
          13:36:46 at com.tikal.jenkins.plugins.multijob.MultiJobBuild.run(MultiJobBuild.java:73)
          13:36:46 at hudson.model.ResourceController.execute(ResourceController.java:97)
          13:36:46 at hudson.model.Executor.run(Executor.java:419)
          13:36:59 Notifying upstream projects of job completion
          {code}           		New: Updated jenkins 2.55-1.1 -> 2.73-1.1 and plugins to newest.
          Multijob plugin was updated 1.24 -> 1.27

           

          Running simple multijob project stop working to OOM Heap. Adding java mx from 5G -> 8G did not help.

           

          The multijob has several build and install jobs which are light and executed fine.

          The next phase is just to call empty jobs to mark up version and it breaks down to OOM heap.
           *13:33:30* [MultiJob] - [ERROR] - Problems occurs on injecting env vars as a build step: Java heap space
            

          This is from end to change build name which also fails for OOM heap.
          {code:java}
          Set build name.
          13:36:46 FATAL: Java heap space
          13:36:46 java.lang.OutOfMemoryError: Java heap space
          13:36:46 at java.util.Arrays.copyOf(Arrays.java:3332)
          13:36:46 at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
          13:36:46 at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448)
          13:36:46 at java.lang.StringBuilder.append(StringBuilder.java:136)
          13:36:46 at hudson.Util.replaceMacro(Util.java:180)
          13:36:46 at hudson.Util.replaceMacro(Util.java:146)
          13:36:46 at hudson.EnvVars.resolve(EnvVars.java:343)
          13:36:46 at hudson.model.AbstractBuild.getEnvironment(AbstractBuild.java:886)
          13:36:46 at org.jenkinsci.plugins.tokenmacro.TokenMacro.expandAll(TokenMacro.java:225)
          13:36:46 at org.jenkinsci.plugins.tokenmacro.TokenMacro.expandAll(TokenMacro.java:216)
          13:36:46 at org.jenkinsci.plugins.tokenmacro.TokenMacro.expandAll(TokenMacro.java:208)
          13:36:46 at org.jenkinsci.plugins.buildnamesetter.BuildNameSetter.setDisplayName(BuildNameSetter.java:74)
          13:36:46 at org.jenkinsci.plugins.buildnamesetter.BuildNameSetter.access$000(BuildNameSetter.java:31)
          13:36:46 at org.jenkinsci.plugins.buildnamesetter.BuildNameSetter$1.tearDown(BuildNameSetter.java:64)
          13:36:46 at hudson.model.Build$BuildExecution.doRun(Build.java:174)
          13:36:46 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:490)
          13:36:46 at com.tikal.jenkins.plugins.multijob.MultiJobBuild$MultiJobRunnerImpl.run(MultiJobBuild.java:136)
          13:36:46 at hudson.model.Run.execute(Run.java:1737)
          13:36:46 at com.tikal.jenkins.plugins.multijob.MultiJobBuild.run(MultiJobBuild.java:73)
          13:36:46 at hudson.model.ResourceController.execute(ResourceController.java:97)
          13:36:46 at hudson.model.Executor.run(Executor.java:419)
          13:36:59 Notifying upstream projects of job completion
          {code}

            Unassigned Unassigned
            heikkisi Heikki Simperi
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: