Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-52813

java.lang.OutOfMemoryError

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Blocker
    • Resolution: Not A Defect
    • Component/s: _unsorted
    • Environment:
      installed Jenkins on Tomcat.
      Jenkins ver. 2.32.1
      Apache Tomcat/7.0.67
      os.version AIX 7.1
      java.version 1.8.0_171
    • Similar Issues:

      Description

      Recently our Jenkins: http://${Jenkins_HOST}:${jenkins_PORT}/jenkins/  is suddenly not reachable frequently . After check the cataline.log, we find below information, do you know how to resolve this issue

      Log:

      Jul 26, 2018 3:30:10 PM hudson.triggers.Trigger checkTriggers
      WARNING: hudson.triggers.TimerTrigger.run() failed for hudson.model.FreeStyleProject@1ceb3922[Test_update_user_profile]
      java.lang.OutOfMemoryError: Failed to create a thread: retVal -1073741830, errno 11
      at java.lang.Thread.startImpl(Native Method)
      at java.lang.Thread.start(Thread.java:981)
      at hudson.remoting.AtmostOneThreadExecutor.execute(AtmostOneThreadExecutor.java:94)
      at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:145)
      at jenkins.util.AtmostOneTaskExecutor.maybeRun(AtmostOneTaskExecutor.java:91)
      at jenkins.util.AtmostOneTaskExecutor.submit(AtmostOneTaskExecutor.java:79)
      at hudson.model.Queue.scheduleMaintenance(Queue.java:1140)
      at hudson.model.Queue.scheduleInternal(Queue.java:605)
      at hudson.model.Queue.schedule2(Queue.java:560)
      at jenkins.model.ParameterizedJobMixIn.scheduleBuild2(ParameterizedJobMixIn.java:138)
      at jenkins.model.ParameterizedJobMixIn.scheduleBuild(ParameterizedJobMixIn.java:94)
      at hudson.model.AbstractProject.scheduleBuild(AbstractProject.java:837)
      at hudson.triggers.TimerTrigger.run(TimerTrigger.java:66)
      at hudson.triggers.Trigger.checkTriggers(Trigger.java:278)
      at hudson.triggers.Trigger$Cron.doRun(Trigger.java:226)
      at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:50)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:522)
      at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:319)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:191)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
      at java.lang.Thread.run(Thread.java:811)

       

      Our jenkins install on server1, and connect agent on server2 via JNLP, all scheduler job are on server2, [Test_update_user_profile] is one of our job on server2.

      Start jenkins on server1 with below JAVA_OPTS

      JAVA_OPTS="-Xmx2048m"
      exec "$_RUNJDB" "$LOGGING_CONFIG" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \
      exec "$_RUNJDB" "$LOGGING_CONFIG" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \
      eval exec "\"$_RUNJAVA\"" "\"$LOGGING_CONFIG\"" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \
      eval exec "\"$_RUNJAVA\"" "\"$LOGGING_CONFIG\"" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \
      eval "\"$_RUNJAVA\"" "\"$LOGGING_CONFIG\"" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \
      eval "\"$_RUNJAVA\"" "\"$LOGGING_CONFIG\"" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \
      eval "\"$_RUNJAVA\"" $LOGGING_MANAGER $JAVA_OPTS \
      eval "\"$_RUNJAVA\"" $LOGGING_MANAGER $JAVA_OPTS \

        Attachments

          Activity

          Hide
          jessica_gao Jessica Gao added a comment -

          hope the above 3 analysis pictures can be useful

          Show
          jessica_gao Jessica Gao added a comment - hope the above 3 analysis pictures can be useful
          Hide
          jessica_gao Jessica Gao added a comment -

          we have 7 agents

          Show
          jessica_gao Jessica Gao added a comment - we have 7 agents
          Hide
          jessica_gao Jessica Gao added a comment -

          Show
          jessica_gao Jessica Gao added a comment -
          Hide
          oleg_nenashev Oleg Nenashev added a comment -

          For 7 agents it is fine to consume 66 Megabytes of memory (caches, etc.), I do not see an issue there.

          If you suspect a memory leak, please record heap dumps for the timeframe when OutOfMemory really happens. If it proves that the agent connections leak memory, I really advise updating to the recent versions of Jenkins. There were several important changes in error handling there, so it does not make much sense to spend more time on investigation.

           

          Show
          oleg_nenashev Oleg Nenashev added a comment - For 7 agents it is fine to consume 66 Megabytes of memory (caches, etc.), I do not see an issue there. If you suspect a memory leak, please record heap dumps for the timeframe when OutOfMemory really happens. If it proves that the agent connections leak memory, I really advise updating to the recent versions of Jenkins. There were several important changes in error handling there, so it does not make much sense to spend more time on investigation.  
          Hide
          oleg_nenashev Oleg Nenashev added a comment -

          No response from the reporter

          Show
          oleg_nenashev Oleg Nenashev added a comment - No response from the reporter

            People

            Assignee:
            oleg_nenashev Oleg Nenashev
            Reporter:
            jessica_gao Jessica Gao
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: