Recently our Jenkins: http://${Jenkins_HOST}:${jenkins_PORT}/jenkins/  is suddenly not reachable frequently . After check the cataline.log, we find below information, do you know how to resolve this issue

Log:

Jul 26, 2018 3:30:10 PM hudson.triggers.Trigger checkTriggers
WARNING: hudson.triggers.TimerTrigger.run() failed for hudson.model.FreeStyleProject@1ceb3922[Test_update_user_profile]
java.lang.OutOfMemoryError: Failed to create a thread: retVal -1073741830, errno 11
at java.lang.Thread.startImpl(Native Method)
at java.lang.Thread.start(Thread.java:981)
at hudson.remoting.AtmostOneThreadExecutor.execute(AtmostOneThreadExecutor.java:94)
at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:145)
at jenkins.util.AtmostOneTaskExecutor.maybeRun(AtmostOneTaskExecutor.java:91)
at jenkins.util.AtmostOneTaskExecutor.submit(AtmostOneTaskExecutor.java:79)
at hudson.model.Queue.scheduleMaintenance(Queue.java:1140)
at hudson.model.Queue.scheduleInternal(Queue.java:605)
at hudson.model.Queue.schedule2(Queue.java:560)
at jenkins.model.ParameterizedJobMixIn.scheduleBuild2(ParameterizedJobMixIn.java:138)
at jenkins.model.ParameterizedJobMixIn.scheduleBuild(ParameterizedJobMixIn.java:94)
at hudson.model.AbstractProject.scheduleBuild(AbstractProject.java:837)
at hudson.triggers.TimerTrigger.run(TimerTrigger.java:66)
at hudson.triggers.Trigger.checkTriggers(Trigger.java:278)
at hudson.triggers.Trigger$Cron.doRun(Trigger.java:226)
at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:50)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:522)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:319)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:191)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.lang.Thread.run(Thread.java:811)

Our jenkins install on server1, and connect agent on server2 via JNLP, all scheduler job are on server2, [Test_update_user_profile] is one of our job on server2.

Start jenkins on server1 with below JAVA_OPTS

JAVA_OPTS="-Xmx2048m"
exec "$_RUNJDB" "$LOGGING_CONFIG" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \
exec "$_RUNJDB" "$LOGGING_CONFIG" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \
eval exec "\"$_RUNJAVA\"" "\"$LOGGING_CONFIG\"" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \
eval exec "\"$_RUNJAVA\"" "\"$LOGGING_CONFIG\"" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \
eval "\"$_RUNJAVA\"" "\"$LOGGING_CONFIG\"" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \
eval "\"$_RUNJAVA\"" "\"$LOGGING_CONFIG\"" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \
eval "\"$_RUNJAVA\"" $LOGGING_MANAGER $JAVA_OPTS \
eval "\"$_RUNJAVA\"" $LOGGING_MANAGER $JAVA_OPTS \