Loading...

This issue is archived. You can view it, but you can't modify it. Learn more

XML

Word

Printable

Type: Bug
Resolution: Fixed
Priority: Minor
Component/s: ssh-agent-plugin
Labels:
- issue-exported-to-github
Environment:
CentOS 6.6, Java OpenJDK 1.7.0_75, Jenkins 1.596.1 (LTS), SSH Agent Plugin 1.5

We had an error on a job today that indicated "too many open files" ...

Started by upstream project "PRODUCTION - LSTORE Image Sync" build number 9020
originally caused by:
 Started by timer
Building on master in workspace <https://ft-itjenkins:9443/job/PRODUCTION%20-%20LSTORE%20Image%20Sync/server=ft-lstore-02/ws/>
[ssh-agent] Using credentials wcontent (FT wcontent key)
[ssh-agent] Looking for ssh-agent implementation...
[ssh-agent]   Java/JNR ssh-agent
[ssh-agent] Started.
[ft-lstore-02] $ /bin/sh -xe /var/run/jenkins/hudson5336150442689398855.sh
FATAL: command execution failed
java.io.IOException: Cannot run program "/bin/sh" (in directory "<https://ft-itjenkins:9443/job/PRODUCTION%20-%20LSTORE%20Image%20Sync/server=ft-lstore-02/ws/")>: error=24, Too many open files
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047)
        at hudson.Proc$LocalProc.<init>(Proc.java:244)
        at hudson.Proc$LocalProc.<init>(Proc.java:216)
        at hudson.Launcher$LocalLauncher.launch(Launcher.java:803)
        at hudson.Launcher$ProcStarter.start(Launcher.java:381)
        at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:97)
        at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66)
        at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
        at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:770)
        at hudson.model.Build$BuildExecution.build(Build.java:199)
        at hudson.model.Build$BuildExecution.doRun(Build.java:160)
        at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:533)
        at hudson.model.Run.execute(Run.java:1759)
        at hudson.matrix.MatrixRun.run(MatrixRun.java:146)
        at hudson.model.ResourceController.execute(ResourceController.java:89)
        at hudson.model.Executor.run(Executor.java:240)
Caused by: java.io.IOException: error=24, Too many open files
        at java.lang.UNIXProcess.forkAndExec(Native Method)
        at java.lang.UNIXProcess.<init>(UNIXProcess.java:186)
        at java.lang.ProcessImpl.start(ProcessImpl.java:130)
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)
        ... 15 more
Build step 'Execute shell' marked build as failure
[ssh-agent] Stopped.

Taking a look at the open files (below), it would appear that the problem is these `/var/run/jenkins/jenkinsXXXXXXXXX.jnr` files. The only indication I have as to where they might be coming from is that the SSH Agent plugin mentions jnr-sockets in their change log. Also our other Jenkins server that doesn't run SSH Agent doesn't have this problem.

[root@ft-itjenkins-02 ~]# lsof -p 31315
COMMAND   PID    USER   FD   TYPE             DEVICE SIZE/OFF      NODE NAME
java    31315 jenkins  cwd    DIR              253,0     4096         2 /
java    31315 jenkins  rtd    DIR              253,0     4096         2 /
java    31315 jenkins  txt    REG              253,0     5328   1578119 /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.75.x86_64/jre/bin/java
java    31315 jenkins  mem    REG              253,2  5206312    524439 /var/lib/sss/mc/group
java    31315 jenkins  mem    REG              253,2    90382    393625 /var/run/jenkins/jffi3000279957673179858.tmp
java    31315 jenkins  mem    REG              253,2    90382    393626 /var/run/jenkins/jffi8429571565719302524.tmp
java    31315 jenkins  mem    REG              253,0  1140496   1048697 /lib64/libglib-2.0.so.0.2800.8
java    31315 jenkins  mem    REG              253,0    17536   1048992 /lib64/libgthread-2.0.so.0.2800.8
java    31315 jenkins  mem    REG              253,0   308848   1048991 /lib64/libgobject-2.0.so.0.2800.8
java    31315 jenkins  mem    REG              253,0   142312   1181912 /usr/lib64/libdbus-glib-1.so.2.1.0
java    31315 jenkins  mem    REG              253,0    10075   1578193 /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.75.x86_64/jre/lib/ext/dnsns.jar
java    31315 jenkins  mem    REG              253,0    16336   1048969 /lib64/security/pam_fprintd.so
java    31315 jenkins  mem    REG              253,0   141768   1048589 /lib64/libaudit.so.1.0.0
java    31315 jenkins  mem    REG              253,0    26060   1180599 /usr/lib64/gconv/gconv-modules.cache
java    31315 jenkins  DEL    REG              253,1                 13 /tmp/ffi2Xmdwb
java    31315 jenkins  mem    REG              253,0    55848   1048706 /lib64/libpam.so.0.82.2
java    31315 jenkins  DEL    REG              253,2             393621 /var/run/jenkins/jna--1712433994/jna7692078112119351647.tmp
java    31315 jenkins  mem    REG              147,0    11027 140744890 /var/lib/jenkins/plugins/build-user-vars-plugin/WEB-INF/lib/classes.jar
java    31315 jenkins  mem    REG              147,0   207723 134320415 /var/lib/jenkins/plugins/build-user-vars-plugin/WEB-INF/lib/commons-lang-2.1.jar
java    31315 jenkins  mem    REG              147,0    25139 134325970 /var/lib/jenkins/plugins/nested-view/WEB-INF/lib/classes.jar
java    31315 jenkins  mem    REG              147,0    18190 201414945 /var/lib/jenkins/plugins/scp/WEB-INF/lib/classes.jar
java    31315 jenkins  mem    REG              147,0   185746 201414944 /var/lib/jenkins/plugins/scp/WEB-INF/lib/jsch-0.1.42.jar
java    31315 jenkins  mem    REG              147,0     5434 201414941 /var/lib/jenkins/plugins/git-chooser-alternative/WEB-INF/lib/classes.jar
java    31315 jenkins  mem    REG              147,0    94728   1999304 /var/lib/jenkins/plugins/view-job-filters/WEB-INF/lib/classes.jar
java    31315 jenkins  mem    REG              147,0   213781   1999303 /var/lib/jenkins/plugins/view-job-filters/WEB-INF/lib/jsch-0.1.44-1.jar
java    31315 jenkins  mem    REG              147,0   293671  67608667 /var/lib/jenkins/plugins/email-ext/WEB-INF/lib/jsoup-1.7.2.jar
java    31315 jenkins  mem    REG              147,0    10485    105053 /var/lib/jenkins/plugins/greenballs/WEB-INF/lib/classes.jar
java    31315 jenkins  mem    REG              147,0   280949  67162590 /var/lib/jenkins/plugins/email-ext/WEB-INF/lib/email-ext.jar
java    31315 jenkins  mem    REG              147,0    15322  67162576 /var/lib/jenkins/plugins/email-ext/WEB-INF/lib/findbugs-annotations-1.3.9-1.jar
java    31315 jenkins  mem    REG              147,0  6139152  67162495 /var/lib/jenkins/plugins/email-ext/WEB-INF/lib/groovy-all-1.8.5.jar
java    31315 jenkins  mem    REG              147,0    76423  23555243 /var/lib/jenkins/plugins/script-security/WEB-INF/lib/classes.jar
java    31315 jenkins  mem    REG              147,0    87571  23555242 /var/lib/jenkins/plugins/script-security/WEB-INF/lib/groovy-sandbox-1.8.jar
java    31315 jenkins  mem    REG              147,0    30360  67162500 /var/lib/jenkins/plugins/jobConfigHistory/WEB-INF/lib/diffutils-1.2.1.jar
java    31315 jenkins  mem    REG              147,0    29516 134320383 /var/lib/jenkins/plugins/ssh-agent/WEB-INF/lib/classes.jar
java    31315 jenkins  mem    REG              147,0   644786 134320382 /var/lib/jenkins/plugins/ssh-agent/WEB-INF/lib/mina-core-2.0.7.jar
java    31315 jenkins  mem    REG              147,0  1997327 134320381 /var/lib/jenkins/plugins/ssh-agent/WEB-INF/lib/bcprov-jdk15on-1.47.jar
java    31315 jenkins  mem    REG              147,0     8845 134320380 /var/lib/jenkins/plugins/ssh-agent/WEB-INF/lib/slf4j-jdk14-1.6.6.jar
java    31315 jenkins  mem    REG              147,0    26176 134320364 /var/lib/jenkins/plugins/ssh-agent/WEB-INF/lib/slf4j-api-1.6.6.jar
java    31315 jenkins  mem    REG              147,0   515071 134320363 /var/lib/jenkins/plugins/ssh-agent/WEB-INF/lib/bcpkix-jdk15on-1.47.jar
java    31315 jenkins  mem    REG              147,0  2107413 134320362 /var/lib/jenkins/plugins/ssh-agent/WEB-INF/lib/jnr-unixsocket-nodep-0.3.1.jar
java    31315 jenkins  mem    REG              147,0    25160 134320275 /var/lib/jenkins/plugins/ssh-agent/WEB-INF/lib/tomcat-apr-5.5.23.jar
java    31315 jenkins  mem    REG              147,0   621694 134320274 /var/lib/jenkins/plugins/ssh-agent/WEB-INF/lib/sshd-core-0.12.0.jar
java    31315 jenkins  mem    REG              147,0    27746 140758219 /var/lib/jenkins/plugins/extended-choice-parameter/WEB-INF/lib/classes.jar
java    31315 jenkins  mem    REG              147,0    15347 140758192 /var/lib/jenkins/plugins/extended-choice-parameter/WEB-INF/lib/opencsv-2.0.jar
java    31315 jenkins  mem    REG              147,0   191122 161612552 /var/lib/jenkins/plugins/publish-over-ssh/WEB-INF/lib/classes.jar
java    31315 jenkins  mem    REG              147,0   223203 161612551 /var/lib/jenkins/plugins/publish-over-ssh/WEB-INF/lib/jsch-0.1.45.jar
java    31315 jenkins  mem    REG              147,0    93041 161612550 /var/lib/jenkins/plugins/publish-over-ssh/WEB-INF/lib/publish-over-0.18.jar
java    31315 jenkins  mem    REG              147,0    97589  67110447 /var/lib/jenkins/plugins/cloudbees-folder/WEB-INF/lib/cloudbees-folder.jar
java    31315 jenkins  mem    REG              147,0    72795  67162479 /var/lib/jenkins/plugins/build-timeout/WEB-INF/lib/classes.jar
java    31315 jenkins  mem    REG              147,0    21510 161612519 /var/lib/jenkins/plugins/git-parameter/WEB-INF/lib/git-parameter.jar
java    31315 jenkins  mem    REG              147,0    76643 161612518 /var/lib/jenkins/plugins/git-parameter/WEB-INF/lib/hamcrest-core-1.1.jar
java    31315 jenkins  mem    REG              147,0   589512 161612517 /var/lib/jenkins/plugins/git-parameter/WEB-INF/lib/httpclient-4.3.3.jar
java    31315 jenkins  mem    REG              147,0   282269 161612516 /var/lib/jenkins/plugins/git-parameter/WEB-INF/lib/httpcore-4.3.2.jar
java    31315 jenkins  mem    REG              147,0    62050 161612515 /var/lib/jenkins/plugins/git-parameter/WEB-INF/lib/commons-logging-1.1.3.jar
java    31315 jenkins  mem    REG              147,0   245039 161612512 /var/lib/jenkins/plugins/git-parameter/WEB-INF/lib/junit-4.11.jar
java    31315 jenkins  mem    REG              147,0   232771 146302882 /var/lib/jenkins/plugins/git-parameter/WEB-INF/lib/commons-codec-1.6.jar
java    31315 jenkins  mem    REG              147,0    16088  67110454 /var/lib/jenkins/plugins/git/WEB-INF/lib/annotation-indexer-1.4.jar
java    31315 jenkins  mem    REG              147,0   587512  67110453 /var/lib/jenkins/plugins/git/WEB-INF/lib/joda-time-2.6.jar
java    31315 jenkins  mem    REG              147,0   346194  67110452 /var/lib/jenkins/plugins/git/WEB-INF/lib/git.jar
java    31315 jenkins  mem    REG              147,0     3161  67110451 /var/lib/jenkins/plugins/git/WEB-INF/lib/bridge-method-annotation-1.13.jar
java    31315 jenkins  mem    REG              147,0   282269  67110450 /var/lib/jenkins/plugins/git/WEB-INF/lib/httpcore-4.3.2.jar
java    31315 jenkins  mem    REG              147,0    62050  67110449 /var/lib/jenkins/plugins/git/WEB-INF/lib/commons-logging-1.1.3.jar
java    31315 jenkins  mem    REG              147,0   232771  67109205 /var/lib/jenkins/plugins/git/WEB-INF/lib/commons-codec-1.6.jar
java    31315 jenkins  mem    REG              147,0   254408 227233469 /var/lib/jenkins/plugins/git-client/WEB-INF/lib/jsch-0.1.50.jar
java    31315 jenkins  mem    REG              147,0    61829 227233464 /var/lib/jenkins/plugins/git-client/WEB-INF/lib/commons-logging-1.2.jar
java    31315 jenkins  mem    REG              147,0   321639 227233463 /var/lib/jenkins/plugins/git-client/WEB-INF/lib/httpcore-4.4.jar
java    31315 jenkins  mem    REG              147,0   719304 227193212 /var/lib/jenkins/plugins/git-client/WEB-INF/lib/httpclient-4.4.jar
java    31315 jenkins  mem    REG              147,0   166618 227131586 /var/lib/jenkins/plugins/git-client/WEB-INF/lib/git-client.jar
java    31315 jenkins  mem    REG              147,0  1959346 227131585 /var/lib/jenkins/plugins/git-client/WEB-INF/lib/org.eclipse.jgit-3.6.2.201501210735-r.jar
java    31315 jenkins  mem    REG              147,0    91191 227064688 /var/lib/jenkins/plugins/git-client/WEB-INF/lib/org.eclipse.jgit.http.server-3.6.2.201501210735-r.jar
java    31315 jenkins  mem    REG              147,0   125146 226967077 /var/lib/jenkins/plugins/git-client/WEB-INF/lib/JavaEWAH-0.7.9.jar
java    31315 jenkins  mem    REG              147,0    15322 201414927 /var/lib/jenkins/plugins/git-client/WEB-INF/lib/findbugs-annotations-1.3.9-1.jar
java    31315 jenkins  mem    REG              147,0     2898 201403870 /var/lib/jenkins/plugins/countjobs-viewstabbar/WEB-INF/lib/classes.jar
java    31315 jenkins  mem    REG              147,0  1229125  12036079 /var/lib/jenkins/plugins/parameterized-trigger/WEB-INF/lib/xercesImpl-2.9.1.jar
java    31315 jenkins  mem    REG              147,0   194354  12036276 /var/lib/jenkins/plugins/parameterized-trigger/WEB-INF/lib/xml-apis-1.3.04.jar
java    31315 jenkins  mem    REG              147,0    20503 201403850 /var/lib/jenkins/plugins/ssh/WEB-INF/lib/classes.jar
java    31315 jenkins  mem    REG              147,0   185746 201403849 /var/lib/jenkins/plugins/ssh/WEB-INF/lib/jsch-0.1.42.jar
java    31315 jenkins  mem    REG              147,0   120586  67162499 /var/lib/jenkins/plugins/jobConfigHistory/WEB-INF/lib/jobConfigHistory.jar
java    31315 jenkins  mem    REG              147,0   694992 207984137 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/classes.jar
java    31315 jenkins  mem    REG              147,0    32265 207984135 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-http-shared4-2.4.jar
java    31315 jenkins  mem    REG              147,0    34117 207619360 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/aether-connector-wagon-0.9.0.M2.jar
java    31315 jenkins  mem    REG              147,0   163864 202497661 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/aether-impl-0.9.0.M3.jar
java    31315 jenkins  mem    REG              147,0   251792 202390128 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/plexus-utils-3.0.17.jar
java    31315 jenkins  mem    REG              147,0     9245 201415015 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven-agent-1.6.jar
java    31315 jenkins  mem    REG              147,0    56907 201326945 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven-embedder-3.1.0.jar
java    31315 jenkins  mem    REG              147,0    28972 201326944 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-ssh-2.4.jar
java    31315 jenkins  mem    REG              147,0   326020 201326911 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/jackrabbit-webdav-2.5.2.jar
java    31315 jenkins  mem    REG              147,0    28555 201326910 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/plexus-sec-dispatcher-1.3.jar
java    31315 jenkins  mem    REG              147,0     5848 201326909 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/jsr250-api-1.0.jar
java    31315 jenkins  mem    REG              147,0    13350 201326908 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/plexus-cipher-1.7.jar
java    31315 jenkins  mem    REG              147,0   562575 201326907 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven-core-3.1.0.jar
java    31315 jenkins  mem    REG              147,0    49820 201326906 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven-plugin-api-3.1.0.jar
java    31315 jenkins  mem    REG              147,0     7686 201326905 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven31-agent-1.6.jar
java    31315 jenkins  mem    REG              147,0     7749 201326904 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven3-agent-1.6.jar
java    31315 jenkins  mem    REG              147,0    49793 201326903 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/plexus-classworlds-2.5.1.jar
java    31315 jenkins  mem    REG              147,0    52437 201326902 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-provider-api-2.4.jar
java    31315 jenkins  mem    REG              147,0    14520 201326901 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven3-interceptor-1.6.jar
java    31315 jenkins  mem    REG              147,0    61308 201326900 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/plexus-interpolation-1.16.jar
java    31315 jenkins  mem    REG              147,0    25408 201326899 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-ssh-common-2.4.jar
java    31315 jenkins  mem    REG              147,0    22292 201326898 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven-interceptor-1.6.jar
java    31315 jenkins  mem    REG              147,0   305001 201326897 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/commons-httpclient-3.1.jar
java    31315 jenkins  mem    REG              147,0  2000557 201326896 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/ant-1.9.2.jar
java    31315 jenkins  mem    REG              253,0   346544   1578141 /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.75.x86_64/jre/lib/amd64/libfontmanager.so
java    31315 jenkins  mem    REG              253,0    14288   1048684 /lib64/libgpg-error.so.0.5.0
java    31315 jenkins  mem    REG              253,0   122040   1049006 /lib64/libselinux.so.1
java    31315 jenkins  mem    REG              253,0    10312   1048585 /lib64/libfreebl3.so
java    31315 jenkins  mem    REG              253,0   265728   1048646 /lib64/libdbus-1.so.3.4.0
java    31315 jenkins  mem    REG              253,0   478496   1048687 /lib64/libgcrypt.so.11.5.3
java    31315 jenkins  mem    REG              253,0    66608   1180980 /usr/lib64/libtasn1.so.3.1.6
java    31315 jenkins  mem    REG              253,0    10192   1048790 /lib64/libkeyutils.so.1.3
java    31315 jenkins  mem    REG              253,0    43728   1048993 /lib64/libkrb5support.so.0.1
java    31315 jenkins  mem    REG              147,0       64       132 /var/lib/jenkins/secret.key
java    31315 jenkins  mem    REG              147,0    16041 201326893 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-ftp-2.4.jar
java    31315 jenkins  mem    REG              147,0   283801 201326892 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven-compat-3.1.0.jar
java    31315 jenkins  mem    REG              147,0    52173 201326891 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven-artifact-3.1.0.jar
java    31315 jenkins  mem    REG              147,0     7698 201326889 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven32-agent-1.6.jar
java    31315 jenkins  mem    REG              147,0    18333 201326888 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/ant-launcher-1.9.2.jar
java    31315 jenkins  mem    REG              147,0   232771 201326887 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/commons-codec-1.6.jar
java    31315 jenkins  mem    REG              147,0    26083 201326886 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/slf4j-api-1.7.4.jar
java    31315 jenkins  mem    REG              147,0    41427 201326885 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven-settings-builder-3.1.0.jar
java    31315 jenkins  mem    REG              147,0   196544 201326884 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/org.eclipse.sisu.plexus-0.0.0.M5.jar
java    31315 jenkins  mem    REG              147,0    41123 201326883 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/commons-cli-1.2.jar
java    31315 jenkins  mem    REG              147,0     2497 201326882 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/javax.inject-1.jar
java    31315 jenkins  mem    REG              147,0    33015 201326881 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/jsr305-1.3.9.jar
java    31315 jenkins  mem    REG              147,0    30094 201326880 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven-repository-metadata-3.1.0.jar
java    31315 jenkins  mem    REG              147,0    25245 201326879 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/aether-transport-wagon-0.9.0.M3.jar
java    31315 jenkins  mem    REG              147,0     4587 201326878 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven2.1-interceptor-1.2.jar
java    31315 jenkins  mem    REG              147,0    44908 201326877 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/cdi-api-1.0.jar
java    31315 jenkins  mem    REG              147,0    14868 201326876 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-ssh-external-2.4.jar
java    31315 jenkins  mem    REG              147,0   159442 201326875 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven-model-builder-3.1.0.jar
java    31315 jenkins  mem    REG              147,0   273370 201326874 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/commons-net-3.1.jar
java    31315 jenkins  mem    REG              147,0    10073 201326873 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/doxia-sink-api-1.0.jar
java    31315 jenkins  mem    REG              147,0   173587 201326872 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/commons-io-2.2.jar
java    31315 jenkins  mem    REG              147,0    19278 201326871 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven32-interceptor-1.6.jar
java    31315 jenkins  mem    REG              147,0     6758 201326870 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven3-interceptor-commons-1.6.jar
java    31315 jenkins  mem    REG              147,0    19064 201326869 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven31-interceptor-1.6.jar
java    31315 jenkins  mem    REG              147,0  1238788 201326868 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/guice-4.0-beta.jar
java    31315 jenkins  mem    REG              147,0    23668 201326867 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/lib-jenkins-maven-embedder-3.11.jar
java    31315 jenkins  mem    REG              147,0   213781 201326866 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/jsch-0.1.44-1.jar
java    31315 jenkins  mem    REG              147,0     4211 201326865 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/plexus-component-annotations-1.5.5.jar
java    31315 jenkins  mem    REG              147,0    46708 201326864 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven-settings-3.1.0.jar
java    31315 jenkins  mem    REG              147,0    10940 201326863 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven-reporting-api-3.0.jar
java    31315 jenkins  mem    REG              147,0    60686 201326862 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/commons-logging-1.1.1.jar
java    31315 jenkins  mem    REG              147,0     4467 201326861 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/aopalliance-1.0.jar
java    31315 jenkins  mem    REG              147,0   142136 201326860 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/aether-util-0.9.0.M3.jar
java    31315 jenkins  mem    REG              147,0    26343 201326859 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-http-shared-2.4.jar
java    31315 jenkins  mem    REG              147,0   163733 201326858 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven-model-3.1.0.jar
java    31315 jenkins  mem    REG              147,0   227275 201326857 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/httpcore-4.2.4.jar
java    31315 jenkins  mem    REG              147,0   433368 201326856 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/httpclient-4.2.5.jar
java    31315 jenkins  mem    REG              147,0   291397 201326855 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/org.eclipse.sisu.inject-0.0.0.M5.jar
java    31315 jenkins  mem    REG              147,0    10479 201326854 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-http-2.4.jar
java    31315 jenkins  mem    REG              147,0    11185 201326852 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/wagon-file-2.4.jar
java    31315 jenkins  mem    REG              147,0    30624 201326848 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/aether-spi-0.9.0.M3.jar
java    31315 jenkins  mem    REG              147,0    60050 201326783 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven-aether-provider-3.1.0.jar
java    31315 jenkins  mem    REG              147,0    14567 201326781 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/lib-jenkins-maven-artifact-manager-1.2.jar
java    31315 jenkins  mem    REG              147,0   284220 201326780 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/commons-lang-2.6.jar
java    31315 jenkins  mem    REG              147,0   278273 201326779 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/jsoup-1.7.1.jar
java    31315 jenkins  mem    REG              147,0     7629 201326778 /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/SECURITY-144-compat-1.0.jar
java    31315 jenkins  mem    REG              253,2    23410    131951 /var/cache/jenkins/war/WEB-INF/lib/sshd-1.6.jar
java    31315 jenkins  mem    REG              253,2    45024    131905 /var/cache/jenkins/war/WEB-INF/lib/hamcrest-core-1.3.jar
java    31315 jenkins  357w  FIFO                0,8      0t0  14115294 pipe
java    31315 jenkins  358u   REG                0,9        0      3791 [eventpoll]
java    31315 jenkins  359r  FIFO                0,8      0t0  14115295 pipe
java    31315 jenkins  360w  FIFO                0,8      0t0  14115295 pipe
java    31315 jenkins  361u   REG                0,9        0      3791 [eventpoll]
java    31315 jenkins  362r  FIFO                0,8      0t0  14115296 pipe
java    31315 jenkins  363w  FIFO                0,8      0t0  14115296 pipe
java    31315 jenkins  364u   REG                0,9        0      3791 [eventpoll]
java    31315 jenkins  365r  FIFO                0,8      0t0  14115297 pipe
java    31315 jenkins  366w  FIFO                0,8      0t0  14115297 pipe
java    31315 jenkins  367u   REG                0,9        0      3791 [eventpoll]
java    31315 jenkins  368u  IPv4           14115299      0t0       TCP *:48675 (LISTEN)
java    31315 jenkins  372u  unix 0xffff88007da91b40      0t0  14679054 /var/run/jenkins/jenkins1370916637023013225.jnr
java    31315 jenkins  374u  unix 0xffff8800125fa240      0t0  14461419 /var/run/jenkins/jenkins8398224656456936291.jnr
java    31315 jenkins  375u  IPv4           14115330      0t0       TCP *:32981 (LISTEN)
java    31315 jenkins  377u  IPv4           14115332      0t0       UDP *:33848
java    31315 jenkins  378w   REG              147,0     1849     22096 /var/lib/jenkins/logs/slaves/ft-dbadmin/slave.log
java    31315 jenkins  379u  IPv4           14115334      0t0       TCP ft-itjenkins-02.infra.deliveryagent.com:57306->ft-dbadmin.infra.deliveryagent.com:ssh (ESTABLISHED)
java    31315 jenkins  380u   REG              253,1     4096        13 /tmp/ffi2Xmdwb (deleted)
java    31315 jenkins  381r   CHR                1,8      0t0      3797 /dev/random
java    31315 jenkins  382u  unix 0xffff88000efcfc80      0t0  14115338 socket
java    31315 jenkins  383r  FIFO                0,8      0t0  14115342 pipe
java    31315 jenkins  384w  FIFO                0,8      0t0  14115342 pipe
java    31315 jenkins  385u  unix 0xffff88001547c7c0      0t0  14115356 socket
java    31315 jenkins  386r  FIFO                0,8      0t0  14115345 pipe
java    31315 jenkins  387w  FIFO                0,8      0t0  14115345 pipe
java    31315 jenkins  390u  unix 0xffff88007da90140      0t0  53943672 socket
java    31315 jenkins  391r   REG              253,2  5206312    524439 /var/lib/sss/mc/group
java    31315 jenkins  392u  unix 0xffff88007a99e1c0      0t0  15980831 /var/run/jenkins/jenkins5924072382995393716.jnr
java    31315 jenkins  397u  unix 0xffff880036fe2d80      0t0  14278623 /var/run/jenkins/jenkins3357919870524830562.jnr
java    31315 jenkins  398u  unix 0xffff8800641f1400      0t0  15776213 /var/run/jenkins/jenkins294283884310094945.jnr
java    31315 jenkins  411u  unix 0xffff8800129cb280      0t0  14985202 /var/run/jenkins/jenkins6757838858038998298.jnr
java    31315 jenkins  416u  unix 0xffff880011fa0280      0t0  15270625 /var/run/jenkins/jenkins8643090794428494396.jnr
java    31315 jenkins  426u  unix 0xffff88000cff0240      0t0  15395690 /var/run/jenkins/jenkins1148518038479457977.jnr

The files appear to be getting created once per hour... here is a snippit of /var/log/jenkins:

srw-------.  1 jenkins jenkins      0 Mar 22 02:23 jenkins2794149906824597553.jnr
srw-------.  1 jenkins jenkins      0 Mar 22 03:23 jenkins6588950915865152808.jnr
srw-------.  1 jenkins jenkins      0 Mar 22 04:23 jenkins6611600580713270558.jnr
srw-------.  1 jenkins jenkins      0 Mar 22 05:23 jenkins8818140860803025721.jnr
srw-------.  1 jenkins jenkins      0 Mar 22 06:23 jenkins4848121313305928439.jnr

is duplicated by

JENKINS-27922 Jenkins job execution becomes unstable - jobs fail with OOM: unable to create new native thread

Resolved

Assignee:: Yoann Dubreuil
Reporter:: Tommy McNeely

Created:: 2015-03-23 20:23
Updated:: 2026-06-04 21:26
Resolved:: 2015-06-02 15:07
Archived:: 2026-06-04 21:27

Details

Description

Attachments

Issue Links

Activity

People

Dates