Since a couple of weeks we are seeing intermittent crashes of our Jenkins master due to some Java code which should not have been reached.

The crash info looks like:

A fatal error has been detected by the Java Runtime Environment:

Internal Error (instanceKlass.cpp:1534), pid=2378, tid=239618880
Error: ShouldNotReachHere()

JRE version: Java(TM) SE Runtime Environment (7.0_45-b18) (build 1.7.0_45-b18)
Java VM: Java HotSpot(TM) Client VM (24.45-b08 mixed mode linux-x86 )
Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before retarting Java again

--------------- T H R E A D ---------------

Current thread (0xb6a86c00): JavaThread "C1 CompilerThread0" daemon [threadin_vm, id=2386, stack(0x0e404000,0x0e485000)]

Stack: [0x0e404000,0x0e485000], sp=0x0e483c70, free space=511k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x503a66] VMError::report_and_die()+0x1a6
V [libjvm.so+0x1db48a] report_should_not_reach_here(char const*, int)+0x4a
V [libjvm.so+0x2743be] instanceKlass::remove_dependent_nmethod(nmethod*)+0x3e

For a complete crash stack please see the attached log file.

It sounds like it has been caused by low memory. While top might indeed have shown that the whole memory was filled up on that machine, the java management console hasn't shown it.

I have already submitted crash report to Oracle a while ago but haven't gotten any reply yet. I think it's worth also filing a bug here in case someone knows about this problem or is also affected by this.