OutOfMemoryError: unable to create new native thread with no obvious reason

Question

When starting my modded Minecraft Server using java, I get the following error:

java.lang.OutOfMemoryError: unable to create new native thread

So my first guess was, that maybe something is creating too many threads, exceeding the limit, but the server only creates around 50 threads, increasing the total number of threads on my machine from 420 to 470, which is nowhere near my limit. I also don't think that the memory should be a problem, since I gave my server about 8 GB which should be more than enough to start.

Here is the java command, I use to start the server:

java -server -XX:PermSize=512M -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -XX:ParallelGCThreads=2 -XX:+AggressiveOpts -jar forge.jar -Xms8G -Xmx8G nogui -Dfml.queryResult=confirm

I used the following command to count the total number of threads

ps -eo nlwp | tail -n +2 | awk '{ num_threads += $1 } END { print num_threads }'

As described here.

I executed that command manually multiple times and immediately before the server crashed, the count was at 470. So unless the server was creating more than 700 thousand threads in under a second, the limit should not have been exceeded.

Still before the server crashed, my bash was giving me these errors:

-bash: fork: retry: Resource temporarily unavailable

which lets me believe, that there indeed has been reached some limit and the problem is not only inside of the vm.

My server has 32 GB of RAM and according to top, the memory usage of the java process was at under 6%, which would be less than 2 GB, so the vm should have more than enough memory left.

According to this troubleshoot, the other meaning of the OutOfMemoryError could be that there is not enough memory left to create new threads, but since only 6 out of my total 32GB are used, this should not be the problem either.

So since the thread count is well under the limit and there is more than enough memory left (in the vm and the rest of the system), what else could be the problem?

These are the limits that are currently set:

mc@h*******:~$ ulimit -a
core file size          (blocks, -c) unlimited
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 772691
max locked memory       (kbytes, -l) 16384
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 772691
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

My Java version:

mc@h*******:~$ java -version
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

My Ubuntu version:

mc@h*******:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.1 LTS
Release:    18.04
Codename:   bionic

I tried creating a threaddump using jstack -l <PID> >> threaddump.log. While the server starts up and everything still works as normal, the log file looks like this:

Error occurred during initialization of VM
java.lang.OutOfMemoryError: unable to create new native thread

When the server has almost finished to start (shortly before the server crashes with the OutOfMemoryError) the log looks like this:

# There is insufficient memory for the Java Runtime Environment to continue.
# Cannot create GC thread. Out of system resources.
# An error report file with more information is saved as:
# /home/mc/hs_err_pid24081.log

I have uploaded the error log here.

I use this to run my local heavily modded game: -Xmx6656M -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:-UseAdaptiveSizePolicy -Xmn128M but thats running due to the mod's on openjdk-8 still. — Videonauth, Oct 05 '18 at 22:02
I am also using Java 8 and I can assure, that the problem are not the arguments themselves. On my old Ubuntu 16.04 server, the arguments worked fine and the server never had any problems with memory. — iComputerfreak, Oct 05 '18 at 22:05

iComputerfreak · Accepted Answer · 2018-10-12T14:36:34.553

Since reinstalling the Oracle JDK did not help, I tried the OpenJDK and it worked. The OutOfMemoryErrors are now gone and everything works as expected.

EDIT

It seems, like switching the JDK did only help temporarily. But I just found out the real origin of the problem. It seems, like the OutOfMemoryError was caused by a specific mod that was in both modpacks, that I tried. I still don't know, how the OOME occured without increased thread count or a full memory, but anyways.

The mod that caused the problems was Astral Sorcery (version 1.12.2-1.9.4). So if you run into the same problems with your modded minecraft server, try removing that mod.

For future reference, OpenJDK in my experience and as told to me by a certified Java expert is much less buggy on Ubuntu than the regular JDK is. — tommy61157, Oct 11 '18 at 13:47

OutOfMemoryError: unable to create new native thread with no obvious reason

1 Answers1

EDIT