Expected behavior (Ubuntu 10.04 LTS):
mike@boontoo10:~$ java -version
java version "1.6.0_45"
Java(TM) SE Runtime Environment (build 1.6.0_45-b06)
Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode)
mike@boontoo10:~$ java -Xmx512m -Xms512m -jar forker.jar 10000000 20 Xmx512m Xms512m
output of "top":
1928 mike 20 0 694m 54m 8624 S 6 5.4 0:00.18 java 1933 mike 20 0 694m 54m 8648 S 6 5.4 0:00.18 java 1935 mike 20 0 694m 54m 8648 S 6 5.4 0:00.18 java 1939 mike 20 0 694m 54m 8648 S 5 5.4 0:00.16 java
(the java7 output looks like this on 10.04):
mike@ubuntu10java7:~$ java -versionjava version "1.7.0_40"
Java(TM) SE Runtime Environment (build 1.7.0_40-b43)
Java HotSpot(TM) 64-Bit Server VM (build 24.0-b56, mixed mode)
2106 mike 20 0 734m 57m 9876 S 4 2.8 0:00.25 java 2091 mike 20 0 734m 57m 9876 S 3 2.8 0:00.26 java 2100 mike 20 0 734m 57m 9876 S 3 2.8 0:00.25 java 2112 mike 20 0 734m 57m 9876 S 3 2.8 0:00.24 java 2097 mike 20 0 734m 57m 9876 S 3 2.8 0:00.24 java
New behavior (Ubuntu 12.04 LTS):
mike@boontoo12:~$ java -version
java version "1.6.0_45"
Java(TM) SE Runtime Environment (build 1.6.0_45-b06)
Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode)
mike@boontoo12:~$ java -Xmx512m -Xms512m -jar forker.jar 10000000 20 Xmx512m Xms512m
output of "top":
20697 mike 20 0 1405m 54m 8488 S 5 1.4 0:00.18 java 20669 mike 20 0 1405m 54m 8492 S 5 1.4 0:00.18 java 20691 mike 20 0 1405m 54m 8492 S 5 1.4 0:00.17 java 20658 mike 20 0 1405m 54m 8492 S 5 1.4 0:00.17 java
Update: New behavior (Ubuntu 12.04 LTS w/ Java 7):
mike@boontoo12java7:~$ java -version
java version "1.7.0_40"
Java(TM) SE Runtime Environment (build 1.7.0_40-b43)
Java HotSpot(TM) 64-Bit Server VM (build 24.0-b56, mixed mode)
output of "top":
4166 mike 20 0 1438m 56m 9572 S 7 2.8 0:00.20 java 4169 mike 20 0 1438m 56m 9572 S 7 2.8 0:00.20 java 4182 mike 20 0 1438m 56m 9572 S 7 2.8 0:00.20 java 4076 mike 20 0 1438m 56m 9572 S 6 2.8 0:00.19 java 4100 mike 20 0 1438m 56m 9572 S 6 2.8 0:00.19 java
ctd... 12.0.4 with OpenJDK 6, similar problem:
mike@boontoo12:~$ java -version
java version "1.6.0_27"
OpenJDK Runtime Environment (IcedTea6 1.12.6) (6b27-1.12.6-1ubuntu0.12.04.2)
OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)
output of "top":
6690 mike 20 0 1500m 55m 8920 S 7 1.4 0:00.22 java 6692 mike 20 0 1500m 55m 8920 S 7 1.4 0:00.21 java 6697 mike 20 0 1500m 55m 8920 S 7 1.4 0:00.21 java 6701 mike 20 0 1500m 51m 8684 S 7 1.3 0:00.21 java 6725 mike 20 0 1500m 55m 8920 S 7 1.4 0:00.21 java
To preempt, here's the OpenJDK 7 attempt:
mike@boontoo12:~$ java -version
java version "1.7.0_25"
OpenJDK Runtime Environment (IcedTea 2.3.10) (7u25-2.3.10-1ubuntu0.12.04.2)
OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode)
output of "top":
9327 mike 20 0 1534m 57m 9296 S 6 1.4 0:00.25 java 9331 mike 20 0 1534m 57m 9276 S 6 1.4 0:00.25 java 9344 mike 20 0 1534m 57m 9300 S 6 1.4 0:00.25 java 9347 mike 20 0 1534m 57m 9300 S 6 1.4 0:00.25 java 9323 mike 20 0 1534m 57m 9300 S 6 1.4 0:00.25 java 9284 mike 20 0 1534m 57m 9308 S 5 1.4 0:00.24 java 9285 mike 20 0 1534m 57m 9300 S 5 1.4 0:00.24 java 9336 mike 20 0 1534m 57m 9276 S 5 1.4 0:00.23 java 9277 mike 20 0 1534m 57m 9300 S 5 1.4 0:00.24 java
As you can see, the vmem utilization has doubled. I'm using the exact same java/jre binary (and jdk), the only difference being the version of Ubuntu.
I'm hoping to get this program back to ~694m of vmem used. How do I do that? What do I even search for?
Please don't suggest that I structure my program differently. This is the smallest toy-example I could write to reproduce a gigantic problem reflected in some Apache Software, but this feels like a kernel/libs problem to me.
Here's the code to duplicate, and the link where all this is summarized in a gist: https://gist.github.com/michael-trelinski
Code:
package com.satoricode.forker;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class ForkIt {
public static void main(String[] args) throws InterruptedException {
int arraySize = Integer.parseInt(args[0]);
System.out.print("Filling up some memory... ");
int[] arr = fillItUp(arraySize);
System.out.println("" + arr.length + " done.");
if (args.length == 1) {
System.out.println("I'm the fork!");
Thread.sleep(10 * 1000);
} else {
// I'm the master
int numForks = Integer.parseInt(args[1]);
String stackCommand = args[2];
String stackInitCommand = args[3];
ExecutorService pool = Executors.newFixedThreadPool(numForks);
System.out.println("Launching " + numForks + " forks with array size " + arraySize + " with -" + stackCommand + " -" + stackInitCommand);
for(int I = 0; I < numForks; I++)
pool.execute(new ForkRunner(arraySize, stackCommand, stackInitCommand));
pool.shutdown();
System.out.print("Waiting for pool to finish");
while(!pool.isTerminated()) {
Thread.sleep(50);
System.out.print(".");
}
System.out.println();
System.out.println("done\n");
}
}
private static int[] fillItUp(int total) {
int[] arr = new int[total];
for (int I = 0; I < total; I++)
arr[I] = I;
return arr;
}
}
package com.satoricode.forker;
import java.io.IOException;
import java.io.InputStream;
public class ForkRunner implements Runnable {
private String stackCommand;
private int arraySize;
private String stackInitCommand;
public ForkRunner(int arraySize, String stackCommand, String stackInitCommand) {
this.arraySize = arraySize;
this.stackCommand = stackCommand;
this.stackInitCommand = stackInitCommand;
}
@Override
public void run() {
try {
String command = "java -" + stackCommand + " -" + stackInitCommand + " -jar forker.jar " + arraySize;
System.out.println("executing: " + command);
Process proc = Runtime.getRuntime().exec(command);
InputStream input = proc.getInputStream();
InputStream error = proc.getErrorStream();
StringBuilder sb = new StringBuilder();
int c = -1;
while ((c = input.read()) != -1) {
sb.append((char) c);
}
while ((c = error.read()) != -1) {
sb.append((char) c);
}
System.out.println(sb.toString());
} catch (IOException e) {
e.printStackTrace();
}
}
}
