Home Tutorials Training Consulting Books Company Contact us


Get more...

Java Networking. This article describes how to write Java performance benchmarks with the JMH (Java Microbenchmark Harness) framework.

1. Using JMH

JMH is provided by set of libraries. To use it in your project you have to add the set of libraries to your project. For example with Maven you can use folowing additional dependencies.

    <dependencies>
        <dependency>
            <groupId>org.openjdk.jmh</groupId>
            <artifactId>jmh-core</artifactId>
            <version>1.33</version>
        </dependency>

        <dependency>
            <groupId>org.openjdk.jmh</groupId>
            <artifactId>jmh-generator-annprocess</artifactId>
            <version>1.33</version>
        </dependency>
    </dependencies>

After this can you write your tests using the @Benchmark annotation.

For example the following two:

HINT: This is an example from https://shipilev.net/blog/2016/arrays-wisdom-ancients/

package benchmarks;

import java.util.ArrayList;
import java.util.Collection;
import java.util.HashSet;
import java.util.concurrent.TimeUnit;

import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Fork;
import org.openjdk.jmh.annotations.Measurement;
import org.openjdk.jmh.annotations.OutputTimeUnit;
import org.openjdk.jmh.annotations.Param;
import org.openjdk.jmh.annotations.Setup;
import org.openjdk.jmh.annotations.State;
import org.openjdk.jmh.annotations.Warmup;

@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.MILLISECONDS)
@Measurement(iterations = 200, time = 1, timeUnit = TimeUnit.MILLISECONDS)
//@Fork(value = 3, jvmArgsAppend = {"-XX:+UseParallelGC", "-Xms1g", "-Xmx1g"})
@Fork(value = 1)
@BenchmarkMode(org.openjdk.jmh.annotations.Mode.AverageTime)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
@State(org.openjdk.jmh.annotations.Scope.Benchmark)
public class ToArrayBench {

    @Param({ "0", "1", "10", "100", "1000" })
    int size;
    @Param({ "arraylist" })//, "hashset"
    String type;
    Collection<Foo> coll;

    @Setup
    public void setup() {
        if (type.equals("arraylist")) {
            coll = new ArrayList<Foo>();
        } else if (type.equals("hashset")) {
            coll = new HashSet<Foo>();
        } else {
            throw new IllegalStateException();
        }
        for (int i = 0; i < size; i++) {
            coll.add(new Foo(i));
        }
    }

    @Benchmark
    public Object[] simple() {
        return coll.toArray();
    }

    @Benchmark
    public Foo[] zero() {
        return coll.toArray(new Foo[0]);
    }

    @Benchmark
    public Foo[] sized() {
        return coll.toArray(new Foo[coll.size()]);
    }

    @Benchmark
    public Foo[] arrayNew() {
        return coll.toArray(Foo[]::new);
    }

    public static class Foo {
        private int i;

        public Foo(int i) {
            this.i = i;
        }

        @Override
        public boolean equals(Object o) {
            if (this == o)
                return true;
            if (o == null || getClass() != o.getClass())
                return false;
            Foo foo = (Foo) o;
            return i == foo.i;
        }

        @Override
        public int hashCode() {
            return i;
        }
    }

}
package benchmarks;

import java.util.ArrayList;
import java.util.Collection;
import java.util.HashSet;
import java.util.concurrent.TimeUnit;

import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Fork;
import org.openjdk.jmh.annotations.Measurement;
import org.openjdk.jmh.annotations.OutputTimeUnit;
import org.openjdk.jmh.annotations.Param;
import org.openjdk.jmh.annotations.Setup;
import org.openjdk.jmh.annotations.State;
import org.openjdk.jmh.annotations.Warmup;

@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.MILLISECONDS)
@Measurement(iterations = 200, time = 1, timeUnit = TimeUnit.MILLISECONDS)
//@Fork(value = 3, jvmArgsAppend = {"-XX:+UseParallelGC", "-Xms1g", "-Xmx1g"})
@Fork(value = 1)
@BenchmarkMode(org.openjdk.jmh.annotations.Mode.AverageTime)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
@State(org.openjdk.jmh.annotations.Scope.Benchmark)
public class LocalInitialization {

    @Setup
    public void setup() {
    }

    @Benchmark
    public void unnecessaryInitialization() {
        String s = "";
        s = "Hello";
    }
    
    @Benchmark
    public void directAssignment() {
        String s =  "Hello";
    }
}

Afterwards you can add this benchmarks to a benchmark runner.

package benchmarks;

import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;

public class BenchmarkRunner {

    public static void main(String[] args) throws RunnerException {
        Options opt = new OptionsBuilder().include(LocalInitialization.class.getSimpleName()).include(ToArrayBench.class.getSimpleName()).build();

        new Runner(opt).run();
    }
}

Build your project via mvn clean verify and run your BenchmarkRunner class via your IDE.

This should run the benchmarks can return the collected data.

Benchmark                                      (size)     (type)  Mode  Cnt     Score     Error  Units
LocalInitialization.directAssignment              N/A        N/A  avgt  200     0,394 ±   0,015  ns/op
LocalInitialization.unnecessaryInitialization     N/A        N/A  avgt  200     0,392 ±   0,012  ns/op
ToArrayBench.arrayNew                               0  arraylist  avgt  200     6,181 ±   0,522  ns/op
ToArrayBench.arrayNew                               1  arraylist  avgt  200    22,034 ±   2,614  ns/op
ToArrayBench.arrayNew                              10  arraylist  avgt  200    34,663 ±   7,253  ns/op
ToArrayBench.arrayNew                             100  arraylist  avgt  200   197,565 ±  20,510  ns/op
ToArrayBench.arrayNew                            1000  arraylist  avgt  200  1513,145 ± 218,155  ns/op
ToArrayBench.simple                                 0  arraylist  avgt  200     7,936 ±   1,204  ns/op
ToArrayBench.simple                                 1  arraylist  avgt  200    11,370 ±   1,380  ns/op
ToArrayBench.simple                                10  arraylist  avgt  200    14,236 ±   2,174  ns/op
ToArrayBench.simple                               100  arraylist  avgt  200    94,752 ±  89,931  ns/op
ToArrayBench.simple                              1000  arraylist  avgt  200   608,612 ±  72,024  ns/op
ToArrayBench.sized                                  0  arraylist  avgt  200     8,028 ±   4,179  ns/op
ToArrayBench.sized                                  1  arraylist  avgt  200    17,983 ±   1,108  ns/op
ToArrayBench.sized                                 10  arraylist  avgt  200    33,522 ±   9,677  ns/op
ToArrayBench.sized                                100  arraylist  avgt  200   172,629 ±  22,120  ns/op
ToArrayBench.sized                               1000  arraylist  avgt  200  1627,501 ± 126,101  ns/op
ToArrayBench.zero                                   0  arraylist  avgt  200     5,601 ±   0,577  ns/op
ToArrayBench.zero                                   1  arraylist  avgt  200    24,291 ±   7,608  ns/op
ToArrayBench.zero                                  10  arraylist  avgt  200    32,702 ±   6,036  ns/op
ToArrayBench.zero                                 100  arraylist  avgt  200   170,355 ±  20,991  ns/op
ToArrayBench.zero                                1000  arraylist  avgt  200  1489,356 ± 181,910  ns/op

2. Resources

You find the code examples in our Github tutorial example as Maven project.

https://github.com/vogellacompany/tutorials/tree/master/jmh

2.1. vogella Java example code

If you need more assistance we offer Online Training and Onsite training as well as consulting

See License for license information.

Legal Privacy Policy Change consent