JMH 使用指南 - java 性能測試

JMH 篇

JMH，即Java Microbenchmark Harness 翻譯：java 微基准測試 工具套件。
## 1.添加依賴
```
<dependency>
   <groupId>org.openjdk.jmh</groupId>
   <artifactId>jmh-core</artifactId>
   <version>1.19</version>
   <scope>provided</scope>
</dependency>
<dependency>
   <groupId>org.openjdk.jmh</groupId>
   <artifactId>jmh-generator-annprocess</artifactId>
   <version>1.19</version>
   <scope>provided</scope>
</dependency>
```
## 2.第一個例子
請參加： JMHFirstBenchmark.java

請參加(晉級)： SecondBenchmark.java
請參加(晉級)： ThirdBenchmark.java

## 3.常用注解說明 

###3.1 @BenchmarkMode(Mode.All)

Mode有：- Throughput: 整體吞吐量，例如“1秒內可以執行多少次調用” (thrpt,參加第5點)
- AverageTime: 調用的平均時間，例如“每次調用平均耗時xxx毫秒”。（avgt）
- SampleTime: 隨機取樣，最后輸出取樣結果的分布，例如“99%的調用在xxx毫秒以內，99.99%的調用在xxx毫秒以內”（simple）
- SingleShotTime: 以上模式都是默認一次 iteration 是 1s，唯有 SingleShotTime 是只運行一次。往往同時把 warmup 次數設為0，用於測試冷啟動時的性能。（ss）

### 3.2 @OutputTimeUnit(TimeUnit.MILLISECONDS)

統計單位， 微秒、毫秒 、分、小時、天
### 3.3 @State

 可參：JMHFirstBenchmark.java
 
類注解，JMH測試類必須使用@State注解，State定義了一個類實例的生命周期，可以類比Spring Bean的Scope。由於JMH允許多線程同時執行測試，不同的選項含義如下：

```
Scope.Thread：默認的State，每個測試線程分配一個實例；
Scope.Benchmark：所有測試線程共享一個實例，用於測試有狀態實例在多線程共享下的性能；
Scope.Group：每個線程組共享一個實例；
```


### 3.4 @Benchmark 
很重要的方法注解，表示該方法是需要進行 benchmark 的對象。和@test 注解一致
### 3.5 @Setup 

方法注解，會在執行 benchmark 之前被執行，正如其名，主要用於初始化。
### 3.6 @TearDown (Level)


方法注解，與@Setup 相對的，會在所有 benchmark 執行結束以后執行，主要用於資源的回收等。
   (Level)   用於控制 @Setup，@TearDown 的調用時機，默認是 Level.Trial。
          
          Trial：每個benchmark方法前后；
          Iteration：每個benchmark方法每次迭代前后；
          Invocation：每個benchmark方法每次調用前后，謹慎使用，需留意javadoc注釋；  
### 3.7 @Param
@Param注解接收一個String數組 ，
可以用來指定某項參數的多種情況。特別適合用來測試一個函數在不同的參數輸入的情況下的性能。
 可參：JMHFirstBenchmark.java

## 4 Options常用選項
### 4.1 include 

benchmark 所在的類的名字，這里可以使用正則表達式對所有類進行匹配。
參考：SecondBenchmark.java

### 4.2 fork 
JVM因為使用了profile-guided optimization而“臭名昭著”，這對於微基准測試來說十分不友好，因為不同測試方法的profile混雜在一起，“互相傷害”彼此的測試結果。對於每個@Benchmark方法使用一個獨立的進程可以解決這個問題，這也是JMH的默認選項。注意不要設置為0，設置為n則會啟動n個進程執行測試（似乎也沒有太大意義）。
fork選項也可以通過方法注解以及啟動參數來設置。

### 4.3 warmupIterations
預熱次數，每次默認1秒。

### 4.4 measurementIterations 
實際測量的迭代次數，每次默認1秒。
 
### 4.5 Group 
方法注解，可以把多個 benchmark 定義為同一個 group，則它們會被同時執行，譬如用來模擬生產者－消費者讀寫速度不一致情況下的表現。

### 4.6 Threads 
每個fork進程使用多少條線程去執行你的測試方法，默認值是Runtime.getRuntime().availableProcessors()。 
 
 
   
## 5 輸出結果  
```

# @BenchmarkMode(Mode.All)
# JMH version: 1.19
# VM version: JDK 1.7.0_80, VM 24.80-b11
# VM invoker: C:\Program Files\Java\jdk1.7.0_80\jre\bin\java.exe
# VM options: -javaagent:D:\Program Files\JetBrains\IntelliJ IDEA 2018.1\lib\idea_rt.jar=51664:D:\Program Files\JetBrains\IntelliJ IDEA 2018.1\bin -Dfile.encoding=UTF-8
# Warmup: 2 iterations, single-shot each
# Measurement: 2 iterations, single-shot each
# Timeout: 10 min per iteration
# Threads: 10 threads
# Benchmark mode: Single shot invocation time
# Benchmark: com.gemantic.wealth.yunmatong.service.jmh.SecondBenchmark.singleThreadBench
# Parameters: (length = 100000)

# Run progress: 99.98% complete, ETA 00:00:00
# Fork: 1 of 1
# Warmup Iteration   1: 34.641 ±(99.9%) 33.844 ms/op
# Warmup Iteration   2: 7.129 ±(99.9%) 9.238 ms/op
Iteration   1: 7.573 ±(99.9%) 4.581 ms/op
Iteration   2: 6.235 ±(99.9%) 4.150 ms/op



# Run complete. Total time: 00:00:36

Benchmark                                                    (length)    Mode     Cnt    Score    Error   Units
SecondBenchmark.multiThreadBench                               100000   thrpt       2  147.758           ops/ms    
SecondBenchmark.singleThreadBench                              100000   thrpt       2    0.983           ops/ms
SecondBenchmark.multiThreadBench                               100000    avgt       2    0.068            ms/op
SecondBenchmark.singleThreadBench                              100000    avgt       2   10.510            ms/op
SecondBenchmark.multiThreadBench                               100000  sample  295532    0.068 ±  0.001   ms/op
SecondBenchmark.multiThreadBench:multiThreadBench·p0.00        100000  sample            0.010            ms/op
SecondBenchmark.multiThreadBench:multiThreadBench·p0.50        100000  sample            0.066            ms/op
SecondBenchmark.multiThreadBench:multiThreadBench·p0.90        100000  sample            0.095            ms/op
SecondBenchmark.multiThreadBench:multiThreadBench·p0.95        100000  sample            0.104            ms/op
SecondBenchmark.multiThreadBench:multiThreadBench·p0.99        100000  sample            0.126            ms/op
SecondBenchmark.multiThreadBench:multiThreadBench·p0.999       100000  sample            0.172            ms/op
SecondBenchmark.multiThreadBench:multiThreadBench·p0.9999      100000  sample            1.729            ms/op
SecondBenchmark.multiThreadBench:multiThreadBench·p1.00        100000  sample            4.309            ms/op
SecondBenchmark.singleThreadBench                              100000  sample    2036   10.196 ±  0.581   ms/op
SecondBenchmark.singleThreadBench:singleThreadBench·p0.00      100000  sample            6.201            ms/op
SecondBenchmark.singleThreadBench:singleThreadBench·p0.50      100000  sample            8.020            ms/op
SecondBenchmark.singleThreadBench:singleThreadBench·p0.90      100000  sample           10.355            ms/op
SecondBenchmark.singleThreadBench:singleThreadBench·p0.95      100000  sample           38.443            ms/op
SecondBenchmark.singleThreadBench:singleThreadBench·p0.99      100000  sample           41.943            ms/op
SecondBenchmark.singleThreadBench:singleThreadBench·p0.999     100000  sample           73.498            ms/op
SecondBenchmark.singleThreadBench:singleThreadBench·p0.9999    100000  sample           74.973            ms/op
SecondBenchmark.singleThreadBench:singleThreadBench·p1.00      100000  sample           74.973            ms/op
SecondBenchmark.multiThreadBench                               100000      ss       2    0.223            ms/op
SecondBenchmark.singleThreadBench                              100000      ss       2    6.904            ms/op

```

6.第一個例子

package com.gemantic.wealth.yunmatong.service.jmh;

import lombok.extern.slf4j.Slf4j;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import org.openjdk.jmh.runner.options.TimeValue;

import java.util.concurrent.TimeUnit;

@Slf4j
@BenchmarkMode(Mode.AverageTime)// 測試方法平均執行時間
@OutputTimeUnit(TimeUnit.MICROSECONDS)// 輸出結果的時間粒度為微秒
@State(Scope.Benchmark) // 每個測試線程一個實例
public class JMHFirstBenchmark {
    /*
     * Most of the time, you need to maintain some state while the benchmark is
     * running. Since JMH is heavily used to build concurrent benchmarks, we
     * opted for an explicit notion of state-bearing objects.
     *
     * Below are two state objects. Their class names are not essential, it
     * matters they are marked with @State. These objects will be instantiated
     * on demand, and reused during the entire benchmark trial.
     *
     * The important property is that state is always instantiated by one of
     * those benchmark threads which will then have the access to that state.
     * That means you can initialize the fields as if you do that in worker
     * threads (ThreadLocals are yours, etc).
     */

    @State(Scope.Benchmark)
    public static class BenchmarkState {
        volatile double x = Math.PI;
    }

    @State(Scope.Thread)
    public static class ThreadState {
        volatile double x = Math.PI;
    }

    @Benchmark
    public void measureUnshared(ThreadState state) {
        // All benchmark threads will call in this method.
        //
        // However, since ThreadState is the Scope.Thread, each thread
        // will have it's own copy of the state, and this benchmark
        // will measure unshared case.
        state.x++;
        try {
            Thread.sleep(500);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        log.info("measureUnshared:"+ state.x);
    }

    @Benchmark
    public void measureShared(BenchmarkState state) {
        // All benchmark threads will call in this method.
        //
        // Since BenchmarkState is the Scope.Benchmark, all threads
        // will share the state instance, and we will end up measuring
        // shared case.
        state.x++;
        try {
            Thread.sleep(500);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        log.info("measureShared:"+ state.x);
    }

    /*
     * ============================== HOW TO RUN THIS TEST: ====================================
     *
     * You are expected to see the drastic difference in shared and unshared cases,
     * because you either contend for single memory location, or not. This effect
     * is more articulated on large machines.
     *
     * You can run this test:
     *
     * a) Via the command line:
     *    $ mvn clean install
     *    $ java -jar target/benchmarks.jar JMHSample_03 -wi 5 -i 5 -t 4 -f 1
     *    (we requested 5 measurement/warmup iterations, with 4 threads, single fork)
     *
     * b) Via the Java API:
     *    (see the JMH homepage for possible caveats when running from IDE:
     *      http://openjdk.java.net/projects/code-tools/jmh/)
     */

    public static void main(String[] args) throws RunnerException {
        // 可以通過注解
        Options opt = new OptionsBuilder()
                .include(JMHFirstBenchmark.class.getSimpleName())
                .warmupIterations(3) // 預熱3次
                .measurementIterations(2).measurementTime(TimeValue.valueOf("1s")) // 運行5次，每次10秒
                .threads(10) // 10線程並發
                .forks(2)
                .build();

        new Runner(opt).run();
    }

}

　　

第二個例子setup&TearDown&Param

 

package com.gemantic.wealth.yunmatong.service.jmh;

import com.gemantic.wealth.yunmatong.service.jmh.service.Calculator;
import com.gemantic.wealth.yunmatong.service.jmh.service.MultithreadCalculator;
import com.gemantic.wealth.yunmatong.service.jmh.service.SinglethreadCalculator;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;

import java.util.concurrent.TimeUnit;

@BenchmarkMode(Mode.All)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@State(Scope.Benchmark)
public class SecondBenchmark {
    @Param({"100000"})
    private int length;
 
    private int[] numbers;
    private Calculator singleThreadCalc;
    private Calculator multiThreadCalc;
 
    public static void main(String[] args) throws RunnerException {
        Options opt = new OptionsBuilder()
                .include(SecondBenchmark.class.getSimpleName()) // .include("JMHF.*") 可支持正則
                .forks(0)
                .warmupIterations(2)
                .measurementIterations(2).threads(10)
                .build();
 
        new Runner(opt).run();
    }

    @Benchmark
    public long singleThreadBench() {
        return singleThreadCalc.sum(numbers);
    }
 
    @Benchmark
    public long multiThreadBench() {
        return multiThreadCalc.sum(numbers);
    }
 
    @Setup(Level.Trial)
    public void prepare() {
        int n = length;
        numbers =new int[n];
        for (int i=0;i<n;i++){
            numbers[i]=i;
        }
        singleThreadCalc = new SinglethreadCalculator();
        multiThreadCalc = new MultithreadCalculator(Runtime.getRuntime().availableProcessors());
    }


    @TearDown
    public void shutdown() {
        singleThreadCalc.shutdown();
        multiThreadCalc.shutdown();
    }
}

　　

第三個例子group

 

 

package com.gemantic.wealth.yunmatong.service.jmh;

import com.gemantic.wealth.yunmatong.service.jmh.service.Calculator;
import com.gemantic.wealth.yunmatong.service.jmh.service.MultithreadCalculator;
import com.gemantic.wealth.yunmatong.service.jmh.service.SinglethreadCalculator;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import org.openjdk.jmh.runner.options.TimeValue;

import java.util.concurrent.TimeUnit;

@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@State(Scope.Benchmark)
public class ThirdBenchmark {

    @State(Scope.Group)
    public static class BenchmarkState {
        volatile double x = Math.PI;
    }

    @Benchmark
    @Group("custom")
    @GroupThreads(10)
    public void read(BenchmarkState state) {
        state.x++;
        try {
            Thread.sleep(5);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        System.out.println("ThirdBenchmark.read: "+ state.x);
    }

    @Benchmark
    @Group("custom")
    public void book(BenchmarkState state) {
        state.x++;
        try {
            Thread.sleep(5);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        System.out.println("ThirdBenchmark.book: "+ state.x);
    }


    public static void main(String[] args) throws RunnerException {
        Options opt = new OptionsBuilder()
                .include(ThirdBenchmark.class.getSimpleName()) // .include("JMHF.*") 可支持正則
                .forks(0)
                .warmupIterations(0)
                .measurementIterations(2).measurementTime(TimeValue.valueOf("10ms")).threads(5)
                .build();

        new Runner(opt).run();
    }
}