Webサーバのベンチマークをとるのが趣味になりつつあるmatsumotoryです。
Webサーバのベンチマークについては、abからはじまりwrk等を使っていたのですが、最近ではほぼh2loadを使っています。
h2loadはnghttp2というHTTP/2ライブラリのアプリケーションに含まれているツールですが、
- HTTP/2（SPDYも）とHTTP/1.xに両対応している
- ベンチマーク側の同時スレッド数を増やせる
- TLS及びSNIもサポートしている
- 最小、最大、平均、標準偏差あたりもちゃんとでる
ので、色々プロトコルを変えつつ同じベンチマークツールで、値の目安を出すにはとても重宝しています。
Nghttp2: HTTP/2 C Library - nghttp2.org
実行結果のサンプルは例えば以下、
$ h2load -c 100 -n 10000 https://localhost:58082/
starting benchmark...
spawning thread #0: 100 total client(s). 10000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES128-GCM-SHA256
Server Temp Key: ECDH P-256 256 bits
Application protocol: http/1.1
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done
finished in 300.79ms, 33245.34 req/s, 4.14MB/s
requests: 10000 total, 10000 started, 10000 done, 10000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 10000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 1306800 bytes total, 859500 bytes headers (space savings 0.00%), 110000 bytes data
min max mean sd +/- sd
time for request: 16.04ms 225.26ms 115.41ms 59.69ms 56.07%
time for connect: 49.75ms 101.39ms 56.42ms 5.38ms 87.00%
time to 1st byte: 67.78ms 279.02ms 100.17ms 23.69ms 78.00%
req/s : 343.79 850.22 449.69 204.01 79.00%
ちゃんとHTTP/1.1を検出してくれたり、HTTP/2に対しては、
$ h2load -c 100 -n 10000 https://localhost:8080/index.html
starting benchmark...
spawning thread #0: 100 total client(s). 10000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES128-GCM-SHA256
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done
finished in 2.12s, 4714.33 req/s, 229.61KB/s
requests: 10000 total, 10000 started, 10000 done, 10000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 10000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 498746 bytes total, 95746 bytes headers (space savings 92.09%), 220000 bytes data
min max mean sd +/- sd
time for request: 167.45ms 1.87s 1.12s 553.00ms 63.00%
time for connect: 47.93ms 313.94ms 89.12ms 69.25ms 90.00%
time to 1st byte: 215.82ms 2.11s 1.21s 587.02ms 63.00%
req/s : 47.35 443.00 125.54 108.58 87.00%
というように自動で検知してくれて非常に便利です。今後、TLSやHTTP/2、SPDYとか色々と混在した環境では非常に重宝するかと思います。Unix Domain Socketにも対応していると個人的な用途ではさらに便利だと思っています。
今後はHTTP/1についてもこれでやっていこうかなぁと思います。
$ h2load -h
Usage: h2load [OPTIONS]... [URI]...
benchmarking tool for HTTP/2 and SPDY server
<URI> Specify URI to access. Multiple URIs can be specified.
URIs are used in this order for each client. All URIs
are used, then first URI is used and then 2nd URI, and
so on. The scheme, host and port in the subsequent
URIs, if present, are ignored. Those in the first URI
are used solely. Definition of a base URI overrides all
scheme, host or port values.
Options:
-n, --requests=<N>
Number of requests across all clients. If it is used
with --timing-script-file option, this option specifies
the number of requests each client performs rather than
the number of requests across all clients.
Default: 1
-c, --clients=<N>
Number of concurrent clients. With -r option, this
specifies the maximum number of connections to be made.
Default: 1
-t, --threads=<N>
Number of native threads.
Default: 1
-i, --input-file=<PATH>
Path of a file with multiple URIs are separated by EOLs.
This option will disable URIs getting from command-line.
If '-' is given as <PATH>, URIs will be read from stdin.
URIs are used in this order for each client. All URIs
are used, then first URI is used and then 2nd URI, and
so on. The scheme, host and port in the subsequent
URIs, if present, are ignored. Those in the first URI
are used solely. Definition of a base URI overrides all
scheme, host or port values.
-m, --max-concurrent-streams=<N>
Max concurrent streams to issue per session. When
http/1.1 is used, this specifies the number of HTTP
pipelining requests in-flight.
Default: 1
-w, --window-bits=<N>
Sets the stream level initial window size to (2**<N>)-1.
For SPDY, 2**<N> is used instead.
Default: 30
-W, --connection-window-bits=<N>
Sets the connection level initial window size to
(2**<N>)-1. For SPDY, if <N> is strictly less than 16,
this option is ignored. Otherwise 2**<N> is used for
SPDY.
Default: 30
-H, --header=<HEADER>
Add/Override a header to the requests.
--ciphers=<SUITE>
Set allowed cipher list. The format of the string is
described in OpenSSL ciphers(1).
-p, --no-tls-proto=<PROTOID>
Specify ALPN identifier of the protocol to be used when
accessing http URI without SSL/TLS.
Available protocols: spdy/2, spdy/3, spdy/3.1, h2c and
http/1.1
Default: h2c
-d, --data=<PATH>
Post FILE to server. The request method is changed to
POST.
-r, --rate=<N>
Specifies the fixed rate at which connections are
created. The rate must be a positive integer,
representing the number of connections to be made per
rate period. The maximum number of connections to be
made is given in -c option. This rate will be
distributed among threads as evenly as possible. For
example, with -t2 and -r4, each thread gets 2
connections per period. When the rate is 0, the program
will run as it normally does, creating connections at
whatever variable rate it wants. The default value for
this option is 0.
--rate-period=<DURATION>
Specifies the time period between creating connections.
The period must be a positive number, representing the
length of the period in time. This option is ignored if
the rate option is not used. The default value for this
option is 1s.
-T, --connection-active-timeout=<DURATION>
Specifies the maximum time that h2load is willing to
keep a connection open, regardless of the activity on
said connection. <DURATION> must be a positive integer,
specifying the amount of time to wait. When no timeout
value is set (either active or inactive), h2load will
keep a connection open indefinitely, waiting for a
response.
-N, --connection-inactivity-timeout=<DURATION>
Specifies the amount of time that h2load is willing to
wait to see activity on a given connection. <DURATION>
must be a positive integer, specifying the amount of
time to wait. When no timeout value is set (either
active or inactive), h2load will keep a connection open
indefinitely, waiting for a response.
--timing-script-file=<PATH>
Path of a file containing one or more lines separated by
EOLs. Each script line is composed of two tab-separated
fields. The first field represents the time offset from
the start of execution, expressed as a positive value of
milliseconds with microsecond resolution. The second
field represents the URI. This option will disable URIs
getting from command-line. If '-' is given as <PATH>,
script lines will be read from stdin. Script lines are
used in order for each client. If -n is given, it must
be less than or equal to the number of script lines,
larger values are clamped to the number of script lines.
If -n is not given, the number of requests will default
to the number of script lines. The scheme, host and
port defined in the first URI are used solely. Values
contained in other URIs, if present, are ignored.
Definition of a base URI overrides all scheme, host or
port values.
-B, --base-uri=<URI>
Specify URI from which the scheme, host and port will be
used for all requests. The base URI overrides all
values defined either at the command line or inside
input files.
--npn-list=<LIST>
Comma delimited list of ALPN protocol identifier sorted
in the order of preference. That means most desirable
protocol comes first. This is used in both ALPN and
NPN. The parameter must be delimited by a single comma
only and any white spaces are treated as a part of
protocol string.
Default: h2,h2-16,h2-14,spdy/3.1,spdy/3,spdy/2,http/1.1
--h1 Short hand for --npn-list=http/1.1
--no-tls-proto=http/1.1, which effectively force
http/1.1 for both http and https URI.
-v, --verbose
Output debug information.
--version Display version information and exit.
-h, --help Display this help and exit.
--
The <DURATION> argument is an integer and an optional unit (e.g., 1s
is 1 second and 500ms is 500 milliseconds). Units are h, m, s or ms
(hours, minutes, seconds and milliseconds, respectively). If a unit
is omitted, a second is used as unit.
