1. 項目地址
https://github.com/buger/goreplay
2. 安裝
wget https://github.com/buger/goreplay/releases/download/v0.16.1/gor_0.16.1_x64.tar.gz
tar xvf gor_0.16.1_x64.tar.gz
cp goreplay /usr/bin
3. 簡單使用
// 測試環境使用的是nginx 有連個機器一台進行反向代理，具體的安裝配置比較簡單
a. 進行流量拷貝並輸出到標准輸出
goreplay --input-raw :80 --output-stdout
response message
GET / HTTP/1.1
Host: XXXXXXXXX
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.75 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate
Accept-Language: zh-CN,zh;q=0.9,en;q=0.8
Cache-Control: max-age=0
If-Modified-Since: Sat, 04 Nov 2017 13:52:14 GMT
If-None-Match: W/"59fdc60e-232"
Proxy-Connection: keep-alive
Upgrade-Insecure-Requests: 1
X-Forwarded-For: XXXXXXXXX
X-Lantern-Version: 4.4.0
b. 流量拷貝其他環境
goreplay --input-raw :80 --output-http="http://xxxxx:port"
c. 流量拷貝存儲為本地文件（也可以使用s3存儲，開源的有minio）
copy:
goreplay --input-raw :80 --output-file=requests.gor
replay:
goreplay --input-file requests.gor --output-http="http://XXXXXXXX:port"
4. 可用的插件
a. input
--input-raw - used to capture HTTP traffic, you should specify IP address or
interface and application port. More about Capturing and replaying traffic.
--input-file - accepts file which previously was recorded using --output-file.
More about Saving and Replaying from file
--input-tcp - used by Gor aggregation instance if you decided forward traffic
from multiple forwarder Gor instances to it.
b. output
--output-http - replay HTTP traffic to given endpoint, accepts base url
--output-file - records incoming traffic to the file. More about Saving and Replaying from file
--output-tcp - forward incoming data to another Gor instance, used in conjunction with --input-tcp. Read more about Aggregator-forwarder setup.
--output-stdout - used for debugging, outputs all data to stdout.
5. 擴展
Rate limiting
a. Limiting replay using absolute number
# staging.server will not get more than ten requests per second
gor --input-tcp :28020 --output-http "http://staging.com|10"
b. Limiting listener using percentage based limiter
# replay server will not get more than 10% of requests
# useful for high-load environments
gor --input-raw :80 --output-tcp "replay.local:28020|10%"
c. Consistent limiting based on Header or URL param value
# Limit based on header value
gor --input-raw :80 --output-tcp "replay.local:28020|10%" --http-header-limiter "X-API-KEY: 10%"
# Limit based on header value
gor --input-raw :80 --output-tcp "replay.local:28020|10%" --http-param-limiter "api_key: 10%"
Request filtering
a. Allow url regexp
# only forward requests being sent to the /api endpoint
gor --input-raw :8080 --output-http staging.com --http-allow-url /api
b. Disallow url regexp
# only forward requests NOT being sent to the /api... endpoint
gor --input-raw :8080 --output-http staging.com --http-disallow-url /api
c. Filter based on regexp of header
# only forward requests with an api version of 1.0x
gor --input-raw :8080 --output-http staging.com --http-allow-header api-version:^1\.0\d
# only forward requests NOT containing User-Agent header value "Replayed by Gor"
gor --input-raw :8080 --output-http staging.com --http-disallow-header "User-Agent: Replayed by Gor"
d. Filter based on HTTP method
gor --input-raw :80 --output-http "http://staging.server" \
--http-allow-method GET \
--http-allow-method OPTIONS
6. 參考資料
// wiki
https://github.com/buger/goreplay/wiki
https://github.com/buger/goreplay
// kafka
https://github.com/buger/goreplay/wiki/Streaming-from-and-to-Apache-Kafka
// Distributed-configuration
https://github.com/buger/goreplay/wiki/Distributed-configuration