perf 是一個調查 Linux 中各種性能問題的有力工具。
# perf --help
usage: perf [--version] [--help] COMMAND [ARGS]
The most commonly used perf commands are:
annotate Read perf.data (created by perf record) and display annotated code
archive Create archive with object files with build-ids found in perf.data file
bench General framework for benchmark suites
buildid-cache Manage build-id cache.
buildid-list List the buildids in a perf.data file
diff Read two perf.data files and display the differential profile
evlist List the event names in a perf.data file
inject Filter to augment the events stream with additional information
kmem Tool to trace/measure kernel memory(slab) properties
kvm Tool to trace/measure kvm guest os
list List all symbolic event types
lock Analyze lock events
record Run a command and record its profile into perf.data
report Read perf.data (created by perf record) and display the profile
sched Tool to trace/measure scheduler properties (latencies)
script Read perf.data (created by perf record) and display trace output
stat Run a command and gather performance counter statistics
test Runs sanity tests.
timechart Tool to visualize total system behavior during a workload
top System profiling tool.
trace strace inspired tool
probe Define new dynamic tracepoints
annotate Read perf.data (created by perf record) and display annotated code
archive Create archive with object files with build-ids found in perf.data file
bench General framework for benchmark suites
buildid-cache Manage build-id cache.
buildid-list List the buildids in a perf.data file
diff Read two perf.data files and display the differential profile
evlist List the event names in a perf.data file
inject Filter to augment the events stream with additional information
kmem Tool to trace/measure kernel memory(slab) properties
kvm Tool to trace/measure kvm guest os
list List all symbolic event types
lock Analyze lock events
record Run a command and record its profile into perf.data
report Read perf.data (created by perf record) and display the profile
sched Tool to trace/measure scheduler properties (latencies)
script Read perf.data (created by perf record) and display trace output
stat Run a command and gather performance counter statistics
test Runs sanity tests.
timechart Tool to visualize total system behavior during a workload
top System profiling tool.
trace strace inspired tool
probe Define new dynamic tracepoints
See 'perf help COMMAND' for more information on a specific command.
perf 包含了：perf-stat (perf stat), perf-top (perf top), perf-record (perf record), perf-list (perf list)
perf list 主要是用於列出有哪些可用的event，可以供 perf top -e eventname 來分析。
perf top -e xxx
上面的 perf list [hw|sw...] 可以知道所有的 tracepoint events, 接下來我們就可以使用 perf top -e tracepoint-event 來專門獲得指定的tracepoint的信息：
上面的 perf list [hw|sw...] 可以知道所有的 tracepoint events, 接下來我們就可以使用 perf top -e tracepoint-event 來專門獲得指定的tracepoint的信息：
perf top常用選項有：
-e <event>：指明要分析的性能事件。
-p <pid>：Profile events on existing Process ID (comma sperated list). 僅分析目標進程及其創建的線程。
-k <path>：Path to vmlinux. Required for annotation functionality. 帶符號表的內核映像所在的路徑。
-K：不顯示屬於內核或模塊的符號。
-U：不顯示屬於用戶態程序的符號。
-d <n>：界面的刷新周期，默認為2s，因為perf top默認每2s從mmap的內存區域讀取一次性能數據。
-g：得到函數的調用關系圖。
# perf top -p `pidof mysqld`
Samples: 221 of event 'cycles', Event count (approx.): 10810377, UID: mysql
10.34% mysqld [.] my_strnncollsp_utf8
5.70% [kernel] [k] _spin_lock_irqsave
5.66% libc-2.12.so [.] __memset_sse2
5.55% [kernel] [k] remove_wait_queue
5.16% libc-2.12.so [.] __GI___strcmp_ssse3
4.36% [kernel] [k] __audit_syscall_exit
3.39% [kernel] [k] lookup_ioctx
3.28% [kernel] [k] find_next_bit
3.22% mysqld [.] lex_one_token(YYSTYPE*, THD*)
2.69% mysqld [.] Item::val_bool()
2.60% [kernel] [k] __do_softirq
2.53% mysqld [.] Protocol::send_result_set_row(List<Item>*)
2.52% [kernel] [k] local_bh_enable_ip
2.51% mysqld [.] my_strnncoll_binary
2.51% libc-2.12.so [.] memcpy
第一列：符號引發的性能事件的比例，默認指占用的cpu周期比例。
第二列：符號所在的DSO(Dynamic Shared Object)，可以是應用程序、內核、動態鏈接庫、模塊。
第三列：DSO的類型。[.]表示此符號屬於用戶態的ELF文件，包括可執行文件與動態鏈接庫)。[k]表述此符號屬於內核或模塊。
第四列：符號名。有些符號不能解析為函數名，只能用地址表示。