1、ltp-ddt簡介
1.1、ltp
LTP(Linux Test Project)是由SGI，OSDL和Bull發起的聯合項目，由IBM，思科，富士通，SUSE，紅帽，甲骨文等公司開發和維護。該項目的目標是向開源社區提供測試，以驗證Linux的可靠性，健壯性和穩定性。
LTP測試套件包含一系列用於測試Linux內核和相關功能的工具。目標是通過將測試自動化帶入測試工作來改進Linux內核和系統庫。
從代碼目錄架構上看，LTP分為以下重要部分：
- 1、testcase目錄:
該目錄包含測試用例的具體實現文件(test logic files)，格式為：C代碼、shell腳本。
ltp-ddt$ ls testcases/
commands cve ddt kdump kernel lib Makefile misc network open_posix_testsuite realtime
ltp-ddt$ ls testcases/kernel/
connectors containers controllers device-drivers firmware fs hotplug include input io ipc lib logging Makefile mem module numa power_management pty sched security syscalls timers tracing
ltp-ddt$ ls testcases/kernel/syscalls/
abort clock_nanosleep execl flock getgroups
accept clock_nanosleep2 execle fmtmsg gethostbyname_r
accept4 clone execlp fork gethostid
access close execv fpathconf gethostname
acct cma execve fstat getitimer
add_key confstr execvp fstatat get_mempolicy
adjtimex connect exit fstatfs getpagesize
alarm copy_file_range exit_group fsync getpeername
...
- 2、runtest目錄：
LTP測試框架不會直接調用testcase，而是通過一個中間包裝腳本測試場景文件(test scenario files)來調用，放在runtest路徑下面。一般一個testcase對應一個runtest文件。
ltp-ddt$ ls runtest/
admin_tools controllers dma_thread_diotest fs_perms_simple input ltp-aiodio.part1 ltplite
can cpuhotplug fcntl-locktests fs_readonly io ltp-aiodio.part2 lvm.part1
...
ltp-ddt$
ltp-ddt$ cat runtest/syscalls
#DESCRIPTION:Kernel system calls
abort01 abort01
accept01 accept01
accept4_01 accept4_01
access01 access01
access02 access02
access03 access03
access04 access04
...
默認運行哪些runtest在scenario_groups/default文件中定義：
ltp-ddt$ ls scenario_groups/
default default-ddt Makefile network
ltp-ddt$
ltp-ddt$ cat scenario_groups/default
syscalls
fs
fs_perms_simple
fsx
dio
io
...
- 3、runltp命令：
runltp是運行ltp測試的總入口命令。
編譯、安裝完ltp以后，使用以下命令運行：
$ ./runltp
也可以單獨的運行某個testcase集：
$ ./runltp -f syscalls
也可以直接調用某個testcase的bin文件：
$ testcases/bin/fork13 -i 37
- 4、其他特性：
1.2、ltp-ddt
LTP-DDT是德州儀器（TI）用於驗證Linux版本的測試應用程序。它基於LTP。
LTP驗證許多內核區域，例如內存管理，調度程序和系統調用。LTP-DDT通過測試來擴展LTP的核心內核測試，以驗證德州儀器開發的內核驅動程序。LTP-DDT專注於嵌入式設備驅動程序測試。它包含數百個驗證設備驅動程序功能和性能的測試。LTP-DDT還包含用於驗證系統用例和整體系統穩定性的測試。
LTP-DDT在LTP的基礎之上進行了一系列的擴展：
- 1、testcase目錄:
ltp-ddt在testcase目錄當中增加了ddt目錄：
ltp-ddt$ ls testcases/ddt/
alsa_test_suite dmtimer_test_suite fbdev_display_test_suite gpio_test_suite ipc_test_suite rtc_test_suite spidev_test_suite utils v4l2_display_davinci_test_suite wdt_test_suite
devfreq_drivers edma_test_suite filesystem_test_suite i2c_test_suite Makefile scripts usb_test_suite v4l2_capture_test_suite v4l2_display_test_suite
testcase除了支持C代碼和shell腳本，還支持python腳本。python腳本放在ddt/scripts目錄當中，以及對應的模板文件testcases/ddt/scripts/TEMPLATE：：
ltp-ddt$ ls testcases/ddt/scripts/
adc alsa blk can clk_fw common crypto dma dmtimer drm eth gpio graphics i2c ipc Makefile mtd pci powermgr pwm rng rtc scsi TEMPLATE thermal touchscreen uart usb video wlan
- 2、runtest目錄：
ltp-ddt在runtest目錄當中增加了ddt目錄：
ltp-ddt$ ls runtest/ddt/
adc_ti gpio_rotaryencoder nand_mtd_dd_write_concurrence_ubifs powermgr_suspend_emmc realtime_v4l2cap_load system_eth_smp_priority_processing
alsa_accesstype gpio-test-kerneldebug nand_mtd_debug_erase powermgr_suspend_ethernet realtime_v4l2cap_load-smp system_gpio_cpufreq
alsa_amixer_switchtoggle gpio-tests nand_mtd_debug_rw powermgr_suspend_graphics remoteproc_r5f system_graphics_cpufreq
alsa_amixer_volumesetting gst_capture nand_mtd_flash_erase powermgr_suspend_i2c rng_test system_gst_cpufreq
...
測試場景文件(test scenario files)增加了以下注釋關鍵字：@name，@desc，@requires和@setup_requires。
@requires和@setup_requires用於根據PLATFORM功能在運行時選擇測試用例。
測試場景文件(test scenario files)具體的格式如下：
// 正式部分，每一行對應一個測試步驟，一個測試步驟的格式為：<TAG> <COMMANDS>
* Follow LTP guidelines. The test scenario file is basically made of one or
more test step lines. Each test step line have following format
<TAG> <COMMANDS>, where
TAG is a string that identifies the test step.
Use following convention to named TAGs so that the test cases can be
selectively run based on AREA, SCOPE and/or TYPE.
<AREA>_<SCOPE>_<TYPE>_<OPT_ID>,
i.e. “NAND_S_FUNC_RW_8K”, “NAND_M_PERF_ALL-SIZES”
The SCOPE tags are used to indicate the amount of time require to run
the tests, giving users ability to filter test cases based on estimated
execution time.
SCOPE TAGS:
'XS', 'S', 'M', 'L', 'XL', 'XXL' (for eXtra Small, Small, Medium, etc.
Just imagine you are buying clothes ;)
We used the following rough guidelines to determine test duration based
on scope:
XS: <= 1 min
S: <= 10 mins
M: <= 1 hour
L: <= 8 hours
XL: <= 24 hours
XXL: <= 1 week
TYPE TAGS:
‘FUNC’, ‘PERF’, ‘STRESS’, ‘USECASE’, ‘COMPLIANCE’, ‘MODULAR’, ‘DOC’
COMMANDS is a list of one or more shell commands separated by semicolon (;),
the test step will pass if the commands return zero, otherwise it fails.
// 默認的模板文件為runtest/ddt/TEMPLATE
* Use the default test scenario file template available at
runtest/ddt/TEMPLATE as a starting point to develop your test scenario.
// 注釋部分，可以使用@requires關鍵字來限定測試需要的ARCH, DRIVER, SOC and/or MACHINE等條件。
* Use the @requires annotation to specify ARCH, DRIVER, SOC and/or MACHINE
requirements to run the test scenario. You can use (), &&, ||, * to specify
the test requirements. Examples:
@requires /net/eth/* && spi_master
To run this test the platform must have an ethernet driver and a
spi_master driver
@requires am3517-evm
This test can only be run on an am3517 EVM.
@requires (mmc_host || nand) && armv*
This test requires mmc or nand drivers and an ARM architecture
// 注釋部分，可以使用@setup_requires來描述測試需要的外部條件，比如連接外部usb設備等等。
* Use the @setup_requires annotation to specify test setup requirements.
Some test cases like USB and Video capture requires special peripherals, such as
USB flash drives, DVD players, video cameras, etc., to be connected to the DUT.
Using @setup_requires the test developer highlights such test setup requirements.
This information might be used by test automation frameworks to allocate test requests
to DUTs that have the appropriate peripherals connected to them.
Please follow the naming conventions identified in section 9) of this document
if the test scenario needs to identify any setup requirements.
You can use underscore(_) to seperate multiple @setup_requires. Examples:
@setup_requires usbhostvideo_usbhostaudio
This test requires usbhostvideo setup and usbhostaudio setup.
ltp-ddt對應新增了默認運行文件scenario_groups/default-ddt：
ltp-ddt$ ls scenario_groups/
default default-ddt Makefile network
ltp-ddt$
ltp-ddt$ cat scenario_groups/default-ddt
ddt/alsa_samplerate
ddt/clcd
ddt/edma
ddt/edma_chain
ddt/edma_link
ddt/emmc_quick_perf
...
- 3、platforms目錄：
ltp-ddt完全新增了一個文件夾platforms/：
ltp-ddt$ ls platforms/
am170x-evm am335x-hsevm am37x-evm am437x-sk am571x-idk am57xx-beagle-x15 am654x-idk da830-omapl137-evm dm385-evm dra71x-hsevm dra76x-evm hikey k2g-hsevm k2l-evm omap5-evm tci6614-evm
...
ltp-ddt$ cat platforms/am335x-evm
armv7l
am335x
am335x-evm
adc/tscadc
can
crypto/crypto-omap
...
該文件根據具體平台設置了一組配置，在測試的時候可以使用這組配置去挑選當前平台能支持的testcase。
平台配置文件的格式如下：
// 默認的模板文件：platforms/TEMPLATE
* Copy the default platform file available at platforms/TEMPLATE to
platforms/<your platform>. <your platform> name is typically the evm name
// 在編寫ltp-ddt腳本時，請使用以下的平台文件名稱與$ MACHINE進行比較
* Please use the following names for platform files and to compare against
$MACHINE when writing ltp-ddt scripts:
am180x-evm arago-armv7 dm355-evm dm814x-evm
am181x-evm dm365-evm dm816x-evm
am3517-evm beagleboard dm368-evm
am37x-evm c6a814x-evm dm37x-evm omap3evm
am387x-evm c6a816x-evm dm6446-evm tnetv107x-evm
am389x-evm da830-omapl137-evm dm6467-evm am335x-evm
arago-armv5 da850-omapl138-evm dm6467t-evm beaglebone
// 文件格式：
// 平台文件前3行必須是： architecture, SoC and EVM
// 后面每一行對應一個驅動，驅動名符合`/sys/class`文件層次下的命名
* Modify your platform file based on the capabilities supported by the new evm
The platform file identifies the architecture, the SoC, the evm and the
supported drivers. The supported drivers lines follow a variation of the
hierarchy used in /sys/class but it is not exactly the same. Hence it is
important to use the platforms/TEMPLATE file as your starting point.
Please note the first 3 lines of the platform file MUST identify, the
architecture, SoC and EVM respectively, follow by one or more driver lines.
Typically the architecture and machine name used in the platform file are
the ones reported by uname -a.
Sample platform file:
armv7l
am3517
am3517-evm
net/eth/davinci_emac
nand/omap2-nand
ehci/ehci-omap
i2c-adapter/i2c_omap
mmc_host/mmci-omap-hs
rtc/rtc-s35390a
watchdog/omap_wdt
...
* You might need to define new override values for your new platform in some
test case files (see section 4.1 above for details). A reasonable strategy
is to try to run an existing test plan and then analyze the test failures
to determine probable test cases where you need to define override values.
- 4、runltp命令：
ltp-ddt的runltp命令可以使用平台文件來定義需要運行的testcase。
運行制定平台的所有testcase：
./runltp -P am335x-evm
運行指定平台的指定testcase：
./runltp -P am335x-evm -f ddt/lmbench
2、環境構造
2.1、交叉編譯
ltp-ddt的交叉編譯：
linux-3.2.0$ make CROSS_COMPILE=arm-linux-gnueabihf- ARCH=arm headers_install
CHK include/linux/version.h
INSTALL include/linux/mmc (1 file)
INSTALL include/linux/netfilter/ipset (4 files)
INSTALL include/linux/netfilter (69 files)
INSTALL include/linux/netfilter_arp (2 files)
INSTALL include/linux/netfilter_bridge (18 files)
INSTALL include/linux/netfilter_ipv4 (15 files)
INSTALL include/linux/netfilter_ipv6 (11 files)
INSTALL include/linux/nfsd (4 files)
INSTALL include/linux/raid (2 files)
INSTALL include/linux/spi (1 file)
INSTALL include/linux/sunrpc (1 file)
INSTALL include/linux/tc_act (7 files)
INSTALL include/linux/tc_ematch (4 files)
INSTALL include/linux/usb (10 files)
INSTALL include/linux/wimax (1 file)
INSTALL include/linux (366 files)
INSTALL include/mtd (5 files)
INSTALL include/rdma (6 files)
INSTALL include/scsi/fc (4 files)
INSTALL include/scsi (3 files)
INSTALL include/sound (8 files)
INSTALL include/video (3 files)
INSTALL include/xen (2 files)
INSTALL include (0 file)
INSTALL include/asm (32 files)
linux-3.2.0$ pwd
~/linux-3.2.0
ltp-ddt$ make autotools
sed -n '1{s:LTP-:m4_define([LTP_VERSION],[:;s:$:]):;p;q}' VERSION > m4/ltp-version.m4
aclocal -I m4
autoconf
autoheader
automake -c -a
configure.ac:20: installing './compile'
configure.ac:18: installing './config.guess'
configure.ac:18: installing './config.sub'
configure.ac:4: installing './install-sh'
configure.ac:4: installing './missing'
make -C testcases/realtime autotools
make[1]: 正在進入目錄 `~/ltp-ddt/testcases/realtime'
aclocal -I ~/ltp-ddt/testcases/realtime/m4
autoconf
autoheader
autoheader
automake -c -a
configure.ac:9: installing './compile'
configure.ac:14: installing './config.guess'
configure.ac:14: installing './config.sub'
configure.ac:4: installing './install-sh'
configure.ac:4: installing './missing'
make[1]:正在離開目錄 `~/ltp-ddt/testcases/realtime'
ltp-ddt$
ltp-ddt$ export CROSS_COMPILER=arm-linux-gnueabihf-
ltp-ddt$ export CC=${CROSS_COMPILER}gcc
ltp-ddt$ export LD=${CROSS_COMPILER}ld
ltp-ddt$ export AR=${CROSS_COMPILER}ar
ltp-ddt$ export STRIP=${CROSS_COMPILER}strip
ltp-ddt$ export RANLIB=${CROSS_COMPILER}ranlib
./configure --host=arm-linux-gnueabihf
./configure --host=arm-linux-gnueabihf CC=arm-linux-gnueabihf-gcc LD=arm-linux-gnueabihf-ld AR=arm-linux-gnueabihf-ar STRIP=arm-linux-gnueabihf-strip RANLIB=arm-linux-gnueabihf-ranlib
ltp-ddt$ make SKIP_IDCHECK=1 KERNEL_USR_INC=~/linux-3.2.0/usr/include/ CROSS_COMPILE=arm-linux-gnueabihf- clean
ltp-ddt$ make SKIP_IDCHECK=1 KERNEL_USR_INC=~/linux-3.2.0/usr/include/ CROSS_COMPILE=arm-linux-gnueabihf-
make DESTDIR=~/rootfs_yaff2/home/root/ltp-ddt-install SKIP_IDCHECK=1 PLATFORM=am335x-evm install
ltp的交叉編譯過程和ltp-ddt一樣。
2.2、文件系統
sudo mkyaffs2image rootfs_yaff2/ ubi.img
交叉編譯sysstat：
./configure --host=arm-linux-gnueabihf --cache-file=~/sysstat-12.0.5/cache_file_0 --prefix=~/sysstat_install/ --exec-prefix=~/sysstat_install/
make
make install
交叉編譯busybox：
1、配置
$ make menuconfig
1.1、靜態編譯：
Build Options --->
通過空格鍵使能Build BusyBox as a static binary（no shared libs）（對應的方括號處顯示星號*即為使能狀態）。
1.2、設置交叉工具鏈：
Build Options --->
設置Cross Compiler prefix如下所示（具體路徑和交叉工具鏈前綴根據實際情況修改）：
/usr/bin/arm-linux-gnueabihf-
1.3、安裝路徑設置：
Busybox Settings --->
Installation Options("make install" behavior) --->
輸入安裝路徑即可：
~/busybox_install/
1.4、不包含/usr目錄：
Busybox Settings --->
General Configuration --->
[*]Don't use /usr
這個必需選中，否則在安裝的時候將會將BusyBox安裝在Ubuntu的/usr目錄中，從而損壞了宿主機的根文件系統！
2、編譯&安裝
$ make
$ make install
3、測試運行
./runltp -P am335x-evm -f syscalls
./runltp -P am335x-evm -f ddt/lmbench
./runltp -P am335x-evm -f nandtest
./runltp -P am335x-evm -f uart
example: runltp -c 2 -i 2 -m 2,4,10240,1 -D 2,10,10240,1 -p -q -l /tmp/result-log.1879 -o /tmp/result-output.1879 -C /tmp/result-failed.1879 -d /home/root/sda1/ltp-ddt-install/opt/ltp