在MPSoC單板上運行Docker
目錄
作者
Hank FU 付漢傑 hankf@xilinx.com
測試環境
Xilinx ZCU106 單板
Xilinx VCU TRD2020.1
Linux 內核配置
根據文檔Docker on Zynq Ultrascale+ (Xilinx Yocto Flow)，在PetaLinux工程的文件project-spec/meta-user/recipes-kernel/linux/linux-xlnx/user.cfg里添加下列配置項。
CONFIG_NAMESPACES=y
CONFIG_NET_NS=y
CONFIG_PID_NS=y
CONFIG_IPC_NS=y
CONFIG_UTS_NS=y
CONFIG_CGROUP_CPUACCT=y
CONFIG_CGROUP_DEVICE=y
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_SCHED=y
CONFIG_CPUSETS=y
CONFIG_MEMCG=y
CONFIG_VETH=y
CONFIG_IP_NF_TARGET_MASQUERADE=y
CONFIG_NETFILTER_XT_MATCH_ADDRTYPE=y
CONFIG_NETFILTER_XT_MATCH_IPVS=y
CONFIG_IP_NF_NAT=y
CONFIG_USER_NS=y
CONFIG_SECCOMP=y
CONFIG_CGROUP_PIDS=y
CONFIG_MEMCG_SWAP=y
CONFIG_MEMCG_SWAP_ENABLED=y
CONFIG_BLK_CGROUP=y
CONFIG_BLK_DEV_THROTTLING=y
CONFIG_CFQ_GROUP_IOSCHED=y
CONFIG_CGROUP_PERF=y
CONFIG_CGROUP_HUGETLB=y
CONFIG_NET_CLS_CGROUP=y
CONFIG_CGROUP_NET_PRIO=y
CONFIG_CFS_BANDWIDTH=y
CONFIG_FAIR_GROUP_SCHED=y
CONFIG_RT_GROUP_SCHED=y
CONFIG_IP_NF_TARGET_REDIRECT=y
CONFIG_IP_VS=y
CONFIG_IP_VS_NFCT=y
CONFIG_IP_VS_PROTO_TCP=y
CONFIG_IP_VS_PROTO_UDP=y
CONFIG_IP_VS_RR=y
CONFIG_EXT3_FS_XATTR=y
CONFIG_EXT3_FS_POSIX_ACL=y
CONFIG_EXT3_FS_SECURITY=y
CONFIG_VXLAN=y
CONFIG_INET_ESP=y
CONFIG_IPVLAN=y
CONFIG_MACVLAN=y
CONFIG_DUMMY=y
CONFIG_NF_NAT_FTP=y
CONFIG_NF_CONNTRACK_FTP=y
CONFIG_NF_NAT_TFTP=y
CONFIG_NF_CONNTRACK_TFTP=y
CONFIG_AUFS_FS=y
CONFIG_BTRFS_FS_POSIX_ACL=y
CONFIG_BLK_DEV_DM=y
CONFIG_DM_THIN_PROVISIONING=y
CONFIG_OVERLAY_FS=y
CONFIG_MD=y
CONFIG_NET_SCHED=y
CONFIG_NET_L3_MASTER_DEV=y
CONFIG_BRIDGE=m
# CONFIG_EXT2_FS is not set
CONFIG_EXT4_USE_FOR_EXT2=y
Linux 文件系統配置
在PetaLinux工程里選擇Packagegroup-petalinux-ocicontainers。
啟動命令
使用命令“dockerd &”可以啟動Docker。運行之前，要先導出變量DOCKER_RAMDISK。
export DOCKER_RAMDISK=true
測試命令
使用命令“docker run --rm hello-world”可以運行一個簡單容器。
常見問題
網絡權限
單板要有訪問網絡權限，要能下載Docker的鏡像。
root@vcu_trd:~# docker run --rm hello-world
Unable to find image 'hello-world:latest' locally
WARN[2020-11-11T09:47:14.257031091Z] Could not get operating system name: Error opening /usr/lib/os-release: open /usr/lib/os-release: no such file or directory
WARN[2020-11-11T09:47:14.286697565Z] Error getting v2 registry: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on [::1]:53: read udp [::1]:41143->[::1]:53: read: connection refused
INFO[2020-11-11T09:47:14.286851881Z] Attempting next endpoint for pull after error: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on [::1]:53: read udp [::1]:41143->[::1]:53: read: connection refused
ERRO[2020-11-11T09:47:14.287032029Z] Handler for POST /v1.40/images/create returned error: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on [::1]:53: read udp [::1]:41143->[::1]:53: read: connection refused
docker: Error response from daemon: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on [::1]:53: read udp [::1]:41143->[::1]:53: read: connection refused.
See 'docker run --help'.
存儲空間
單板上可能缺乏足夠存儲空間。可以使用dockerd的選項data-root指定一個有足夠空間的目錄給dockerd使用。在ZCU106單板上，使用ramdisk時，最好進入/run/目錄運行docker.
#ERROR Download failed: write /var/lib/docker/tmp/GetImageBlob091922966: no space left on device
docker啟動失敗
另外，docker啟動過程中，可能沒能啟動containerd，出現下列錯誤。
root@vcu_trd:/run/test# docker run --rm hello-world
docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
See 'docker run --help'.
root@vcu_trd:/run/test# docker-compose pull
ERROR: Couldn't connect to Docker daemon at http+docker://localhost - is it running?
If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.
這時候可以殺死dockerd進程，再次啟動dockerd。可以借助下列命令來快速殺死dockerd進程和啟動dockerd。
export DOCKER_RAMDISK=true
echo -e "\nDOCKER_RAMDISK flag: $DOCKER_RAMDISK\n" | tee -a $docker_log_file
CONTAINERD_PID=$(ps -A | grep -v grep | grep containerd | grep -v .sh |awk '{printf $1}')
DOCKERD_PID=$(ps -A | grep -v grep | grep dockerd | grep -v .sh |awk '{printf $1}')
echo -e "containerd PID: $CONTAINERD_PID\n" | tee -a $docker_log_file
echo -e "dockerd PID: $DOCKERD_PID\n" | tee -a $docker_log_file
if [ ! "$CONTAINERD_PID" = "" ]; then
echo -e "kill containerd PID: $CONTAINERD_PID\n" | tee -a $docker_log_file
kill $CONTAINERD_PID
fi
if [ ! "$DOCKERD_PID" = "" ]; then
echo -e "kill dockerd PID: $DOCKERD_PID\n" | tee -a $docker_log_file
kill $DOCKERD_PID
# Error starting daemon: pid file found, ensure docker is not running or delete /var/run/docker.pid
rm -f /var/run/docker.pid | tee -a $docker_log_file
fi
sleep 1
echo -e "\nStart Docker...\n" | tee -a $docker_log_file
ls -l /run/docker | tee -a $docker_log_file
mkdir -p /run/docker/data | tee -a $docker_log_file
dockerd --data-root /run/docker/data | tee -a $docker_log_file &
sleep 12
ps -A | grep docker | tee -a $docker_log_file
ps -A | grep containerd | tee -a $docker_log_file
ls -l /var/run/docker.sock | tee -a $docker_log_file
文件/var/run/docker.pid已經存在
啟動啟動dockerd時，可能報錯，說文件/var/run/docker.pid已經存在。重啟動dockerd時，需要先刪除文件/var/run/docker.pid。
# Error starting daemon: pid file found, ensure docker is not running or delete /var/run/docker.pid
docker: Error response from daemon: OCI runtime create failed
PetaLinux編譯的Linux啟動后，已經啟動dockerd。直接運行docker的hello-world ，出現下列錯誤。
root@vcu_trd:~# ps -A | grep -v grep | grep containerd
937 ? 00:00:00 containerd
root@vcu_trd:~# ps -A | grep -v grep | grep dockerd
926 ? 00:00:00 dockerd
root@vcu_trd:/run/test# docker run --rm hello-world
docker: Error response from daemon: OCI runtime create failed: container_linux.go:346: starting container process caused "process_linux.go:449: container init caused \"rootfs_linux.go:109: jailing process inside rootfs caused \\\"pivot_root invalid argument\\\"\"": unknown.
ERRO[0001] error waiting for container: context canceled
借助上面描述的腳本設置DOCKER_RAMDISK為true，並重啟動dockerd，運行docker的hello-world成功。
運行記錄如下：
Run Docker hello-world...
Unable to find image 'hello-world:latest' locally
WARN[2020-11-12T01:47:00.103866590Z] Could not get operating system name: Error opening /usr/lib/os-release: open /usr/lib/os-release: no such file or directory
latest: Pulling from library/hello-world
256ab8fe8778: Pulling fs layer
256ab8fe8778: Download complete
256ab8fe8778: Pull complete
Digest: sha256:8c5aeeb6a5f3ba4883347d3747a7249f491766ca1caa47e5da5dfcf6b9b717c0
Status: Downloaded newer image for hello-world:latest
INFO[2020-11-12T01:47:10.600246198Z] shim containerd-shim started address="/containerd-shim/moby/b506434ba15b4ba4896d909c1b0bd24fa9b788c6a75f26ea53da7b9679694fd8/shim.sock" debug=false pid=1618
Hello from Docker!
This message shows that your installation appears to be working correctly.
To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
(arm64v8)
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
to your terminal.
To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/
For more examples and ideas, visit:
https://docs.docker.com/get-started/
INFO[2020-11-12T01:47:11.352948396Z] shim reaped id=b506434ba15b4ba4896d909c1b0bd24fa9b788c6a75f26ea53da7b9679694fd8
INFO[2020-11-12T01:47:11.362202321Z] ignoring event module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
WARN[2020-11-12T01:47:11.453657929Z] b506434ba15b4ba4896d909c1b0bd24fa9b788c6a75f26ea53da7b9679694fd8 cleanup: failed to unmount IPC: umount /run/docker/data/containers/b506434ba15b4ba4896d909c1b0bd24fa9b788c6a75f26ea53da7b9679694fd8/mounts/shm, flags: 0x2: no such file or directory
pivot_root invalid argument: unknown
PetaLinux編譯的Linux啟動后，運行docker的hello-world ，出現下列錯誤“pivot_root invalid argument: unknown”。借助上面描述的腳本設置DOCKER_RAMDISK為true，並重啟動dockerd，運行docker的hello-world成功。
root@vcu_trd:/run/test# export DOCKER_RAMDISK=true
root@vcu_trd:/run/test# echo -e "\nDOCKER_RAMDISK flag: $DOCKER_RAMDISK\n" | tee -a $docker_log_file
DOCKER_RAMDISK flag: true
root@vcu_trd:/run/test# docker run --rm hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
256ab8fe8778: Pull complete
Digest: sha256:8c5aeeb6a5f3ba4883347d3747a7249f491766ca1caa47e5da5dfcf6b9b717c0
Status: Downloaded newer image for hello-world:latest
docker: Error response from daemon: OCI runtime create failed: container_linux.go:346: starting container process caused "process_linux.go:449: container init caused \"rootfs_linux.go:109: jailing process inside rootfs caused \\\"pivot_root invalid argument\\\"\"": unknown.