故障描述:
虛擬機重啟之后,以前創建的容器無法啟動。若一執行啟動操作,則會報錯。
[root@docker ~]# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 6fe70546bad1 registry "/entrypoint.sh /etc/" 33 hours ago Exited (2) 44 minutes ago registry [root@k8s-master ~]# docker start registry Error response from daemon: devmapper: Error mounting '/dev/mapper/docker-253:0-101647595-159fb55f6c5d487601a87dd0bd72995039e617d0a6bbe786432a16a85cdf0a9a' on '/var/lib/docker/devicemapper/mnt/159fb55f6c5d487601a87dd0bd72995039e617d0a6bbe786432a16a85cdf0a9a'. fstype=xfs options=nouuid,context="system_u:object_r:svirt_sandbox_file_t:s0:c884,c955": invalid argument <4>[ 2181.498331] XFS (dm-4): unknown mount option [context="system_u:object_r:svirt_sandbox_file_t:s0:c884]. <4>[ 2225.188924] XFS (dm-4): unknown mount option [context="system_u:object_r:svirt_sandbox_file_t:s0:c884]. Error: failed to start containers: registry
故障原因:
這個錯誤的主要原因是:在 SELinux 設置為 enable 的時候,創建了該容器(registry)。之后,修改了 /etc/selinux/config 文件,將 selinux 修改為 disabled。
然后虛擬機重啟之后,selinux 處於 關閉狀態,則原先在 SELinux 為 enable 時所創建的容器就會無法啟動,報出如上錯誤。
解決辦法:
方法1: 將 SELinux 重新設置為 enable,然后重啟虛擬機,即可修復。
方法2:修改容器的配置。
找到 registry 容器的配置文件,將 MountLabel 和 ProcessLabel 兩個鍵的值設置為空。 ("MountLabel":"","ProcessLabel":"")
然后重啟docker服務,容器即可修復。
[root@docker ~]# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 6fe70546bad1 registry "/entrypoint.sh /etc/" 33 hours ago Exited (2) About a minute ago registry [root@docker ~]# docker start registry Error response from daemon: devmapper: Error mounting '/dev/mapper/docker-253:0-101647595-159fb55f6c5d487601a87dd0bd72995039e617d0a6bbe786432a16a85cdf0a9a' on '/var/lib/docker/devicemapper/mnt/159fb55f6c5d487601a87dd0bd72995039e617d0a6bbe786432a16a85cdf0a9a'. fstype=xfs options=nouuid,context="system_u:object_r:svirt_sandbox_file_t:s0:c884,c955": invalid argument <4>[ 10.063135] XFS (dm-4): unknown mount option [context="system_u:object_r:svirt_sandbox_file_t:s0:c884]. <4>[ 29.217066] XFS (dm-4): unknown mount option [context="system_u:object_r:svirt_sandbox_file_t:s0:c884]. Error: failed to start containers: registry [root@docker ~]# getenforce Disabled [root@docker ~]# cat /var/lib/docker/containers/6fe70546bad13255f6bee215e8a3462bc59222282adca56bc0fb37f086ab084f/config.v2.json {"StreamConfig":{},"State":{"Running":false,"Paused":false,"Restarting":false,"OOMKilled":false,"RemovalInProgress":false,"Dead":false,"Pid":0,"ExitCode":2,"Error":"devmapper: Error mounting '/dev/mapper/docker-253:0-101647595-159fb55f6c5d487601a87dd0bd72995039e617d0a6bbe786432a16a85cdf0a9a' on '/var/lib/docker/devicemapper/mnt/159fb55f6c5d487601a87dd0bd72995039e617d0a6bbe786432a16a85cdf0a9a'. fstype=xfs options=nouuid,context=\"system_u:object_r:svirt_sandbox_file_t:s0:c884,c955\": invalid argument\n\u003c4\u003e[ 10.063135] XFS (dm-4): unknown mount option [context=\"system_u:object_r:svirt_sandbox_file_t:s0:c884].\n\u003c4\u003e[ 29.217066] XFS (dm-4): unknown mount option [context=\"system_u:object_r:svirt_sandbox_file_t:s0:c884].\n","StartedAt":"2020-10-21T14:26:02.794541014Z","FinishedAt":"2020-10-21T14:46:01.039144732Z","Health":null},"ID":"6fe70546bad13255f6bee215e8a3462bc59222282adca56bc0fb37f086ab084f","Created":"2020-10-20T05:15:43.313444377Z","Managed":false,"Path":"/entrypoint.sh","Args":["/etc/docker/registry/config.yml"],"Config":{"Hostname":"6fe70546bad1","Domainname":"","User":"","AttachStdin":false,"AttachStdout":false,"AttachStderr":false,"ExposedPorts":{"5000/tcp":{}},"Tty":false,"OpenStdin":false,"StdinOnce":false,"Env":["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"],"Cmd":["/etc/docker/registry/config.yml"],"Image":"registry","Volumes":{"/var/lib/registry":{}},"WorkingDir":"","Entrypoint":["/entrypoint.sh"],"OnBuild":null,"Labels":{}},"Image":"sha256:2d4f4b5309b1e41b4f83ae59b44df6d673ef44433c734b14c1c103ebca82c116","NetworkSettings":{"Bridge":"","SandboxID":"e561b284b016f17ab5a47f2ae64f48c49c91d063ba953954dc8b764b4f81fe52","HairpinMode":false,"LinkLocalIPv6Address":"","LinkLocalIPv6PrefixLen":0,"Networks":{"bridge":{"IPAMConfig":null,"Links":null,"Aliases":null,"NetworkID":"9d7ce7252a1fb6654a1671a1eb90c2aaa2c465eb87219aba93eb152c71c1bd79","EndpointID":"","Gateway":"","IPAddress":"","IPPrefixLen":0,"IPv6Gateway":"","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"MacAddress":""}},"Service":null,"Ports":null,"SandboxKey":"/var/run/docker/netns/e561b284b016","SecondaryIPAddresses":null,"SecondaryIPv6Addresses":null,"IsAnonymousEndpoint":false},"LogPath":"","Name":"/registry","Driver":"devicemapper","MountLabel":"system_u:object_r:svirt_sandbox_file_t:s0:c884,c955","ProcessLabel":"system_u:system_r:svirt_lxc_net_t:s0:c884,c955","RestartCount":0,"HasBeenStartedBefore":false,"HasBeenManuallyStopped":false,"MountPoints":{"/var/lib/registry":{"Source":"/opt/myregistry","Destination":"/var/lib/registry","RW":true,"Name":"","Driver":"","Relabel":"","Propagation":"rprivate","Named":false,"ID":""}},"AppArmorProfile":"","HostnamePath":"/var/lib/docker/containers/6fe70546bad13255f6bee215e8a3462bc59222282adca56bc0fb37f086ab084f/hostname","HostsPath":"/var/lib/docker/containers/6fe70546bad13255f6bee215e8a3462bc59222282adca56bc0fb37f086ab084f/hosts","ShmPath":"/var/lib/docker/containers/6fe70546bad13255f6bee215e8a3462bc59222282adca56bc0fb37f086ab084f/shm","ResolvConfPath":"/var/lib/docker/containers/6fe70546bad13255f6bee215e8a3462bc59222282adca56bc0fb37f086ab084f/resolv.conf","SeccompProfile":"","NoNewPrivileges":false} [root@docker ~]# vim /var/lib/docker/containers/6fe70546bad13255f6bee215e8a3462bc59222282adca56bc0fb37f086ab084f/config.v2.json [root@docker ~]# cat /var/lib/docker/containers/6fe70546bad13255f6bee215e8a3462bc59222282adca56bc0fb37f086ab084f/config.v2.json {"StreamConfig":{},"State":{"Running":false,"Paused":false,"Restarting":false,"OOMKilled":false,"RemovalInProgress":false,"Dead":false,"Pid":0,"ExitCode":2,"Error":"devmapper: Error mounting '/dev/mapper/docker-253:0-101647595-159fb55f6c5d487601a87dd0bd72995039e617d0a6bbe786432a16a85cdf0a9a' on '/var/lib/docker/devicemapper/mnt/159fb55f6c5d487601a87dd0bd72995039e617d0a6bbe786432a16a85cdf0a9a'. fstype=xfs options=nouuid,context=\"system_u:object_r:svirt_sandbox_file_t:s0:c884,c955\": invalid argument\n\u003c4\u003e[ 10.063135] XFS (dm-4): unknown mount option [context=\"system_u:object_r:svirt_sandbox_file_t:s0:c884].\n\u003c4\u003e[ 29.217066] XFS (dm-4): unknown mount option [context=\"system_u:object_r:svirt_sandbox_file_t:s0:c884].\n","StartedAt":"2020-10-21T14:26:02.794541014Z","FinishedAt":"2020-10-21T14:46:01.039144732Z","Health":null},"ID":"6fe70546bad13255f6bee215e8a3462bc59222282adca56bc0fb37f086ab084f","Created":"2020-10-20T05:15:43.313444377Z","Managed":false,"Path":"/entrypoint.sh","Args":["/etc/docker/registry/config.yml"],"Config":{"Hostname":"6fe70546bad1","Domainname":"","User":"","AttachStdin":false,"AttachStdout":false,"AttachStderr":false,"ExposedPorts":{"5000/tcp":{}},"Tty":false,"OpenStdin":false,"StdinOnce":false,"Env":["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"],"Cmd":["/etc/docker/registry/config.yml"],"Image":"registry","Volumes":{"/var/lib/registry":{}},"WorkingDir":"","Entrypoint":["/entrypoint.sh"],"OnBuild":null,"Labels":{}},"Image":"sha256:2d4f4b5309b1e41b4f83ae59b44df6d673ef44433c734b14c1c103ebca82c116","NetworkSettings":{"Bridge":"","SandboxID":"e561b284b016f17ab5a47f2ae64f48c49c91d063ba953954dc8b764b4f81fe52","HairpinMode":false,"LinkLocalIPv6Address":"","LinkLocalIPv6PrefixLen":0,"Networks":{"bridge":{"IPAMConfig":null,"Links":null,"Aliases":null,"NetworkID":"9d7ce7252a1fb6654a1671a1eb90c2aaa2c465eb87219aba93eb152c71c1bd79","EndpointID":"","Gateway":"","IPAddress":"","IPPrefixLen":0,"IPv6Gateway":"","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"MacAddress":""}},"Service":null,"Ports":null,"SandboxKey":"/var/run/docker/netns/e561b284b016","SecondaryIPAddresses":null,"SecondaryIPv6Addresses":null,"IsAnonymousEndpoint":false},"LogPath":"","Name":"/registry","Driver":"devicemapper","MountLabel":"","ProcessLabel":"","RestartCount":0,"HasBeenStartedBefore":false,"HasBeenManuallyStopped":false,"MountPoints":{"/var/lib/registry":{"Source":"/opt/myregistry","Destination":"/var/lib/registry","RW":true,"Name":"","Driver":"","Relabel":"","Propagation":"rprivate","Named":false,"ID":""}},"AppArmorProfile":"","HostnamePath":"/var/lib/docker/containers/6fe70546bad13255f6bee215e8a3462bc59222282adca56bc0fb37f086ab084f/hostname","HostsPath":"/var/lib/docker/containers/6fe70546bad13255f6bee215e8a3462bc59222282adca56bc0fb37f086ab084f/hosts","ShmPath":"/var/lib/docker/containers/6fe70546bad13255f6bee215e8a3462bc59222282adca56bc0fb37f086ab084f/shm","ResolvConfPath":"/var/lib/docker/containers/6fe70546bad13255f6bee215e8a3462bc59222282adca56bc0fb37f086ab084f/resolv.conf","SeccompProfile":"","NoNewPrivileges":false}
[root@docker ~]# systemctl restart docker.service [root@docker ~]# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 6fe70546bad1 registry "/entrypoint.sh /etc/" 33 hours ago Up 2 seconds 0.0.0.0:5000->5000/tcp registry
這個時候,registry容器啟動后的配置:
[root@docker ~]# cat /var/lib/docker/containers/6fe70546bad13255f6bee215e8a3462bc59222282adca56bc0fb37f086ab084f/config.v2.json {"StreamConfig":{},"State":{"Running":true,"Paused":false,"Restarting":false,"OOMKilled":false,"RemovalInProgress":false,"Dead":false,"Pid":1911,"ExitCode":0,"Error":"","StartedAt":"2020-10-21T14:56:30.801231296Z","FinishedAt":"2020-10-21T14:46:01.039144732Z","Health":null},"ID":"6fe70546bad13255f6bee215e8a3462bc59222282adca56bc0fb37f086ab084f","Created":"2020-10-20T05:15:43.313444377Z","Managed":false,"Path":"/entrypoint.sh","Args":["/etc/docker/registry/config.yml"],"Config":{"Hostname":"6fe70546bad1","Domainname":"","User":"","AttachStdin":false,"AttachStdout":false,"AttachStderr":false,"ExposedPorts":{"5000/tcp":{}},"Tty":false,"OpenStdin":false,"StdinOnce":false,"Env":["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"],"Cmd":["/etc/docker/registry/config.yml"],"Image":"registry","Volumes":{"/var/lib/registry":{}},"WorkingDir":"","Entrypoint":["/entrypoint.sh"],"OnBuild":null,"Labels":{}},"Image":"sha256:2d4f4b5309b1e41b4f83ae59b44df6d673ef44433c734b14c1c103ebca82c116","NetworkSettings":{"Bridge":"","SandboxID":"b957358666a82bf75f9ff35515df6277a1784856d39bde47932ffdd72b24cee7","HairpinMode":false,"LinkLocalIPv6Address":"","LinkLocalIPv6PrefixLen":0,"Networks":{"bridge":{"IPAMConfig":null,"Links":null,"Aliases":null,"NetworkID":"c63595ac1d3560732b6dbb9b0d7596f71ba2e7f0fb22b528c61bb4729f156122","EndpointID":"00f37efd073a40b57b70cc628cbbfc9989e116f33393185e6e319c5838647051","Gateway":"172.16.62.1","IPAddress":"172.16.62.2","IPPrefixLen":24,"IPv6Gateway":"","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"MacAddress":"02:42:ac:10:3e:02"}},"Service":null,"Ports":{"5000/tcp":[{"HostIp":"0.0.0.0","HostPort":"5000"}]},"SandboxKey":"/var/run/docker/netns/b957358666a8","SecondaryIPAddresses":null,"SecondaryIPv6Addresses":null,"IsAnonymousEndpoint":false},"LogPath":"","Name":"/registry","Driver":"devicemapper","MountLabel":"","ProcessLabel":"","RestartCount":0,"HasBeenStartedBefore":false,"HasBeenManuallyStopped":false,"MountPoints":{"/var/lib/registry":{"Source":"/opt/myregistry","Destination":"/var/lib/registry","RW":true,"Name":"","Driver":"","Relabel":"","Propagation":"rprivate","Named":false,"ID":""}},"AppArmorProfile":"","HostnamePath":"/var/lib/docker/containers/6fe70546bad13255f6bee215e8a3462bc59222282adca56bc0fb37f086ab084f/hostname","HostsPath":"/var/lib/docker/containers/6fe70546bad13255f6bee215e8a3462bc59222282adca56bc0fb37f086ab084f/hosts","ShmPath":"/var/lib/docker/containers/6fe70546bad13255f6bee215e8a3462bc59222282adca56bc0fb37f086ab084f/shm","ResolvConfPath":"/var/lib/docker/containers/6fe70546bad13255f6bee215e8a3462bc59222282adca56bc0fb37f086ab084f/resolv.conf","SeccompProfile":"","NoNewPrivileges":false}
(結果發現 MountLabel 和 ProcessLabel 這兩個鍵的值依然為空)
同時,這種改法只針對單一的容器,那么幾十個容器,不可能一一修改,建議使用方法1。
