背景
ansible 節點 到三個k8s節點，執行ansible UCA -m ping -o 只有 一個成功 ， 兩個失敗。
ansible到目標主機使用秘鑰登錄。
節點
ansible
k8s1
k8s2
k8s3
排查過程描述:
ansible默認通過ssh去連接目標主機，並執行制定命令腳本。
此次遇到問題在執行ansible UCA -m ping -o 時輸出如下：
[root@ansible ~]# ansible UCA -m ping -o
[WARNING]: Unable to parse /root/hosts as an inventory source
Wednesday 10 June 2020 16:16:48 +0800 (0:00:00.062) 0:00:00.062 ********
[WARNING]: Unhandled error in Python interpreter discovery for host HD1-SHMY1-UCA-K8S-Node2: Failed to connect to the host via ssh: Warning: Permanently
added '100.66.0.2' (ECDSA) to the list of known hosts. Permission denied (publickey).
[WARNING]: Unhandled error in Python interpreter discovery for host HD1-SHMY1-UCA-K8S-Node3: Failed to connect to the host via ssh: Warning: Permanently
added '100.66.0.3' (ECDSA) to the list of known hosts. Permission denied (publickey).
HD1-SHMY1-UCA-K8S-Node2 | UNREACHABLE!: Data could not be sent to remote host "100.66.0.2". Make sure this host can be reached over ssh: Permission denied (publickey).
HD1-SHMY1-UCA-K8S-Node3 | UNREACHABLE!: Data could not be sent to remote host "100.66.0.3". Make sure this host can be reached over ssh: Permission denied (publickey).
HD1-SHMY1-UCA-K8S-Node1 | SUCCESS => {"ansible_facts": {"discovered_interpreter_python": "/usr/bin/python"}, "changed": false, "ping": "pong"}
Wednesday 10 June 2020 16:16:49 +0800 (0:00:00.924) 0:00:00.986 ********
===============================================================================
ping ---------------------------------------------------------------------------------------------------------------------------------------------- 0.92s
Playbook run took 0 days, 0 hours, 0 minutes, 0 seconds
從輸出里看是權限不允許, 於是嘗試ssh連接
[root@ansible ~]# ssh 100.66.0.2 -vvv
OpenSSH_7.4p1, OpenSSL 1.0.2k-fips 26 Jan 2017
debug1: Reading configuration data /root/.ssh/config
debug1: /root/.ssh/config line 1: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 58: Applying options for *
debug2: resolving "100.66.0.2" port 22
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to 100.66.0.2 [100.66.0.2] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
...中間輸出省略
debug1: SSH2_MSG_SERVICE_ACCEPT received
__debug3: send packet: type 50__
__debug3: receive packet: type 51__
debug1: Authentications that can continue: publickey
debug3: start over, passed a different list publickey
debug3: preferred gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /root/.ssh/id_rsa
debug3: send_pubkey_test
__debug3: send packet: type 50__
debug2: we sent a publickey packet, wait for reply
__debug3: receive packet: type 51__
debug1: Authentications that can continue: publickey
debug1: Trying private key: /root/.ssh/id_dsa
debug3: no such identity: /root/.ssh/id_dsa: No such file or directory
debug1: Trying private key: /root/.ssh/id_ecdsa
debug3: no such identity: /root/.ssh/id_ecdsa: No such file or directory
debug1: Trying private key: /root/.ssh/id_ed25519
debug3: no such identity: /root/.ssh/id_ed25519: No such file or directory
debug2: we did not send a packet, disable method
debug1: No more authentication methods to try.
Permission denied (publickey).
看上邊輸出, 由於我使用了key登錄， 故關注此處
debug1: Offering RSA public key: /root/.ssh/id_rsa
__debug3: send packet: type 50__
debug2: we sent a publickey packet, wait for reply
__debug3: receive packet: type 51__
debug1:提供了我的key文件路徑之后， send了一個type 50的包，等待reply時收到了一個type 51的包, 谷歌一通后發現：
http://www.snailbook.com/docs/assigned-numbers.txt
4.1.2. Initial Assignments
Message ID Value Reference
----------- ----- ---------
SSH_MSG_USERAUTH_REQUEST 50 [SSH-USERAUTH]
SSH_MSG_USERAUTH_FAILURE 51 [SSH-USERAUTH]
http://www.snailbook.com/docs/userauth.txt
6. Authentication Protocol Message Numbers
These are the general authentication message codes:
SSH_MSG_USERAUTH_REQUEST 50
SSH_MSG_USERAUTH_FAILURE 51
SSH_MSG_USERAUTH_SUCCESS 52
SSH_MSG_USERAUTH_BANNER 53
7. Public Key Authentication Method: "publickey"
The following method-specific message numbers are used by the
"publickey" authentication method.
SSH_MSG_USERAUTH_PK_OK 60
也就是數發送了個ssh用戶認證請求之后，返回了個用戶驗證錯誤，正確的返回應該是type 60的包。
於是我又ssh了k8s1（可以正常ssh的），輸出如下：
debug1: Offering RSA public key: /root/.ssh/id_rsa
debug3: send_pubkey_test
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 60
果然正常的驗證正確返回值是type 60的包。
常規的配置檢查了一遍沒有異常， 此時ls -l /root/.ssh ， 發現三個k8s節點的文件有些異常
k8s1
[root@k8s-node1 .ssh]# ll
total 12
-rw------- 1 root root 2569 Jun 10 16:37 authorized_keys
-rw------- 1 root root 1679 Apr 17 09:34 id_rsa
-rw-r--r-- 1 root root 410 Apr 17 09:34 id_rsa.pub
k8s2
[root@k8s-node2 .ssh]# ll
total 8
-rw------- 1 root root 2571 Jun 10 16:37 authorized_keys
-rw------- 1 root root 1679 Jun 10 16:05 id_rsa
k8s3
[root@k8s-node3 .ssh]# ll
total 8
-rw------- 1 root root 2571 Jun 10 16:37 authorized_keys
-rw------- 1 root root 1679 Jun 10 16:05 id_rsa
k8s1的authorized_keys大小和2、3的不一致，但是cat了一下內容完全一樣。
於是將k8s1上的authorized_keys scp到2和3上，於是 都可以通過秘鑰訪問了。
結論
ssh發送用戶認證請求之后，驗證key失敗。 至於authorized_keys的大小為什么不一致，由於不能重現，就不得而知了。