Kubernetes WebSocket: Error during WebSocket handshake: Unexpected response code: 403

Problem

筆者從 Vue 中建立 ws 連接訪問 Kubernetes WebSocket 獲取容器日志，得到此問題：

WebSocket connection to 'ws://xx.xx.xx.xx:8080/api/v1/namespaces/default/pods/test-1/log?follow=true&tailLines=1000&timestamps=true&container=test-1' failed: 
Error during WebSocket handshake: Unexpected response code: 403Error during WebSocket handshake: Unexpected response code: 403

Solution

在和同事討論以及查閱了一些資料后，發現在連接 ws 連接時缺少了 Kubernetes WebSocket 需要遵循的子協議 ：

Kubernetes WebSocket 接口遵循 WebSocke t協議，並在此基礎上設計了子協議，在編寫WebSocket客戶端與Kubernetes通信時，必須選擇並遵守其中一種協議。該子協議協議並沒有官方文檔，但我們可以從k8s的代碼（ https://github.com/kubernetes/apiserver/blob/master/pkg/util/wsstream/conn.go ）中找到相關說明：

// the channel number (zero indexed) the message was sent on. Messages in both directions should
// prefix their messages with this channel byte. When used for remote execution, the channel numbers
// are by convention defined to match the POSIX file-descriptors assigned to STDIN, STDOUT, and STDERR
// (0, 1, and 2). No other conversion is performed on the raw subprotocol - writes are sent as they
// are received by the server.
//
// Example client session:
//
//    CONNECT http://server.com with subprotocol "channel.k8s.io"
//    WRITE []byte{0, 102, 111, 111, 10} # send "foo\n" on channel 0 (STDIN)
//    READ  []byte{1, 10}                # receive "\n" on channel 1 (STDOUT)
//    CLOSE
//
const ChannelWebSocketProtocol = "channel.k8s.io"

// The Websocket subprotocol "base64.channel.k8s.io" base64 encodes each message with a character
// indicating the channel number (zero indexed) the message was sent on. Messages in both directions
// should prefix their messages with this channel char. When used for remote execution, the channel
// numbers are by convention defined to match the POSIX file-descriptors assigned to STDIN, STDOUT,
// and STDERR ('0', '1', and '2'). The data received on the server is base64 decoded (and must be
// be valid) and data written by the server to the client is base64 encoded.
//
// Example client session:
//
//    CONNECT http://server.com with subprotocol "base64.channel.k8s.io"
//    WRITE []byte{48, 90, 109, 57, 118, 67, 103, 111, 61} # send "foo\n" (base64: "Zm9vCgo=") on channel '0' (STDIN)
//    READ  []byte{49, 67, 103, 61, 61} # receive "\n" (base64: "Cg==") on channel '1' (STDOUT)
//    CLOSE
//
const Base64ChannelWebSocketProtocol = "base64.channel.k8s.io"

另外，在一個 k8s client 項目 kubebox 中，並沒有使用這兩個子協議，而是使用了 binary.k8s.io（(https://github.com/astefanutti/kubebox/blob/master/lib/client.js#L204)[https://github.com/astefanutti/kubebox/blob/master/lib/client.js#L204]），和同事討論之后，得到一些猜測：

普通的接口用websocket 都是 json 不需要做子協議，log 本質是 二進制流，用 binary 子協議；而 exec 是雙向用channel。

筆者最終采用了這個子協議解決問題：

initSocket() {
  const WebSocketUrl = "ws://xx.xx.xx.xx:8080/api/v1/namespaces/default/pods/test-1/log?follow=true&tailLines=1000&timestamps=true&container=test-1"
  this.socket = new WebSocket(WebSocketUrl, 'binary.k8s.io')
  ...
}

參考資料：
http://blog.allen-mo.com/2018/04/17/kubernetes_websocket/
https://github.com/astefanutti/kubebox/blob/master/lib/client.js#L204
https://github.com/kubernetes/apiserver/blob/master/pkg/util/wsstream/conn.go