使用weave管理docker網絡

 

weave簡介

---

Weave creates a virtual network that connects Docker containers deployed across multiple hosts.

Applications use the network just as if the containers were all plugged into the same network switch, with no need to configure port mappings, links, etc. Services provided by application containers on the weave network can be made accessible to the outside world, regardless of where those containers are running. Similarly, existing internal systems can be exposed to application containers irrespective of their location.

Weave can traverse firewalls and operate in partially connected networks. Traffic can be encrypted, allowing hosts to be connected across an untrusted network.

With weave you can easily construct applications consisting of multiple containers, running anywhere.

Weave works alongside Docker's existing (single host) networking capabilities, so these can continue to be used by containers.

 

weave簡單使用

---

sudo wget -O /usr/local/bin/weave https://raw.githubusercontent.com/zettio/weave/master/weave
sudo chmod a+x /usr/local/bin/weave

啟動weave路由器，這個路由器其實也是在docker中啟動的：

[root@h-46mow360 ~]# weave launch
Unable to find image 'zettio/weave' locally
3b3a3db2c186fccb5203dcc269b3febbbbf126591a7ebd8117a8a5250683749f

[root@h-46mow360 ~]# brctl show
bridge name bridge id       STP enabled interfaces
docker0     8000.56847afe9799   no      veth801050a
weave       8000.7afc2a03325e   no      vethwepl2146
[root@h-46mow360 ~]# docker ps
CONTAINER ID        IMAGE                           COMMAND                CREATED              STATUS              PORTS                                            NAMES
3b3a3db2c186        zettio/weave:git-a34e214201cb   "/home/weave/weaver    About a minute ago   Up About a minute   0.0.0.0:6783->6783/tcp, 0.0.0.0:6783->6783/udp   weave

在兩台物理機上分別啟動一個容器：

c1=$(weave run 10.0.3.3/24 -t -i -v /sys/fs/cgroup:/sys/fs/cgroup:ro -v /tmp/$(mktemp -d):/run systemd:systemd /usr/lib/systemd/systemd)

c2=$(weave run 10.0.3.5/24 -t -i -v /sys/fs/cgroup:/sys/fs/cgroup:ro -v /tmp/$(mktemp -d):/run systemd:systemd /usr/lib/systemd/systemd)

這個時候，兩個容器之間是不通的，需要在兩台weave的路由器之間建立連接：( if there is a firewall between $HOST1 and $HOST2, you must open port 6783 for TCP and UDP)

weave connect 10.33.0.9

這樣，兩台容器之間通了：

# nsenter --mount --uts --ipc --net --pid --target $(docker inspect --format "{{.State.Pid}}" "$c2")

-bash-4.2# ping -c 3 10.0.3.3

PING 10.0.3.3 (10.0.3.3) 56(84) bytes of data.

64 bytes from 10.0.3.3: icmp_seq=1 ttl=64 time=2.34 ms

64 bytes from 10.0.3.3: icmp_seq=2 ttl=64 time=1.52 ms

64 bytes from 10.0.3.3: icmp_seq=3 ttl=64 time=1.13 ms

--- 10.0.3.3 ping statistics ---

3 packets transmitted, 3 received, 0% packet loss, time 2003ms

rtt min/avg/max/mdev = 1.131/1.667/2.345/0.505 ms

但容器內部還是無法與宿主機之間通信。如果想讓容器與宿主機網絡聯通，需要將weave路由器與宿主機網絡聯通，即執行：  weave expose 10.0.3.102/24

 

weave其他特性

 

• 應用隔離：不同子網容器之間默認隔離的，即便它們位於同一台物理機上也相互不通；不同物理機之間的容器默認也是隔離的
• 物理機之間容器互通：weave connect $OTHER_HOST
• 動態添加網絡：對於不是通過weave啟動的容器，可以通過weave attach 10.0.1.1/24 $id來添加網絡（detach刪除網絡）
• 安全性：可以通過weave launch -password wEaVe設置一個密碼用於weave peers之間加密通信
• 與宿主機網絡通信：weave expose 10.0.1.102/24，這個IP會配在weave網橋上
• 查看weave路由狀態：weave ps
• 通過NAT實現外網訪問docker容器

 

實現原理

 

Weave creates a network bridge on the host. Each container is connected to that bridge via a veth pair, the container side of which is given the IP address & netmask supplied in ‘weave run’. Also connected to the bridge is the weave router container.

A weave router captures Ethernet packets from its bridge-connected interface in promiscuous mode, using ‘pcap’. This typically excludes traffic between local containers, and between the host and local containers, all of which is routed straight over the bridge by the kernel. Captured packets are forwarded over UDP to weave router peers running on other hosts. On receipt of such a packet, a router injects the packet on its bridge interface using ‘pcap’ and/or forwards the packet to peers.

Weave routers learn which peer host a particular MAC address resides on. They combine this knowledge with topology information in order to make routing decisions and thus avoid forwarding every packet to every peer. The topology information captures which peers are connected to which other peers; weave can route packets in partially connected networks with changing topology.

Weave routers establish TCP connections to each other, over which they perform a protocol handshake and subsequently exchange topology information. These connections are encrypted if so configured. Peers also establish UDP “connections”, possibly encrypted, for the aforementioned packet forwarding. These “connections” are duplex and can traverse firewalls.

 

 

其他需要注意的問題

 

1. MTU considerations

 

MTU is the payload size of an ethernet frame. The ethernet headers are

not included in this and add a further 14 bytes (an ethernet header is

really at least 22 bytes. However only 14 bytes actually carry data -

the two mac fields and a type/length field. The other parts of the

header are there for signalling at the electrical layer and thus don't

get captured by pcap, nor need forwarding). We are then passing this

over UDP which adds a further 8 bytes, and over IP, which adds at

least another 20 bytes of headers (IP can have variable header

length. Yay). We also include the namehash as a 32 byte prefix.

 

Therefore, we have 20(ip) + 8(udp) + 32(namehash) + 2(length prefix) +

14(ethernet) = 76 bytes of overhead. Thus we should make sure the

internal MTU is 76 bytes lower than the external MTU. So on a normal

1500 byte ethernet MTU, we should ensure the other containers use

nothing greater than 1424.

 

2. 重啟容器

 

如果使用weave，則就不能再使用docker自帶的auto-restart feature（如docker run --restart=always redis），因為weave是在docker之外為容器配置的網絡，容器重啟的時候docker本身不會做這些事情。因而，還需額外的工具來管理容器的狀態（比如systemd, upstart等），這些工具要調用weave命令（weave run/start/attach）來啟動容器。

3. 為了便於開機自動啟動weave，可以創建一個weave.service：

 

[Unit]

Description=Weave Network

Documentation= http://zettio.github.io/weave/
After=docker.service

 

[Service]
ExecStartPre=/usr/local/bin/weave launch
ExecStart=/usr/bin/docker logs -f weave
SuccessExitStatus=2
ExecStop=/usr/local/bin/weave stop

 

[Install]
WantedBy=multi-user.target

 

參考文檔  https://zettio.github.io/weave/