docker網絡配置方法總結

docker啟動時，會在宿主主機上創建一個名為docker0的虛擬網絡接口，默認選擇172.17.42.1/16，一個16位的子網掩碼給容器提供了65534個IP地址。docker0只是一個在綁定到這上面的其他網卡間自動轉發數據包的虛擬以太網橋，它可以使容器和主機相互通信,容器與容器間通信。問題是，如何讓位於不同主機上的docker容器可以通信。如何有效配置docker網絡目前來說還是一個較復雜的工作，因而也涌現了很多的開源項目來解決這個問題，如flannel、Kubernetes、weave、pipework等等。

 

1. flannel 

 

CoreOS團隊出品，是一個基於etcd的覆蓋網絡(overlay network)並為每台主機提供一個獨立子網的服務。Rudder簡化了集群中Docker容器的網絡配置，避免了多主機上容器子網沖突的問題，更可以大幅度減少端口映射方面的工作。具體代碼見https://github.com/coreos/flannel，其工作原理為：

 

An overlay network is first configured with an IP range and the size of the subnet for each host. For example, one could configure the overlay to use 10.100.0.0/16 and each host to receive a /24 subnet. Host A could then receive 10.100.5.0/24 and host B could get 10.100.18.0/24. flannel uses etcd to maintain a mapping between allocated subnets and real host IP addresses. For the data path, flannel uses UDP to encapsulate IP datagrams to transmit them to the remote host. We chose UDP as the transport protocol for its ease of passing through firewalls. For example, AWS Classic cannot be configured to pass IPoIP or GRE traffic as its security groups only support TCP/UDP/ICMP.（摘自https://coreos.com/blog/introducing-rudder/）

 

2. Kubernetes

 

Kubernetes是由Google推出的針對容器管理和編排的開源項目，它讓用戶能夠在跨容器主機集群的情況下輕松地管理、監測、控制容器化應用部署。Kubernete有一個特殊的與SDN非常相似的網絡化概念：通過一個服務代理創建一個可以分配給任意數目容器的IP地址，前端的應用程序或使用該服務的用戶僅通過這一IP地址調用服務，不需要關心其他的細節。這種代理方案有點SDN的味道，但是它並不是構建在典型的SDN的第2-3層機制之上。

 

Kubernetes uses a proxying method, whereby a particular service — defined as a query across containers — gets its own IP address. Behind that address could be hordes of containers that all provide the same service — but on the front end, the application or user tapping that service just uses the one IP address.

This means the number of containers running a service can grow or shrink as necessary, and no customer or application tapping the service has to care. Imagine if that service were a mobile network back-end process, for instance; during traffic surges, more containers running the process could be added, and they could be deleted once traffic returned to normal. Discovery of the specific containers running the service is handled in the background, as is the load balancing among those containers. Without the proxying, you could add more containers, but you’d have to tell users and applications about it; Google’s method eliminates that need for configuration. (https://www.sdncentral.com/news/docker-kubernetes-containers-open-sdn-possibilities/2014/07/)

 

3. 為不同宿主機上所有容器配置相同網段的IP地址，配置方法見http://www.cnblogs.com/feisky/p/4063162.html，這篇文章是基於Linux bridge的，當然也可以用其他的方法，如用OpenvSwitch+GRE建立宿主機之間的連接：

 

# From http://goldmann.pl/blog/2014/01/21/connecting-docker-containers-on-multiple-hosts/ 

# Edit this variable: the 'other' host.

REMOTE_IP=188.226.138.185

 

# Edit this variable: the bridge address on 'this' host.

BRIDGE_ADDRESS=172.16.42.1/24

 

# Name of the bridge (should match /etc/default/docker).

BRIDGE_NAME=docker0

 

# bridges

 

# Deactivate the docker0 bridge

ip link set $BRIDGE_NAME down

# Remove the docker0 bridge

brctl delbr $BRIDGE_NAME

# Delete the Open vSwitch bridge

ovs-vsctl del-br br0

# Add the docker0 bridge

brctl addbr $BRIDGE_NAME

# Set up the IP for the docker0 bridge

ip a add $BRIDGE_ADDRESS dev $BRIDGE_NAME

# Activate the bridge

ip link set $BRIDGE_NAME up

# Add the br0 Open vSwitch bridge

ovs-vsctl add-br br0

# Create the tunnel to the other host and attach it to the

# br0 bridge

ovs-vsctl add-port br0 gre0 -- set interface gre0 type=gre options:remote_ip=$REMOTE_IP

# Add the br0 bridge to docker0 bridge

brctl addif $BRIDGE_NAME br0

 

# iptables rules

 

# Enable NAT

iptables -t nat -A POSTROUTING -s 172.16.42.0/24 ! -d 172.16.42.0/24 -j MASQUERADE

# Accept incoming packets for existing connections

iptables -A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

# Accept all non-intercontainer outgoing packets

iptables -A FORWARD -i docker0 ! -o docker0 -j ACCEPT

# By default allow all outgoing traffic

iptables -A FORWARD -i docker0 -o docker0 -j ACCEPT

 

# Restart Docker daemon to use the new BRIDGE_NAME

service docker restart

 

4. 使用weave為容器配置IP（使用方法見http://www.cnblogs.com/feisky/p/4093717.html），weave的特性包括

• 應用隔離：不同子網容器之間默認隔離的，即便它們位於同一台物理機上也相互不通；不同物理機之間的容器默認也是隔離的
• 物理機之間容器互通：weave connect $OTHER_HOST
• 動態添加網絡：對於不是通過weave啟動的容器，可以通過weave attach 10.0.1.1/24 $id來添加網絡（detach刪除網絡）
• 安全性：可以通過weave launch -password wEaVe設置一個密碼用於weave peers之間加密通信
• 與宿主機網絡通信：weave expose 10.0.1.102/24，這個IP會配在weave網橋上
• 查看weave路由狀態：weave ps
• 通過NAT實現外網訪問docker容器

 

5. 修改主機docker默認的虛擬網段，然后在各自主機上分別把對方的docker網段加入到路由表中，配合iptables即可實現docker容器誇主機通信。配置方法如下：

 

設有兩台虛擬機

• v1: 192.168.124.51
• v2: 192.168.124.52

更改虛擬機docker0網段，v1為172.17.1.1/24，v2為172.17.2.1/24

＃v1

sudo ifconfig docker0 172.17.1.1 netmask 255.255.255.0
sudo bash -c 'echo DOCKER_OPTS="-B=docker0" >> /etc/default/docker'
sudo service docker restart

# v2
sudo ifconfig docker0 172.17.2.1 netmask 255.255.255.0
sudo bash -c 'echo DOCKER_OPTS="-B=docker0" >> /etc/default/docker'
sudo service docker restart

 

然后在v1上把v2的docker虛擬網段加入到路由表中，在v2上將v1的docker虛擬網段加入到自己的路由表中

# v1 192.168.124.51
sudo route add -net 172.17.2.0 netmask 255.255.255.0 gw 192.168.124.52
sudo iptables -t nat -F POSTROUTING
sudo iptables -t nat -A POSTROUTING -s 172.17.1.0/24 ! -d 172.17.0.0/16 -j MASQUERADE

# v2 192.168.124.52
sudo route add -net 172.17.1.0  netmask 255.255.255.0  gw 192.168.124.51
sudo iptables -t nat -F POSTROUTING
sudo iptables -t nat -A POSTROUTING -s 172.17.2.0/24 ! -d 172.17.0.0/16 -j MASQUERADE

至此，兩台虛擬機中的docker容器可以互相訪問了。