創建tenant1虛機vm1
1）在網絡節點添加tenant1流表
ovs-ofctl mod-flows br-tun "table=21,dl_vlan=1,actions=strip_vlan,set_tunnel:1,output:2" ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=1,table=2,tun_id=1,actions=mod_vlan_vid:1,resubmit(,10)"
2）在網絡節點添加tenant1的dhcp agent服務
ovs-vsctl -- --if-exists del-port tap01 -- add-port br-int tap01 -- set interface tap01 type=internal ovs-vsctl --timeout=10 set Port tap01 tag=1 ip netns add qdhcp01 ip netns exec qdhcp01 ip link set lo up ip link set tap01 netns qdhcp01 ip netns exec qdhcp01 ip link set tap01 up ip netns exec qdhcp01 ip -4 addr add 10.0.0.3/24 brd 10.0.0.255 scope global dev tap01 ip netns exec qdhcp01 ip route replace default via 10.0.0.1 dev tap01 ip netns exec qdhcp01 dnsmasq --no-hosts --no-resolv --strict-order --bind-interfaces --interface=tap01 --except-interface=lo --dhcp-range=set:tag0,10.0.0.6,10.0.0.90,86400s --dhcp-lease-max=256 --dhcp-option=3,10.0.0.1
3）在計算節點添加虛機tenant1流表
ovs-ofctl mod-flows br-tun "table=21,dl_vlan=1,actions=strip_vlan,set_tunnel:1,output:2" ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=1,table=2,tun_id=1,actions=mod_vlan_vid:1,resubmit(,10)"
4）創建vm1掛接的網橋設備
brctl addbr qbr01 ip link set qbr01 up ip link add qvo01 type veth peer name qvb01 brctl addif qbr01 qvb01 ovs-vsctl add-port br-int qvo01 ip link set qvb01 up ip link set qvo01 up ovs-vsctl set port qvo01 tag=1 ip tuntap add tap01 mode tap ip link set dev tap01 up brctl addif qbr01 tap01
5）准備虛機映像文件
cp cirros-0.3.4-x86_64-disk.img cirros-vm1.img cp cirros-0.3.4-x86_64-kernel cirros-kernel-vm1
6）通過qemu啟動虛機vm1
qemu-system-x86_64 -nographic -kernel ./cirros-kernel-vm1 -hda cirros-vm1.img -append "root=/dev/sda1 console=ttyS0" -net nic,macaddr=52:54:00:12:34:60 -net tap,ifname="tap01",script=no,downscript=no
7)虛機vm1啟動過程中可以看到dhcp獲取的ip信息
udhcpc (v1.20.1) started Sending discover... Sending select for 10.0.0.81... Lease of 10.0.0.81 obtained, lease time 86400
8）登錄進vm1虛機
login as 'cirros' user. default password: 'cubswin:)'. use 'sudo' for root. cirros login: 輸入用戶名cirros，密碼cubswin:) $ ip addr|grep eth0 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000 inet 10.0.0.81/24 brd 10.0.0.255 scope global eth0 $ ip route list default via 10.0.0.1 dev eth0 10.0.0.0/24 dev eth0 src 10.0.0.81
5.2.2創建tenant1虛機vm2
1）創建vm2掛接的網橋設備
brctl addbr qbr02 ip link set qbr02 up ip link add qvo02 type veth peer name qvb02 brctl addif qbr02 qvb02 ovs-vsctl add-port br-int qvo02 ip link set qvb02 up ip link set qvo02 up ovs-vsctl set port qvo02 tag=1 ip tuntap add tap02 mode tap ip link set dev tap02 up brctl addif qbr02 tap02
2）准備虛機映像文件
cp cirros-0.3.4-x86_64-disk.img cirros-vm2.img cp cirros-0.3.4-x86_64-kernel cirros-kernel-vm2
3）通過qemu啟動虛機vm2
qemu-system-x86_64 -nographic -kernel ./cirros-kernel-vm2 -hda cirros-vm2.img -append "root=/dev/sda1 console=ttyS0" -net nic,macaddr=52:54:00:12:34:61 -net tap,ifname="tap02",script=no,downscript=no
4）虛機vm2啟動過程中可以看到dhcp獲取的ip信息
udhcpc (v1.20.1) started Sending discover... Sending select for 10.0.0.82... Lease of 10.0.0.82 obtained, lease time 86400
5.2.3創建tenant2虛機vm3
1）在網絡節點添加tenant2流表
ovs-ofctl mod-flows br-tun "table=21,dl_vlan=2,actions=strip_vlan,set_tunnel:2,output:2" ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=1,table=2,tun_id=2,actions=mod_vlan_vid:2,resubmit(,10)"
2）在網絡節點添加tenant2的dhcp agent服務
ovs-vsctl -- --if-exists del-port tap02 -- add-port br-int tap02 -- set interface tap02 type=internal ovs-vsctl --timeout=10 set Port tap02 tag=2 ip netns add qdhcp02 ip netns exec qdhcp02 ip link set lo up ip link set tap02 netns qdhcp02 ip netns exec qdhcp02 ip link set tap02 up ip netns exec qdhcp02 ip -4 addr add 10.0.1.3/24 brd 10.0.1.255 scope global dev tap02 ip netns exec qdhcp02 ip route replace default via 10.0.1.1 dev tap02 ip netns exec qdhcp02 dnsmasq --no-hosts --no-resolv --strict-order --bind-interfaces --interface=tap02 --except-interface=lo --dhcp-range=set:tag0,10.0.1.6,10.0.1.90,86400s --dhcp-lease-max=256 --dhcp-option=3,10.0.1.1
3）在計算節點添加虛機tenant2流表
ovs-ofctl mod-flows br-tun "table=21,dl_vlan=2,actions=strip_vlan,set_tunnel:2,output:2" ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=1,table=2,tun_id=2,actions=mod_vlan_vid:2,resubmit(,10)"
4）創建vm3掛接的網橋設備
brctl addbr qbr03 ip link set qbr03 up ip link add qvo03 type veth peer name qvb03 brctl addif qbr03 qvb03 ovs-vsctl add-port br-int qvo03 ip link set qvb03 up ip link set qvo03 up ovs-vsctl set port qvo03 tag=2 ip tuntap add tap03 mode tap ip link set dev tap03 up brctl addif qbr03 tap03
5）准備虛機映像文件
cp cirros-0.3.4-x86_64-disk.img cirros-vm3.img cp cirros-0.3.4-x86_64-kernel cirros-kernel-vm3
6）通過qemu啟動虛機vm3
qemu-system-x86_64 -nographic -kernel ./cirros-kernel-vm3 -hda cirros-vm3.img -append "root=/dev/sda1 console=ttyS0" -net nic,macaddr=52:54:00:12:34:62 -net tap,ifname="tap03",script=no,downscript=no
7）虛機vm3啟動過程中可以看到dhcp獲取的ip信息
udhcpc (v1.20.1) started Sending discover... Sending select for 10.0.1.83... Lease of 10.0.1.83 obtained, lease time 86400
5.2.4創建tenant3虛機vm4
1）在網絡節點添加tenant3流表
ovs-ofctl add-flow br-int "hard_timeout=0,idle_timeout=0,priority=3,in_port=2,dl_vlan=103,actions=mod_vlan_vid:3,NORMAL" ovs-ofctl add-flow br-eth2 "hard_timeout=0,idle_timeout=0,priority=4,in_port=2,dl_vlan=3,actions=mod_vlan_vid:103,NORMAL"
2）在網絡節點添加tenant3的dhcp agent服務
ovs-vsctl -- --if-exists del-port tap03 -- add-port br-int tap03 -- set interface tap03 type=internal ovs-vsctl --timeout=10 set Port tap03 tag=3 ip netns add qdhcp03 ip netns exec qdhcp03 ip link set lo up ip link set tap03 netns qdhcp03 ip netns exec qdhcp03 ip link set tap03 up ip netns exec qdhcp03 ip -4 addr add 10.0.2.3/24 brd 10.0.2.255 scope global dev tap03 ip netns exec qdhcp03 ip route replace default via 10.0.2.1 dev tap03 ip netns exec qdhcp03 dnsmasq --no-hosts --no-resolv --strict-order --bind-interfaces --interface=tap03 --except-interface=lo --dhcp-range=set:tag0,10.0.2.6,10.0.2.90,86400s --dhcp-lease-max=256 --dhcp-option=3,10.0.2.1
3）在計算節點添加tenant3流表
ovs-ofctl add-flow br-int "hard_timeout=0,idle_timeout=0,priority=3,in_port=2,dl_vlan=103,actions=mod_vlan_vid:3,NORMAL" ovs-ofctl add-flow br-eth2 "hard_timeout=0,idle_timeout=0,priority=4,in_port=2,dl_vlan=3,actions=mod_vlan_vid:103,NORMAL"
4）創建vm4掛接的網橋設備
brctl addbr qbr04 ip link set qbr04 up ip link add qvo04 type veth peer name qvb04 brctl addif qbr04 qvb04 ovs-vsctl add-port br-int qvo04 ip link set qvb04 up ip link set qvo04 up ovs-vsctl set port qvo04 tag=3 ip tuntap add tap04 mode tap ip link set dev tap04 up brctl addif qbr04 tap04
5）准備虛機映像文件
cp cirros-0.3.4-x86_64-disk.img cirros-vm4.img cp cirros-0.3.4-x86_64-kernel cirros-kernel-vm4
6）通過qemu啟動虛機vm4
qemu-system-x86_64 -nographic -kernel ./cirros-kernel-vm4 -hda cirros-vm4.img -append "root=/dev/sda1 console=ttyS0" -net nic,macaddr=52:54:00:12:34:63 -net tap,ifname="tap04",script=no,downscript=no
7）虛機vm3啟動過程中可以看到dhcp獲取的ip信息
udhcpc (v1.20.1) started Sending discover... Sending select for 10.0.2.84... Lease of 10.0.2.84 obtained, lease time 86400
5.3各個租戶虛機創建完成后網絡功能驗證
5.3.1租戶虛機網絡環境驗證
1）在vm1中ping dhcp
$ ping 10.0.0.3 PING 10.0.0.3 (10.0.0.3): 56 data bytes 64 bytes from 10.0.0.3: seq=0 ttl=64 time=10.461 ms 64 bytes from 10.0.0.3: seq=1 ttl=64 time=3.099 ms 64 bytes from 10.0.0.3: seq=2 ttl=64 time=2.730 ms
同時在網絡節點qdhcp01命名空間中tap01設備上抓包
[root@ofs-network ~]# ip netns exec qdhcp01 tcpdump -i tap01 -enf 13:29:21.606157 92:7c:c4:bf:6c:20 > 52:54:00:12:34:60, ethertype ARP (0x0806), length 42: Request who-has 10.0.0.81 tell 10.0.0.3, length 28 13:29:21.610128 52:54:00:12:34:60 > 92:7c:c4:bf:6c:20, ethertype ARP (0x0806), length 42: Reply 10.0.0.81 is-at 52:54:00:12:34:60, length 28 13:29:21.620086 52:54:00:12:34:60 > 92:7c:c4:bf:6c:20, ethertype IPv4 (0x0800), length 98: 10.0.0.81 > 10.0.0.3: ICMP echo request, id 23553, seq 5, length 64 13:29:21.620199 92:7c:c4:bf:6c:20 > 52:54:00:12:34:60, ethertype IPv4 (0x0800), length 98: 10.0.0.3 > 10.0.0.81: ICMP echo reply, id 23553, seq 5, length 64 13:29:22.623044 52:54:00:12:34:60 > 92:7c:c4:bf:6c:20, ethertype IPv4 (0x0800), length 98: 10.0.0.81 > 10.0.0.3: ICMP echo request, id 23553, seq 6, length 64 13:29:22.623224 92:7c:c4:bf:6c:20 > 52:54:00:12:34:60, ethertype IPv4 (0x0800), length 98: 10.0.0.3 > 10.0.0.81: ICMP echo reply, id 23553, seq 6, length 64
2）在vm2中ping dhcp
$ ping 10.0.0.3 PING 10.0.0.3 (10.0.0.3): 56 data bytes 64 bytes from 10.0.0.3: seq=0 ttl=64 time=6.101 ms 64 bytes from 10.0.0.3: seq=1 ttl=64 time=2.937 ms 64 bytes from 10.0.0.3: seq=2 ttl=64 time=2.863 ms
同時在網絡節點qdhcp01命名空間中tap01設備上抓包
[root@ofs-network ~]# ip netns exec qdhcp01 tcpdump -i tap01 -enf 13:28:06.326290 52:54:00:12:34:61 > 92:7c:c4:bf:6c:20, ethertype ARP (0x0806), length 42: Request who-has 10.0.0.3 tell 10.0.0.82, length 28 13:28:06.326329 92:7c:c4:bf:6c:20 > 52:54:00:12:34:61, ethertype ARP (0x0806), length 42: Reply 10.0.0.3 is-at 92:7c:c4:bf:6c:20, length 28 13:28:23.593235 52:54:00:12:34:61 > 92:7c:c4:bf:6c:20, ethertype IPv4 (0x0800), length 98: 10.0.0.82 > 10.0.0.3: ICMP echo request, id 21761, seq 0, length 64 13:28:23.593297 92:7c:c4:bf:6c:20 > 52:54:00:12:34:61, ethertype IPv4 (0x0800), length 98: 10.0.0.3 > 10.0.0.82: ICMP echo reply, id 21761, seq 0, length 64 13:28:24.595536 52:54:00:12:34:61 > 92:7c:c4:bf:6c:20, ethertype IPv4 (0x0800), length 98: 10.0.0.82 > 10.0.0.3: ICMP echo request, id 21761, seq 1, length 64 13:28:24.595720 92:7c:c4:bf:6c:20 > 52:54:00:12:34:61, ethertype IPv4 (0x0800), length 98: 10.0.0.3 > 10.0.0.82: ICMP echo reply, id 21761, seq 1, length 64
3）在vm1中ping vm2
$ ping 10.0.0.82 PING 10.0.0.82 (10.0.0.82): 56 data bytes 64 bytes from 10.0.0.82: seq=0 ttl=64 time=61.110 ms 64 bytes from 10.0.0.82: seq=1 ttl=64 time=7.910 ms 64 bytes from 10.0.0.82: seq=2 ttl=64 time=4.795 ms
同時在計算節點上vm2虛機qbr02安全組橋上抓包
[root@ofs-compute ~]# tcpdump -i qbr02 -enf 13:31:57.142529 52:54:00:12:34:61 > 52:54:00:12:34:60, ethertype ARP (0x0806), length 42: Request who-has 10.0.0.81 tell 10.0.0.82, length 28 13:31:57.143956 52:54:00:12:34:60 > 52:54:00:12:34:61, ethertype ARP (0x0806), length 42: Reply 10.0.0.81 is-at 52:54:00:12:34:60, length 28 13:32:04.650250 52:54:00:12:34:60 > 52:54:00:12:34:61, ethertype IPv4 (0x0800), length 98: 10.0.0.81 > 10.0.0.82: ICMP echo request, id 24065, seq 0, length 64 13:32:04.650929 52:54:00:12:34:61 > 52:54:00:12:34:60, ethertype IPv4 (0x0800), length 98: 10.0.0.82 > 10.0.0.81: ICMP echo reply, id 24065, seq 0, length 64 13:32:05.652430 52:54:00:12:34:60 > 52:54:00:12:34:61, ethertype IPv4 (0x0800), length 98: 10.0.0.81 > 10.0.0.82: ICMP echo request, id 24065, seq 1, length 64 13:32:05.653128 52:54:00:12:34:61 > 52:54:00:12:34:60, ethertype IPv4 (0x0800), length 98: 10.0.0.82 > 10.0.0.81: ICMP echo reply, id 24065, seq 1, length 64
其他租戶虛機同樣方法可以驗證。
參考資料：
SammyLiu的《Neutron 理解》系列 http://www.cnblogs.com/sammyliu/p/4622563.html
深入理解Neutron -- OpenStack 網絡實現 https://www.gitbook.com/book/yeasy/openstack_understand_neutron/details