一、kvm虛擬機熱添加技術
熱添加技術就是不停機的情況下，在線熱添加硬盤，內存，cpu，網卡等設備，熱添加技術一般都是在虛擬機資源不夠了，又不能停機的情況下使用的，熱添加技術是虛擬機相對於物理機的一個很大的優勢，它讓資源分配變得更靈活！
1.熱添加硬盤
1.1. 創建硬盤
[root@kvm01 ~]# virsh list --all
Id Name State
----------------------------------------------------
1 web01 running
[root@kvm01 ~]# qemu-img create -f qcow2 /data/centos7-add01.qcow2 5G
Formatting '/data/centos7-add01.qcow2', fmt=qcow2 size=5368709120 encryption=off cluster_size=65536 lazy_refcounts=off
1.2 熱添加硬盤
[root@kvm01 ~]# virsh attach-disk web01 /data/centos7-add01.qcow2 vdb --live --cache=none --subdriver=qcow2
Disk attached successfully
[root@kvm01 ~]# virsh console web01
Connected to domain web01
Escape character is ^]
CentOS Linux 7 (Core)
Kernel 3.10.0-514.el7.x86_64 on an x86_64
localhost login: root
Password:
Last login: Sat Sep 5 17:26:12 from 10.0.0.253
[root@localhost ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sr0 11:0 1 1024M 0 rom
vda 253:0 0 10G 0 disk
└─vda1 253:1 0 10G 0 part /
vdb 253:16 0 5G 0 disk
1.3 對新添加的磁盤進行分區，格式化，掛載等操作
[root@localhost ~]# fdisk -l
Disk /dev/vda: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000047ba
Device Boot Start End Blocks Id System
/dev/vda1 * 2048 20971519 10484736 83 Linux
Disk /dev/vdb: 5368 MB, 5368709120 bytes, 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
[root@localhost ~]# fdisk /dev/vdb
Welcome to fdisk (util-linux 2.23.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x1ef6234a.
Command (m for help): p
Disk /dev/vdb: 5368 MB, 5368709120 bytes, 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x1ef6234a
Device Boot Start End Blocks Id System
Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p
Partition number (1-4, default 1):
First sector (2048-10485759, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-10485759, default 10485759):
Using default value 10485759
Partition 1 of type Linux and of size 5 GiB is set
Command (m for help): p
Disk /dev/vdb: 5368 MB, 5368709120 bytes, 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x1ef6234a
Device Boot Start End Blocks Id System
/dev/vdb1 2048 10485759 5241856 83 Linux
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@localhost ~]# mkfs.ext4 /dev/vdb1
mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
327680 inodes, 1310464 blocks
65523 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1342177280
40 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736
Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
[root@localhost ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sr0 11:0 1 1024M 0 rom
vda 253:0 0 10G 0 disk
└─vda1 253:1 0 10G 0 part /
vdb 253:16 0 5G 0 disk
└─vdb1 253:17 0 5G 0 part
[root@localhost ~]# mount /dev/vdb1 /mnt
[root@localhost ~]# ll /mnt
total 16
drwx------. 2 root root 16384 Sep 10 22:44 lost+found
[root@localhost ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/vda1 9.8G 1.1G 8.2G 11% /
devtmpfs 488M 0 488M 0% /dev
tmpfs 497M 0 497M 0% /dev/shm
tmpfs 497M 6.6M 490M 2% /run
tmpfs 497M 0 497M 0% /sys/fs/cgroup
tmpfs 100M 0 100M 0% /run/user/0
/dev/vdb1 4.8G 20M 4.6G 1% /mnt
2. 熱添加網卡
[root@kvm01 ~]# virsh list --all
Id Name State
----------------------------------------------------
1 web01 running
2.1 熱添加網卡
[root@kvm01 ~]# virsh attach-interface web01 --type network --model virtio --source default
Interface attached successfully
[root@kvm01 ~]# virsh console web01
Connected to domain web01
Escape character is ^]
[root@localhost ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:f4:2d:06 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.160/24 brd 192.168.122.255 scope global dynamic eth0
valid_lft 1935sec preferred_lft 1935sec
inet6 fe80::4ff4:9fdc:8840:a641/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:74:74:45 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.64/24 brd 192.168.122.255 scope global dynamic eth1
valid_lft 3593sec preferred_lft 3593sec
inet6 fe80::8e05:c2f9:46b6:e8a4/64 scope link
valid_lft forever preferred_lft forever
3.熱添加cpu
3.1 添加cpu前，操作系統的個數
[root@kvm01 ~]# virt-install --virt-type kvm --os-type=linux --os-variant rhel6 --name web02 --memory 1024 --vcpus 1,maxvcpus=4 --disk /data/web02.qcow2,format=qcow2,size=10 --boot hd --network network=default --graphics vnc,listen=0.0.0.0 --noautoconsole
Starting install...
Domain creation completed.
[root@kvm01 ~]# virsh list --all
Id Name State
----------------------------------------------------
1 web01 running
2 web02 running
[root@kvm01 ~]# virsh console web02
Connected to domain web02
Escape character is ^]
Red Hat Enterprise Linux Server release 6.4 (Santiago)
Kernel 2.6.32-358.el6.x86_64 on an x86_64
localhost login: root
Password:
Last login: Sun Aug 30 22:50:01 from 10.0.0.253
[root@localhost ~]# cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 61
model name : Intel Core Processor (Broadwell, IBRS)
stepping : 2
cpu MHz : 2711.998
cache size : 4096 KB
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx rdtscp lm constant_tsc up rep_good unfair_spinlock pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx hypervisor lahf_lm 3dnowprefetch xsaveopt fsgsbase bmi1 avx2 smep bmi2 invpcid
bogomips : 5423.99
clflush size : 64
cache_alignment : 64
address sizes : 43 bits physical, 48 bits virtual
power management:
3.2 熱添加cpu
[root@kvm01 ~]# virsh setvcpus web02 --count=2
#查看系統cpu
[root@localhost ~]# cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 61
model name : Intel Core Processor (Broadwell, IBRS)
stepping : 2
cpu MHz : 2711.998
cache size : 4096 KB
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx rdtscp lm constant_tsc rep_good unfair_spinlock pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx hypervisor lahf_lm 3dnowprefetch xsaveopt fsgsbase bmi1 avx2 smep bmi2 invpcid
bogomips : 5423.99
clflush size : 64
cache_alignment : 64
address sizes : 43 bits physical, 48 bits virtual
power management:
processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 61
model name : Intel Core Processor (Broadwell, IBRS)
stepping : 2
cpu MHz : 2711.998
cache size : 4096 KB
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx rdtscp lm constant_tsc rep_good unfair_spinlock pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx hypervisor lahf_lm 3dnowprefetch xsaveopt fsgsbase bmi1 avx2 smep bmi2 invpcid
bogomips : 5423.99
clflush size : 64
cache_alignment : 64
address sizes : 43 bits physical, 48 bits virtual
power management:
4.熱添加內存
4.1 添加內存前操作系統
[root@kvm01 ~]# virt-install --virt-type kvm --os-type=linux --os-variant rhel6 --name web03 --memory 512,maxmemory=2048 --vcpus=1,maxvcpus=2 --disk /data/web03.qcow2,format=qcow2,size=10 --boot hd --network network=default --graphics vnc,listen=0.0.0.0 --noautoconsole
Starting install...
Domain creation completed.
[root@kvm01 ~]# virsh list --all
Id Name State
----------------------------------------------------
1 web01 running
3 web03 running
[root@kvm01 ~]# virsh console web03
Connected to domain web03
Escape character is ^]
Red Hat Enterprise Linux Server release 6.4 (Santiago)
Kernel 2.6.32-358.el6.x86_64 on an x86_64
localhost login: root
Password:
Last login: Thu Sep 10 23:29:22 on ttyS0
[root@localhost ~]# free -m
total used free shared buffers cached
Mem: 341 132 209 0 6 35
-/+ buffers/cache: 90 250
Swap: 2015 0 2015
4.2 熱添加內存
[root@kvm01 ~]# virsh setmem web03 1G
[root@localhost ~]# free -m
total used free shared buffers cached
Mem: 853 142 710 0 7 43
-/+ buffers/cache: 91 761
Swap: 2015 0 2015
注：內存已增加
二、kvm虛擬機熱遷移
1. 什么是熱遷移
相比KVM虛擬機冷遷移中需要拷貝虛擬機虛擬磁盤文件，kvm虛擬機熱遷移無需拷貝虛擬磁盤文件，但是需要遷移到的宿主機之間需要有相同的目錄結構虛擬機磁盤文件，也就是共享存儲，這次通過大家熟悉的nfs來實現，當然也可以采用Glusterfs等分布式文件系統來實現。
2.熱遷移過程
假設有2台宿主機kvm01和kvm02,在kvm01上掛起虛擬機vm01，發送vm的虛擬機配置文件和運行時內存中的數據到kvm02, 接受完畢，kvm02恢復vm01,熱遷移完成
3. 熱遷移架構圖
4. 環境准備
5. 實戰部署
5.1 安裝nfs插件（kvm插件已安裝）
[root@kvm01 ~]# yum install nfs-utils openssh-askpass -y
5.2 配置網橋
[root@kvm01 ~]# virsh iface-bridge eth0 br0
Created bridge br0 with attached device eth0
5.3 在nfs01上安裝配置nfs
[root@nfs01 ~]# yum install nfs-utils -y
[root@nfs01 ~]# vim /etc/exports
[root@nfs01 ~]# cat /etc/exports
/data 10.0.0.0/24 (rw,sync)
[root@nfs01 ~]# mkdir /data/opt
[root@nfs01 ~]# chown -R nfsnobody:nfsnobody /data
[root@nfs01 ~]# systemctl restart rpcbind
[root@nfs01 ~]# systemctl restart nfs
[root@nfs01 ~]# showmount -e
Export list for nfs01:
/data 10.0.0.0/24
5.4 在kvm01和kvm02上掛載共享目錄
[root@kvm01 ~]# showmount -e 10.0.0.31
Export list for 10.0.0.31:
/data 10.0.0.0/24
[root@kvm01 ~]# mount -t nfs 10.0.0.31:/data /opt
[root@kvm01 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda3 48G 18G 31G 37% /
devtmpfs 2.0G 0 2.0G 0% /dev
tmpfs 2.0G 0 2.0G 0% /dev/shm
tmpfs 2.0G 8.7M 2.0G 1% /run
tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
/dev/sda1 1014M 116M 899M 12% /boot
tmpfs 394M 0 394M 0% /run/user/0
10.0.0.31:/data 18G 3.5G 15G 19% /opt
5.5 安裝一台基於橋接模式的虛擬機
將磁盤鏡像文件web02.qcow2上傳到10.0.0.31:/data目錄下，可以看到kvm01和kvm02的/opt目錄下都有此磁盤文件
[root@nfs01 ~]# ll /data
total 10487616
-rw-r--r-- 1 nfsnobody nfsnobody 10739318784 Sep 12 17:16 web02.qcow2
[root@kvm01 ~]# ll /opt
total 10487616
-rw-r--r-- 1 nfsnobody nfsnobody 10739318784 Sep 12 17:16 web02.qcow2
[root@kvm02 ~]# ll /opt
total 10487616
-rw-r--r-- 1 nfsnobody nfsnobody 10739318784 Sep 12 17:16 web02.qcow2
#在kvm01上安裝一台虛擬機
[root@kvm01 ~]# virt-install --virt-type kvm --os-type=linux --os-variant rhel6 --name web02 --memory 512,maxmemory=2048 --vcpus 1 --disk /opt/web02.qcow2 --boot hd --network bridge=br0 --graphics vnc,listen=0.0.0.0 --noautoconsole
Starting install...
Domain creation completed.
[root@kvm01 ~]# virsh list --all
Id Name State
----------------------------------------------------
2 web02 running
- web01 shut off
- web03 shut off
5.5 將宿主機10.0.0.131上的kvm虛擬機web03遷移到10.0.0.7（命令行遷移）
[root@kvm01 ~]# virsh migrate --live --verbose web02 qemu+ssh://10.0.0.7/system --unsafe
root@10.0.0.7's password:
Migration: [100 %]
[root@kvm01 ~]# virsh list --all
Id Name State
----------------------------------------------------
- web01 shut off
- web02 shut off
- web03 shut off
#在kvm02上查看
[root@kvm02 ~]# virsh list --all
Id Name State
----------------------------------------------------
1 web02 running
[root@kvm02 ~]# virsh console web02
Connected to domain web02
Escape character is ^]
Red Hat Enterprise Linux Server release 6.4 (Santiago)
Kernel 2.6.32-358.el6.x86_64 on an x86_64
localhost login: root
Password:
Last login: Sun Aug 30 22:50:01 from 10.0.0.253
[root@localhost ~]# uname -a
Linux localhost 2.6.32-358.el6.x86_64 #1 SMP Tue Jan 29 11:47:41 EST 2013 x86_64 x86_64 x86_64 GNU/Linux
至此遷移成功！！！！
5.6 在kvm01上安裝圖形界面、vnc服務端和virt-manager（工具遷移）
[root@kvm01 ~]# yum groups install "GNOME Desktop" -y
[root@kvm01 ~]# yum install tigervnc-server -y
[root@kvm01 ~]# yum install virt-manager -y
5.7 啟動vnc服務端
vncserver :1 啟動5901端口的vnc服務端
[root@kvm01 ~]# vncserver :1
You will require a password to access your desktops.
Password:
Verify:
Would you like to enter a view-only password (y/n)? n
A view-only password is not used
New 'kvm01:1 (root)' desktop is kvm01:1
Creating default startup script /root/.vnc/xstartup
Creating default config /root/.vnc/config
Starting applications specified in /root/.vnc/xstartup
Log file is /root/.vnc/kvm01:1.lo
vncserver -kill :1 關閉5901端口的vnc服務端
[root@kvm01 ~]# vncserver -kill :1
Killing Xvnc process ID 1656
[root@kvm01 ~]# vncserver :1
New 'kvm01:1 (root)' desktop is kvm01:1
Starting applications specified in /root/.vnc/xstartup
Log file is /root/.vnc/kvm01:1.log
[root@kvm01 ~]# netstat -lntup
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:5901 0.0.0.0:* LISTEN 2150/Xvnc
5.8 使用vnc連接宿主機，使用virt-manager進行遷移
#遷移前
[root@kvm01 ~]# virsh list --all
Id Name State
----------------------------------------------------
2 web02 running
- web01 shut off
- web03 shut off
[root@kvm02 ~]# virsh list --all
Id Name State
----------------------------------------------------
#vnc登錄，實施遷移
遷移完成！！！
查看kvm01和kvm02虛擬機狀況
[root@kvm01 ~]# virsh list --all
Id Name State
----------------------------------------------------
- web01 shut off
- web03 shut off
[root@kvm02 ~]# virsh list --all
Id Name State
----------------------------------------------------
2 web02 running
[root@kvm02 ~]# virsh console web02
Connected to domain web02
Escape character is ^]
Red Hat Enterprise Linux Server release 6.4 (Santiago)
Kernel 2.6.32-358.el6.x86_64 on an x86_64
localhost login: root
Password:
Last login: Sat Sep 12 17:22:59 on ttyS0
[root@localhost ~]# uname -a
Linux localhost 2.6.32-358.el6.x86_64 #1 SMP Tue Jan 29 11:47:41 EST 2013 x86_64 x86_64 x86_64 GNU/Linux