1.把15加入集群
2.安装ceph组件
2.1网关和监视器
2.2 osd创建 需要gpt分区
https://www.cnblogs.com/EasonJim/p/9583268.html parted /dev/sdb mklabel gpt y 然后在web界面重置驱动
3.手动创建一个文件系统
4.第一个是手动创建的 后面两个是自动创建的
5.块存储挂载
6.文件系统挂载 注意选择内容的选项
https://blog.51cto.com/yuweibing/2306831
删除pv
https://blog.csdn.net/qq_39626154/article/details/90477803
挂载目录
https://blog.51cto.com/kerry/2287648
ceph
https://www.jianshu.com/p/9a38408654b7
ceph2
M98WF-NY2PP-73243-PC8R6-V6B4Y Retail Key
报错
Degraded data redundancy: 8154/38763 objects degraded (21.036%), 89 pgs degraded, 89 pgs undersized
降级就是在发生了一些故障比如OSD挂掉之后，Ceph 将这个 OSD 上的所有 PG 标记为 Degraded。
降级的集群可以正常读写数据，降级的 PG 只是相当于小毛病而已，并不是严重的问题。
Undersized的意思就是当前存活的PG 副本数为 2，小于副本数3，将其做此标记，表明存货副本数不足，也不是严重的问题
/bin/ceph osd pool set test_pool min_size 1 #例子 set pool 1 min_size to 1 ceph osd pool set deyi min_size 1 #设置deyi池最小份数 set pool 1 min_size to 1
- 降级就是在发生了一些故障比如OSD挂掉之后，Ceph 将这个 OSD 上的所有 PG 标记为 Degraded。
- 降级的集群可以正常读写数据，降级的 PG 只是相当于小毛病而已，并不是严重的问题。
- Undersized的意思就是当前存活的PG 副本数为 2，小于副本数3，将其做此标记，表明存货副本数不足，也不是严重的问题。
作者：Lucien_168
链接：https://www.imooc.com/article/43575
来源：慕课网
本文原创发布于慕课网 ，转载请注明出处，谢谢合作
- 降级就是在发生了一些故障比如OSD挂掉之后，Ceph 将这个 OSD 上的所有 PG 标记为 Degraded。
- 降级的集群可以正常读写数据，降级的 PG 只是相当于小毛病而已，并不是严重的问题。
- Undersized的意思就是当前存活的PG 副本数为 2，小于副本数3，将其做此标记，表明存货副本数不足，也不是严重的问题。
作者：Lucien_168
链接：https://www.imooc.com/article/43575
来源：慕课网
本文原创发布于慕课网 ，转载请注明出处，谢谢合作
故障参考
https://www.imooc.com/article/43575
官方文档
http://docs.ceph.org.cn/rbd/rbd/
Haswell >QEMU 1162 726 Haswell, no TSX 1165.7 yum install -y wget yum install -y openssh-server systemctl restart sshd systemctl enable sshd
Config locked migrate
pve迁移失败解锁
qm unlock 120 pct unlock 120 虚拟机解锁
移除集群
First, make a backup of the cluster: cp -a /etc/pve /root/pve_backup Stop cluster service: /etc/init.d/pve-cluster stop Umount /etc/pve if it is mounted: umount /etc/pve Stop corosync service: /etc/init.d/cman stop Remove cluster configuration: rm /etc/cluster/cluster.conf rm -rf /var/lib/pve-cluster/* Start again cluster service: /etc/init.d/pve-cluster start Now, you can create new cluster: pvecm create newcluster Restore cluster and virtual machines configuration from the backup: cp /root/pve_backup/*.cfg /etc/pve/ cp /root/pve_backup/qemu-server/*.conf /etc/pve/qemu-server/ cp /root/pve_backup/openvz/* /etc/pve/openvz/ UPDATE: This post is also valid to change the hostname of a node in a cluster or to move a node between two clusters. When you have removed a node from the cluster, it still appears in the proxmox nodes tree, to remove it from the tree you have to delete the node directory from another node in the cluster: rm -rf /etc/pve/nodes/HOSTNAME https://blog.csdn.net/xiangrublog/article/details/42006465 Corosync Cluster Engine Authentication key generator. Gathering 2048 bits for key from /dev/urandom. Writing corosync key to /etc/corosync/authkey. Writing corosync config to /etc/pve/corosync.conf Restart corosync and cluster filesystem TASK OK systemctl stop pve-cluster systemctl stop corosync pmxcfs -l rm /etc/pve/corosync.conf rm /etc/corosync/* killall pmxcfs systemctl start pve-cluster
有时候集群web可能无法打开重启服务后正常
systemctl restart pve-cluster
pvecm delnode oldnode pvecm expected 1 rm /var/lib/corosync/*
如果迁移失败也需要重启集群服务还有sshd服务
一般是/etc/ssh/ssh_known_hosts文件内的指纹有问题
免密授权
ssh-copy-id root@192.168.1.15
systemctl restart pve-cluster
systemctl restart sshd
pve11 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/vdAfztZLQ8BwndORjsvMB0jrBx1wMcMCGUsdJm/zef3qznxGhN2nVo4aOge/JR22xWRDfue34k+rGq0EPyCBSQXeCuAUQXcLJOt9xh8NNd/Hto0QuSkSvicCxTVMSxs/7idm4dKL+V3eELnoL+k9mKKYa+qWY3oda5AezToI3Tu8FcGf/gOOyEVvHUyb16u7ZFP14Y9KVDNY4SP80Fxp/eRICOL3DCsjARLyTb5HfHy6FDwyX0U60US0gYtsNS1lcg6IHY8X9OjvAsMuvVo2Y6YjmHzySXWdJINjzuaNPc9FplA+HQ5pMkB1eg3slbaUPLDb3JFyKGUJi2WcHQ/Z 192.168.1.11 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/vdAfztZLQ8BwndORjsvMB0jrBx1wMcMCGUsdJm/zef3qznxGhN2nVo4aOge/JR22xWRDfue34k+rGq0EPyCBSQXeCuAUQXcLJOt9xh8NNd/Hto0QuSkSvicCxTVMSxs/7idm4dKL+V3eELnoL+k9mKKYa+qWY3oda5AezToI3Tu8FcGf/gOOyEVvHUyb16u7ZFP14Y9KVDNY4SP80Fxp/eRICOL3DCsjARLyTb5HfHy6FDwyX0U60US0gYtsNS1lcg6IHY8X9OjvAsMuvVo2Y6YjmHzySXWdJINjzuaNPc9FplA+HQ5pMkB1eg3slbaUPLDb3JFyKGUJi2WcHQ/Z pve13 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1w0Zob1ZZyzDjdPH4c5cm0rjhILVcQ1/KcA8JSXRLL2w5GrFbxEB8hvk+MTHug7CJcj7GsS/EY0I3YKA3wRdWVyG2LTKzCprILK/cdfVbSj7zGMLAP/iXLD0iKsNEZIIkto9acLgRBWNCb4P7Lz3vAdvYx04SZQschY7kxs4X8JTSboIfcV4xA8ACdy6JH46MXhicBTssdiU2GD/SSXis+uosaBcaoXElgrAnuuMcZaPp02fsrMgnOSeJ0mivZz4Biu2jDDWIAweWyupJimh3hUa8922hyhCF3s12h0ScZcfg9kcGw/twRp1h8JVTGrQHlJeSXwFIVSk0t6xOdkOd 192.168.1.13 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1w0Zob1ZZyzDjdPH4c5cm0rjhILVcQ1/KcA8JSXRLL2w5GrFbxEB8hvk+MTHug7CJcj7GsS/EY0I3YKA3wRdWVyG2LTKzCprILK/cdfVbSj7zGMLAP/iXLD0iKsNEZIIkto9acLgRBWNCb4P7Lz3vAdvYx04SZQschY7kxs4X8JTSboIfcV4xA8ACdy6JH46MXhicBTssdiU2GD/SSXis+uosaBcaoXElgrAnuuMcZaPp02fsrMgnOSeJ0mivZz4Biu2jDDWIAweWyupJimh3hUa8922hyhCF3s12h0ScZcfg9kcGw/twRp1h8JVTGrQHlJeSXwFIVSk0t6xOdkOd pve15 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5netAIihYgPT3tEk0oVQfzuNMHx3N12u59J9D8AHHMFlpxaQCxs98izSwGpVNcrSzy0hfJ1q4NJ3Ni8n1Er6Wiikr4heFcChPW2s14skg3fRnEj06msoRnZLBDP+2QTuG3gKX1mINhSotqa7v7KXLYLwLRzvvH2XZcUKT6YV32gLpUT7XruXlEdvjqGxkDiWhAUrJPRlhQXMy50L3R0tVC2ZhfHBc+kBwkC4han3d7Qtq7utwN9tloJg+nzuN/+HmZMli2oZjpwZEdbWx5Pd1Te9ImQShMivbUkbUnS69q4VA+cQlfnwgHTAUgMpQhe0/OTPrWnQRzsfI0wA/ES5h 192.168.1.15 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5netAIihYgPT3tEk0oVQfzuNMHx3N12u59J9D8AHHMFlpxaQCxs98izSwGpVNcrSzy0hfJ1q4NJ3Ni8n1Er6Wiikr4heFcChPW2s14skg3fRnEj06msoRnZLBDP+2QTuG3gKX1mINhSotqa7v7KXLYLwLRzvvH2XZcUKT6YV32gLpUT7XruXlEdvjqGxkDiWhAUrJPRlhQXMy50L3R0tVC2ZhfHBc+kBwkC4han3d7Qtq7utwN9tloJg+nzuN/+HmZMli2oZjpwZEdbWx5Pd1Te9ImQShMivbUkbUnS69q4VA+cQlfnwgHTAUgMpQhe0/OTPrWnQRzsfI0wA/ES5h
用nfs做共享迁移 qcow2迁移速度快可以用 raw格式迁移慢 会有问题 #查看集群资源状况 #pvesh get /cluster/resources #取得虚拟机当前状态 #pvesh get /nodes/<节点id>/qemu/<虚拟机id>/status/current
pvesh create /nodes/pve11/qemu/150/status/stop
共享存储移除 导致的虚拟机无法删除问题处理
删除虚拟机文件就可以了
rm -rf /etc/pve/nodes/pve15/lxc/105.conf #删除ct虚拟机
普通虚拟机是这个目录
/etc/pve/nodes/pve15/qemu-server/
帮助参考路径
https://192.168.1.xx:8006/pve-docs/chapter-pvecm.html#_remove_a_cluster_node
vi /etc/pve/nodes/pve13/qemu-server/105.conf
硬件设备异常可以手动删除
虚拟机配置文件和镜像文件
根据迁移后的位置修改硬盘路径应该就迁移成功了
ls /mnt/pve/bgdata/images ls /etc/pve/nodes/pve11/qemu-server/ cat /etc/pve/nodes/pve11/qemu-server/118.conf root@pve11:~# ls /mnt/pve/ bgdata cephfs deyi dydir nfs
另一种nfs迁移的方式
http://blog.sina.com.cn/s/blog_14b674edd0102xwc0.html
集群中剔除节点
1.集群节点查看
root@pve31:~# pvecm nodes
Membership information
----------------------
Nodeid Votes Name
1 1 pve33
2 1 pve32
3 1 pve31 (local)
2.关闭被删除的pve33节点电源或者关机后删除节点
root@pve31:~# pvecm delnode pve33
Killing node 1
3.查看集群信息pev33已经被移除集群
root@pve31:~# pvecm status
Quorum information
------------------
Date: Sat Oct 12 09:13:27 2019
Quorum provider: corosync_votequorum
Nodes: 2
Node ID: 0x00000003
Ring ID: 2/32
Quorate: Yes
Votequorum information
----------------------
Expected votes: 2
Highest expected: 2
Total votes: 2
Quorum: 2
Flags: Quorate
Membership information
----------------------
Nodeid Votes Name
0x00000002 1 192.168.130.32
0x00000003 1 192.168.130.31 (local)
集群其他主机vnc无法访问
原因是免密认证失败
ssh-keygen -f "/etc/ssh/ssh_known_hosts" -R "192.168.130.31"#删除认证重新认证就可以了 ssh 192.168.130.31
esxi集群管理文档
https://pubs.vmware.com/vsphere-51/index.jsp?topic=%2Fcom.vmware.vsphere.vcenterhost.doc%2FGUID-F14212C4-94D1-4DE0-B4B1-B9B6214AF055.html
虚拟机嵌套
https://blog.51cto.com/kusorz/1925172?cid=718307