網絡結構
Ceph 使用以太網連接內部各存儲節點以及連接 client 和集群。Ceph 推薦使用兩個網絡：
- 前端（北向）網絡（ a public (front-side) network）：連接客戶端和集群 The public network handles client traffic and communication with Ceph monitors.
- 后端/東西向網絡 （a cluster (back-side) network）：連接 Ceph 各存儲節 The cluster (back-side) network handles OSD heartbeats, replication, backfilling and recovery traffic.
來自redHat的建議：
We recommend allocating bandwidth to the cluster (back-side) network such that it is a multiple of the front-side network using osd pool default size as the basis for your multiple. We also recommend running the public and cluster networks on separate NICs. We recommend keeping the cluster network separate from the public network and not connected to the internet to prevent DDOS attacks from crippling heartbeats, replication, backfilling and recovery.
網絡拓撲：
這么做，主要是從性能（OSD 節點之間會有大量的數據拷貝操作）和安全性（兩網分離）考慮。
你可以在 Ceph 配置文件的 [global] 部分配置兩個網絡：
public network = {public-network/netmask}
cluster network = {cluster-network/netmask}
參考：具體可以參考：
- Deploying Ceph with High Performance Networks, Architectures and benchmarks for Block Storage Solutions
- Deploying Ceph with High Performance Networks
- CHAPTER 2. NETWORKING RECOMMENDATIONS
- Ceph with a cluster and public network on IPv6 談到了 IPV6 的支持。
端口
Monitors use port 6789 by default. Ensure you have the port open for each monitor host. Each Ceph OSD Daemon on a Ceph Node may use up to three ports, beginning at port 6800:
- One for talking to clients and monitors. (front-side)
- One for sending data to other OSDs (replication, backfill and recovery). (back-side)
- One for heartbeating. (back-side)
You need to open at least three ports per OSD beginning at port 6800 on a Ceph node to ensure that the OSDs can peer. The port for talking to monitors and clients must be open on the public (front-side) network. The ports for sending data to other OSDs and heartbeating must be open on the cluster (back-side) network.
If you want to use a different port range than 6800:7100 for Ceph daemons, you must adjust the following settings in your Ceph configuration file:
ms bind port min = {min-port-num}
ms bind port max = {max-port-num}
Ceph monitors bind on port 6789 by default. If you want to use a different port number than 6789, you may specify the the IP address and port in your Ceph configuration. For example:
[mon.monname]
host = {hostname}
mon addr = {ip-address}:{port}
參考引用地址：