隨着數據量持續增多,后續遲早會出現一台機器硬件瓶頸問題的。而mongodb主打的就是海量數據架構,“分片”就用這個來解決這個問題。
從圖中可以看到有四個組件:mongos、config server、shard、replica set。
mongos,數據庫集群請求的入口,所有的請求都通過mongos進行協調,不需要在應用程序添加一個路由選擇器,mongos自己就是一個請求分發中心,它負責把對應的數據請求請求轉發到對應的shard服務器上。在生產環境通常有多mongos作為請求的入口,防止其中一個掛掉所有的mongodb請求都沒有辦法操作。
config server,顧名思義為配置服務器,存儲所有數據庫元信息(路由、分片)的配置。mongos本身沒有物理存儲分片服務器和數據路由信息,只是緩存在內存里,配置服務器則實際存儲這些數據。mongos第一次啟動或者關掉重啟就會從 config server 加載配置信息,以后如果配置服務器信息變化會通知到所有的 mongos 更新自己的狀態,這樣 mongos 就能繼續准確路由。在生產環境通常有多個 config server 配置服務器,因為它存儲了分片路由的元數據,防止數據丟失!
shard,分片(sharding)是指將數據庫拆分,將其分散在不同的機器上的過程。將數據分散到不同的機器上,不需要功能強大的服務器就可以存儲更多的數據和處理更大的負載。基本思想就是將集合切成小塊,這些塊分散到若干片里,每個片只負責總數據的一部分,最后通過一個均衡器來對各個分片進行均衡(數據遷移)。
replica set,中文翻譯副本集,其實就是shard的備份,防止shard掛掉之后數據丟失。復制提供了數據的冗余備份,並在多個服務器上存儲數據副本,提高了數據的可用性, 並可以保證數據的安全性。
仲裁者(Arbiter),是復制集中的一個MongoDB實例,它並不保存數據。仲裁節點使用最小的資源並且不要求硬件設備,不能將Arbiter部署在同一個數據集節點中,可以部署在其他應用服務器或者監視服務器中,也可部署在單獨的虛擬機中。為了確保復制集中有奇數的投票成員(包括primary),需要添加仲裁節點做為投票,否則primary不能運行時不會自動切換primary。.
總結一下,應用請求mongos來操作mongodb的增刪改查,配置服務器存儲數據庫元信息,並且和mongos做同步,數據最終存入在shard(分片)上,為了防止數據丟失同步在副本集中存儲了一份,仲裁在數據存儲到分片的時候決定存儲到哪個節點
config server存儲分片集群的元數據。元數據反應了分片集群內所有數據和組件的狀態和組織,它包括每個分片的chunk列表及定義chunk的范圍。
mongos 實例緩存這些數據,並將讀寫操作路由到正確的分片上。mongos在集群發生元數據更改(例如,“塊拆分”或者添加shard)是更新緩存,shard從config server讀取chunk元數據。
config server 還存儲身份驗證配置信息,如基於角色的訪問控制或集群的內部身份驗證設置。
config server可用性
1)如果所有的config server都變得不可用,則集群可能無法使用。為確保config server保持可用且完好無損,config server的備份很重要。config server上的數據及存儲與整個shard集群的數據相比比較小,並且config server的負載相對較低。
2)config server 將元數據存儲在config庫中,在對config server上進行任何維護之前,應始終備份config庫。
下面開始准備部署mongodb分片群集
這個架構是不是足夠好呢?其實還有很多地方可以優化,比如我們把所有的仲裁節點放在一台機器,其余兩台機器承擔了全部讀寫操作,但是作為仲裁節點的機器非常的空閑的,例如機器出現故障,會直接導致整個分片不可用。
所以,我們采取下面這種結構來部署
環境部署:mongodb版本3.6.6
服務器214 服務器215 服務器216
mongos mongos mongos
config server config server config server
shard server1 主節點 shard server1 副節點 shard server1 仲裁
shard server2 仲裁 shard server2 主節點 shard server2 副節點
shard server3 副節點 shard server3 仲裁 shard server3 主節點
1、端口分配
mongos:20000
config:21000
shard1:27001
shard2:27002
shard3:27003
2、安裝部署mongodb
配置文件路徑:/etc/mongodb
數據存放路徑:/data/mongodb
啟動對應服務:mongod -f logpath
關閉對應服務:mongod --shutdown -f logpath
分別在每台機器建立mongos、config、shard1、shard2、shard3六個目錄,因為mongos不存儲數據,只需要建立日志文件目錄即可
[root@localhost ~]# cat mkdir.sh #!/bin/bash
mkdir /etc/mongodb/ mkdir -p /data/mongodb/mongos/log mkdir -p /data/mongodb/config/data mkdir -p /data/mongodb/config/log mkdir -p /data/mongodb/shard1/data mkdir -p /data/mongodb/shard1/log mkdir -p /data/mongodb/shard2/data mkdir -p /data/mongodb/shard2/log mkdir -p /data/mongodb/shard3/data mkdir -p /data/mongodb/shard3/log
3、config server集群配置
mongodb3.4以后要求配置服務器也創建副本集,不然集群搭建不成功
[root@localhost ~]# cat /etc/mognodb/config.conf |egrep -v "^$|^#" systemLog: destination: file logAppend: true path: /var/log/mongodb/config.log storage: #dbPath: /data/mongodb dbPath: /data/mongodb/config/data journal: enabled: true processManagement: fork: true # fork and run in background pidFilePath: /var/run/mongodb/config.pid # location of pidfile timeZoneInfo: /usr/share/zoneinfo net: port: 21000 #bindIp: 127.0.0.1 # Listen to local interface only, comment to listen on all interfaces. bindIp: 192.168.214.214 # Listen to local interface only, comment to listen on all interfaces. sharding: clusterRole: configsvr replication: replSetName: configs
啟動三台服務器的config server
[root@localhost ~]# mongod -f /etc/mongodb/config.conf about to fork child process, waiting until server is ready for connections. forked process: 2154 child process started successfully, parent exiting
登錄任意一台配置服務器,初始化配置副本集
[root@localhost ~]# mongo 192.168.214.214:21000
> config = { ... id : "configs", ... members : [ ... {_id : 0, host : "192.168.214.214:21000" }, ... {_id : 1, host : "192.168.214.215:21000" }, ... {_id : 2, host : "192.168.214.216:21000" } ... ] ... } { "id" : "configs", "members" : [ { "_id" : 0, "host" : "192.168.214.214:21000" }, { "_id" : 1, "host" : "192.168.214.215:21000" }, { "_id" : 2, "host" : "192.168.214.216:21000" } ] }
# 初始化配置 rs.initiate(config)
其中,"_id" : "configs"應與配置文件中配置的 replicaction.replSetName 一致,"members" 中的 "host" 為三個節點的 ip 和 port
4、分片副本集配置
1)配置第一個分片副本集shard1
[root@localhost ~]# cat /etc/mongodb/shard1.conf |egrep -v "^$|^#" systemLog: destination: file logAppend: true path: /data/mongodb/shard1/shard1.log storage: #dbPath: /var/lib/mongo dbPath: /data/mongodb/shard1/data journal: enabled: true processManagement: fork: true # fork and run in background pidFilePath: /var/run/mongodb/shard1.pid # location of pidfile timeZoneInfo: /usr/share/zoneinfo net: port: 27001 #bindIp: 127.0.0.1 # Listen to local interface only, comment to listen on all interfaces. bindIp: 192.168.214.214 # Listen to local interface only, comment to listen on all interfaces. sharding: clusterRole: shardsvr replication: replSetName: shard1
啟動三台服務器的shard1 server
[root@localhost ~]# mongod -f /etc/mongodb/config.conf about to fork child process, waiting until server is ready for connections. forked process: 2360 child process started successfully, parent exiting
登陸任意一台服務器,初始化副本集
[root@localhost ~]# mongo 192.168.214.214:27001
> config = { ... _id : "shard1", ... members : [ ... {_id : 0, host : "192.168.214.214:27001" }, ... {_id : 1, host : "192.168.214.215:27001" }, ... {_id : 2, host : "192.168.214.216:27001" , arbiterOnly: true} ... ] ... } { "_id" : "shard1", "members" : [ { "_id" : 0, "host" : "192.168.214.214:27001" }, { "_id" : 1, "host" : "192.168.214.215:27001" }, { "_id" : 2, "host" : "192.168.214.216:27001" } ] }
# 初始化配置 rs.initiate(config)
2)配置第二個分片副本集shard2
[root@localhost mongodb]# cat /etc/mongodb/shard2.conf |egrep -v "^#|^$" systemLog: destination: file logAppend: true path: /data/mongodb/shard2/shard2.log storage: #dbPath: /var/lib/mongo dbPath: /data/mongodb/shard2/data journal: enabled: true processManagement: fork: true # fork and run in background pidFilePath: /var/run/mongodb/shard2.pid # location of pidfile timeZoneInfo: /usr/share/zoneinfo net: port: 27002 #bindIp: 127.0.0.1 # Listen to local interface only, comment to listen on all interfaces. bindIp: 192.168.214.215 # Listen to local interface only, comment to listen on all interfaces. sharding: clusterRole: shardsvr replication: replSetName: shard2
啟動三台服務器的shard2 server
[root@localhost mongodb]# mongod -f /etc/mongodb/shard2.conf about to fork child process, waiting until server is ready for connections. forked process: 2482 child process started successfully, parent exiting
陸任意一台服務器,初始化副本集
[root@localhost mongodb]# mongo 192.168.214.215:27002
> config = { ... _id : "shard2", ... members : [ ... {_id : 0, host : "192.168.214.214:27002" , arbiterOnly: true }, ... {_id : 1, host : "192.168.214.215:27002" }, ... {_id : 2, host : "192.168.214.216:27002" } ... ] ... } { "_id" : "shard2", "members" : [ { "_id" : 0, "host" : "192.168.214.214:27002", "arbiterOnly" : true }, { "_id" : 1, "host" : "192.168.214.215:27002" }, { "_id" : 2, "host" : "192.168.214.216:27002" } ] }
# 初始化配置 rs.initiate(config)
3)配置第三個分片副本集shard3
[root@localhost mongodb]# cat /etc/mongodb/shard3.conf |egrep -v "^$|^#" systemLog: destination: file logAppend: true path: /data/mongodb/shard3/shard3.log storage: #dbPath: /var/lib/mongo dbPath: /data/mongodb/shard3/data journal: enabled: true processManagement: fork: true # fork and run in background pidFilePath: /var/run/mongodb/shard3.pid # location of pidfile timeZoneInfo: /usr/share/zoneinfo net: port: 27003 #bindIp: 127.0.0.1 # Listen to local interface only, comment to listen on all interfaces. bindIp: 192.168.214.216 # Listen to local interface only, comment to listen on all interfaces. sharding: clusterRole: shardsvr replication: replSetName: shard3
啟動三台服務器的shard3 server
[root@localhost mongodb]# mongod -f shard3.conf about to fork child process, waiting until server is ready for connections. forked process: 2755 child process started successfully, parent exiting
陸任意一台服務器,初始化副本集
> config = { ... _id : "shard3", ... members : [ ... {_id : 0, host : "192.168.214.214:27003" }, ... {_id : 1, host : "192.168.214.215:27003" , arbiterOnly: true}, ... {_id : 2, host : "192.168.214.216:27003" } ... ] ... } { "_id" : "shard3", "members" : [ { "_id" : 0, "host" : "192.168.214.214:27003" }, { "_id" : 1, "host" : "192.168.214.215:27003", "arbiterOnly" : true }, { "_id" : 2, "host" : "192.168.214.216:27003" } ] }
> rs.initiate(config); { "ok" : 1 }
5、配置路由服務器mongos
先啟動配置服務器和分片服務器,后啟動路由實例啟動路由實例:(三台機器)
[root@localhost mongodb]# cat mongos.conf |egrep -v "^#|^$"
systemLog:
destination: file
logAppend: true
path: /data/mongodb/mongos/mongos.log
#dbPath: /var/lib/mongo
processManagement:
fork: true # fork and run in background
pidFilePath: /var/run/mongodb/mongos.pid # location of pidfile
timeZoneInfo: /usr/share/zoneinfo
net:
port: 20000
#bindIp: 127.0.0.1 # Listen to local interface only, comment to listen on all interfaces.
bindIp: 192.168.214.214 # Listen to local interface only, comment to listen on all interfaces.
#監聽的配置服務器,只能有1個或者3個 configs為配置服務器的副本集名字
sharding:
configDB: configs/192.168.214.214:21000,192.168.214.215:21000,192.168.214.216:21000
#設置最大連接數
maxConns=20000
啟動三台服務器的mongos server
[root@localhost mongodb]# mongos -f /etc/mongodb/mongos.conf about to fork child process, waiting until server is ready for connections. forked process: 2858 child process started successfully, parent exiting
6、啟用分片
目前搭建了mongodb配置服務器、路由服務器,各個分片服務器,不過應用程序連接到mongos路由服務器並不能使用分片機制,還需要在程序里設置分片配置,讓分片生效。
片鍵是集合的一個鍵,MongoDB根據這個鍵拆分數據。例如,如果選擇基於“username”進行分片,MongoDB會根據不同的用戶名進行分片。選擇片鍵可以認為是選擇集合中數據的順序。它與索引是個相似的概念:隨着集合的不斷增長,片鍵就會成為集合上最重要的索引。只有被索引過的鍵才能夠作為片鍵
[root@localhost ~]# mongo 192.168.214.214:20000
#使用admin數據庫
mongos> use admin switched to db admin
#串聯路由服務器與分配副本集 mongos> sh.addShard("shard1/192.168.214.214:27001,192.168.214.215:27001,192.168.214.216:27001") 27003"){ "shardAdded" : "shard1", "ok" : 1, "$clusterTime" : { "clusterTime" : Timestamp(1555544056, 5944), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } }, "operationTime" : Timestamp(1555544056, 5944) } mongos> sh.addShard("shard2/192.168.214.214:27002,192.168.214.215:27002,192.168.214.216:27002") { "shardAdded" : "shard2", "ok" : 1, "$clusterTime" : { "clusterTime" : Timestamp(1555544056, 5948), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } }, "operationTime" : Timestamp(1555544056, 5948) } mongos> sh.addShard("shard3/192.168.214.214:27003,192.168.214.215:27003,192.168.214.216:27003") { "shardAdded" : "shard3", "ok" : 1, "$clusterTime" : { "clusterTime" : Timestamp(1555544056, 5955), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } }, "operationTime" : Timestamp(1555544056, 5955) }
#查看集群狀態
mongos> sh.status() --- Sharding Status --- sharding version: { "_id" : 1, "minCompatibleVersion" : 5, "currentVersion" : 6, "clusterId" : ObjectId("5cb6b39a03fdf8f1e8c882d1") } shards: { "_id" : "shard1", "host" : "shard1/192.168.214.214:27001,192.168.214.215:27001", "state" : 1 } { "_id" : "shard2", "host" : "shard2/192.168.214.215:27002,192.168.214.216:27002", "state" : 1 } { "_id" : "shard3", "host" : "shard3/192.168.214.214:27003,192.168.214.216:27003", "state" : 1 } most recently active mongoses: "3.6.6" : 1 autosplit: Currently enabled: yes balancer: Currently enabled: yes Currently running: no Failed balancer rounds in last 5 attempts: 0 Migration Results for the last 24 hours: No recent migrations databases: { "_id" : "config", "primary" : "config", "partitioned" : true }
7、測試
目前配置服務、路由服務、分片服務、副本集服務都已經串聯起來了,但我們的目的是希望插入數據,數據能夠自動分片。連接在mongos上,准備讓指定的數據庫、指定的集合分片生效。
設置分片chunk大小
use config db.settings.save({ "_id" : "chunksize", "value" : 1 }) 設置1M是為了測試,否則要插入大量數據才能分片。
激活數據庫分片功能
語法:( { enablesharding : "數據庫名稱" } )
mongos> db.runCommand( { enablesharding : "test" } )
或者mongos>sh.enableSharding("test")
##use到admin庫下執行,否則會報錯
mongos>use admin
switched to db admin
mongos> sh.enableSharding("test") { "ok" : 1, "$clusterTime" : { "clusterTime" : Timestamp(1555544056, 6258), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } }, "operationTime" : Timestamp(1555544056, 6258) }
##指定數據庫里需要分片的集合和片鍵
#指定數據庫里需要分片的集合和片鍵 use test db.users.createIndex({user_id : 1})
db.runCommand( { shardcollection : "test.users",key : {user_id: 1} } )
或者sh.shardCollection("test.users", {user_id: 1})
mongos> sh.shardCollection("test.users", {user_id: 1}) { "collectionsharded" : "test.users", "collectionUUID" : UUID("da124f00-1ddd-445d-8bb9-d443a4b113ac"), "ok" : 1, "$clusterTime" : { "clusterTime" : Timestamp(1555544056, 6427), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } }, "operationTime" : Timestamp(1555544056, 6427) }
我們設置test的users表需要分片,根據 id 自動分片到 shard1 ,shard2,shard3 上面去。要這樣設置是因為不是所有mongodb 的數據庫和表 都需要分片!
##插入數據測試分片效果
mongos> use test switched to db test mongos> for (var i = 1; i <= 1000000; i++) db.users.save({user_id:i,username:"user"+i}); WriteResult({ "nInserted" : 1 })
#查看分片情況如下,此處省略其他信息
db.集合.stats();
mongos> db.users.stats() { "sharded" : true, "capped" : false, "ns" : "test.users", "count" : 177864, "size" : 11272191, "storageSize" : 4935680, "totalIndexSize" : 4984832, "indexSizes" : { "_id_" : 2232320, "user_id_1" : 2752512 }, "avgObjSize" : 63, "nindexes" : 2, "nchunks" : 21, "shards" : { "shard1" : { "ns" : "test.users", "size" : 3401817, "count" : 53604, "avgObjSize" : 63, "storageSize" : 2142208, "capped" : false, "wiredTiger" : { "metadata" : { "formatVersion" : 1 }, "shard2" : { "ns" : "test.users", "size" : 3417868, "count" : 53971, "avgObjSize" : 63, "storageSize" : 1355776, "capped" : false, "wiredTiger" : { "metadata" : { "formatVersion" : 1 }, "shard3" : { "ns" : "test.users", "size" : 4452506, "count" : 70289, "avgObjSize" : 63, "storageSize" : 1437696, "capped" : false, "wiredTiger" : { "metadata" : { "formatVersion" : 1 }, }
上面集合的基本信息,可以看到分片成功,各個分片都有數據(count)
#查看分片信息
mongos> db.printShardingStatus()
或mongos>sh.status()
mongos> sh.status() --- Sharding Status --- sharding version: { "_id" : 1, "minCompatibleVersion" : 5, "currentVersion" : 6, "clusterId" : ObjectId("5cb6b39a03fdf8f1e8c882d1") } shards: { "_id" : "shard1", "host" : "shard1/192.168.214.214:27001,192.168.214.215:27001", "state" : 1 } { "_id" : "shard2", "host" : "shard2/192.168.214.215:27002,192.168.214.216:27002", "state" : 1 } { "_id" : "shard3", "host" : "shard3/192.168.214.214:27003,192.168.214.216:27003", "state" : 1 } active mongoses: "3.6.6" : 1 autosplit: Currently enabled: yes balancer: Currently enabled: yes Currently running: no Collections with active migrations: wsy.wsy started at Wed Apr 24 2019 18:42:13 GMT+0800 (CST) Failed balancer rounds in last 5 attempts: 0 Migration Results for the last 24 hours: 8 : Success databases: { "_id" : "config", "primary" : "config", "partitioned" : true } config.system.sessions shard key: { "_id" : 1 } unique: false balancing: true chunks: shard1 1 { "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : shard1 Timestamp(1, 0) { "_id" : "test", "primary" : "shard2", "partitioned" : true } test.users shard key: { "user_id" : 1 } unique: false balancing: true chunks: shard1 7 shard2 7 shard3 7 too many chunks to print, use verbose if you want to force print { "_id" : "wsy", "primary" : "shard1", "partitioned" : true } wsy.wsy shard key: { "user_id" : 1 } unique: false balancing: true chunks: shard1 2 shard2 3 shard3 4 { "user_id" : { "$minKey" : 1 } } -->> { "user_id" : 2 } on : shard2 Timestamp(5, 1) { "user_id" : 2 } -->> { "user_id" : 16914 } on : shard3 Timestamp(3, 0) { "user_id" : 16914 } -->> { "user_id" : 25370 } on : shard1 Timestamp(4, 1) { "user_id" : 25370 } -->> { "user_id" : 38126 } on : shard1 Timestamp(3, 3) { "user_id" : 38126 } -->> { "user_id" : 46582 } on : shard2 Timestamp(4, 2) { "user_id" : 46582 } -->> { "user_id" : 56101 } on : shard2 Timestamp(4, 3) { "user_id" : 56101 } -->> { "user_id" : 64557 } on : shard3 Timestamp(5, 2) { "user_id" : 64557 } -->> { "user_id" : 74077 } on : shard3 Timestamp(5, 3) { "user_id" : 74077 } -->> { "user_id" : { "$maxKey" : 1 } } on : shard3 Timestamp(5, 4)
至此,mongodb數據分片集群部署並測試完成。
出現:
too many chunks to print, use verbose if you want to force print
想要看到詳細的信息則需要執行:
mongos> sh.status({"verbose":1}) 或則 mongos> db.printShardingStatus("vvvv") 或則 mongos> printShardingStatus(db.getSisterDB("config"),1)
8、chunk
在一個shard server內部,MongoDB還是會把數據分為chunks,每個chunk代表這個shard server內部一部分數據。chunk的產生,會有以下兩個用途:
Splitting:當一個chunk的大小超過配置中的chunk size時,MongoDB的后台進程會把這個chunk切分成更小的chunk,從而避免chunk過大的情況。
Balancing:在MongoDB中,balancer是一個后台進程,負責chunk的遷移,從而均衡各個shard server的負載,系統初始1個chunk,chunk size默認值64M,生產庫上選擇適合業務的chunk size是最好的。MongoDB會自動拆分和遷移chunks。
分片集群的數據分布(shard節點)
(1)使用chunk來存儲數據
(2)進群搭建完成之后,默認開啟一個chunk,大小是64M,
(3)存儲需求超過64M,chunk會進行分裂,如果單位時間存儲需求很大,設置更大的chunk
(4)chunk會被自動均衡遷移。
1)chunksize的選擇
小的chunksize:數據均衡是遷移速度快,數據分布更均勻。數據分裂頻繁,路由節點消耗更多資源。大的chunksize:數據分裂少。數據塊移動集中消耗IO資源。通常100-200M
2)chunk分裂與遷移
隨着數據的增長,其中的數據大小超過了配置的chunk size,默認是64M,則這個chunk就會分裂成兩個。數據的增長會讓chunk分裂得越來越多
此時,每個shard 上的chunk數量就會不平衡。這時候,mongos中的一個組件balancer 就會執行自動平衡。把chunk從chunk數量最多的shard節點挪動到數量最少的節點
3)chunksize對分裂及遷移的影響
MongoDB 默認的 chunkSize 為64MB,如無特殊需求,建議保持默認值;chunkSize 會直接影響到 chunk 分裂、遷移的行為。
chunkSize 越小,chunk 分裂及遷移越多,數據分布越均衡;反之,chunkSize 越大,chunk 分裂及遷移會更少,但可能導致數據分布不均。
chunkSize 太小,容易出現 jumbo chunk(即shardKey 的某個取值出現頻率很高,這些文檔只能放到一個 chunk 里,無法再分裂)而無法遷移;chunkSize 越大,則可能出現 chunk 內文檔數太多(chunk 內文檔數不能超過 250000 )而無法遷移。
chunk 自動分裂只會在數據寫入時觸發,所以如果將 chunkSize 改小,系統需要一定的時間來將 chunk 分裂到指定的大小。
chunk 只會分裂,不會合並,所以即使將 chunkSize 改大,現有的 chunk 數量不會減少,但 chunk 大小會隨着寫入不斷增長,直到達到目標大小。
9、balancer操作
在上面步驟中,本來每個分片上只有一個塊,最后最后在shard0000、shard0001、shard0002都被拆分了,這是因為mongos在收到寫請求的時候,會檢查當前塊的拆分閥值點。到達該閥值的時候,會向分片發起一個拆分的請求。分片內的數據進行了遷移(有一定的消耗),最后通過一個均衡器來對數據進行轉移分配
均衡器:均衡器負責數據遷移,周期性的檢查分片是否存在不均衡,如果不存在則會開始塊的遷移,config.locks集合里的state表示均衡器是否找正在運行,0表示非活動狀態,2表示正在均衡。均衡遷移數據的過程會增加系統的負載:目標分片必須查詢源分片的所有文檔,將文檔插入目標分片中,再清除源分片的數據。可以關閉均衡器(不建議):關閉會導致各分片數據分布不均衡,磁盤空間得不到有效的利用。
以上步驟中實驗了users集合和wsy集合進行數據分片,可看出users集合的數據已經進過經過balancer,進行數據分配達到了均衡,都是7,而wsy集合還沒有達到均衡。過段時間可查看效果。(可發現都已經是3,分配均衡)
{ "_id" : "wsy", "primary" : "shard1", "partitioned" : true } wsy.wsy shard key: { "user_id" : 1 } unique: false balancing: true chunks: shard1 3 shard2 3 shard3 3 { "user_id" : { "$minKey" : 1 } } -->> { "user_id" : 2 } on : shard2 Timestamp(5, 1) { "user_id" : 2 } -->> { "user_id" : 16914 } on : shard3 Timestamp(6, 1) { "user_id" : 16914 } -->> { "user_id" : 25370 } on : shard1 Timestamp(4, 1) { "user_id" : 25370 } -->> { "user_id" : 38126 } on : shard1 Timestamp(3, 3) { "user_id" : 38126 } -->> { "user_id" : 46582 } on : shard2 Timestamp(4, 2) { "user_id" : 46582 } -->> { "user_id" : 56101 } on : shard2 Timestamp(4, 3) { "user_id" : 56101 } -->> { "user_id" : 64557 } on : shard3 Timestamp(5, 2) { "user_id" : 64557 } -->> { "user_id" : 74077 } on : shard3 Timestamp(5, 3) { "user_id" : 74077 } -->> { "user_id" : { "$maxKey" : 1 } } on : shard1 Timestamp(6, 0)
查看mongo集群是否開啟了 balance 狀態
mongos> sh.getBalancerState() true
也可以通過在路由節點mongos上執行sh.status() 查看balance狀態。如果balance開啟,查看是否正在有數據的遷移
mongos> sh.isBalancerRunning() false
如果未開啟,執行命令
sh.setBalancerState( true )
10、啟動關閉
mongodb的啟動順序是,先啟動配置服務器,在啟動分片,最后啟動mongos.
mongod -f /usr/local/mongodb/conf/config.conf mongod -f /usr/local/mongodb/conf/shard1.conf mongod -f /usr/local/mongodb/conf/shard2.conf mongod -f /usr/local/mongodb/conf/shard3.conf mongod -f /usr/local/mongodb/conf/mongos.conf
關閉時,直接killall殺掉所有進程
killall mongod
killall mongos
或者 ps -ef|grep mongo|grep -v grep|awk '{print $2}'|xargs kill -9
11、若需添加用戶驗證功能,則需要在每個配置文件中指定keyfile文件。
先創建對應庫的用戶及相應權限
關閉實例,創建keyfile文件,修改配置文件
重新啟動實例,驗證
創建用戶
db.createUser( ... { ... user: "root", ... pwd: "root", ... roles: [ { role: "root", db: "admin" } ] ... } ... );
Successfully added user: {
"user" : "root",
"roles" : [
{
"role" : "root",
"db" : "admin"
}
]
}
db.createUser({ ... user: "admin", ... pwd: "admin", ... roles: [ { role: "userAdminAnyDatabase", db: "admin" } ] ... });
Successfully added user: {
"user" : "admin",
"roles" : [
{
"role" : "userAdminAnyDatabase",
"db" : "admin"
}
]
}
創建keyfile文件:
openssl rand -base64 666 > /root/keyfile 其中666是文件大小 /root/keyfile : 文件存放路徑 ##該key的權限必須是600;也可改為400 chmod 600 /root/keyfile
添加以下配置(每個實例的配置文件中):
security: authorization: enabled #若啟動實例報錯,可刪除此行 keyFile: /root/keyfile clusterAuthMode: keyFile
啟動實例:
[root@localhost ~]# mongod -f /etc/mongodb/config.conf about to fork child process, waiting until server is ready for connections. forked process: 11662 child process started successfully, parent exiting [root@localhost ~]# mongod -f /etc/mongodb/shard1.conf about to fork child process, waiting until server is ready for connections. forked process: 11748 child process started successfully, parent exiting [root@localhost ~]# mongod -f /etc/mongodb/shard2.conf about to fork child process, waiting until server is ready for connections. forked process: 11837 child process started successfully, parent exiting [root@localhost ~]# mongod -f /etc/mongodb/shard3.conf about to fork child process, waiting until server is ready for connections. forked process: 11927 child process started successfully, parent exiting [root@localhost ~]# vim /etc/mongodb/mongos.conf [root@localhost ~]# mongos -f /etc/mongodb/mongos.conf about to fork child process, waiting until server is ready for connections. forked process: 12023 child process started successfully, parent exiting [root@localhost ~]# ps -ef |grep mongo root 11662 1 1 21:58 ? 00:00:03 mongod -f /etc/mongodb/config.conf root 11748 1 1 21:58 ? 00:00:03 mongod -f /etc/mongodb/shard1.conf root 11837 1 2 21:58 ? 00:00:04 mongod -f /etc/mongodb/shard2.conf root 11927 1 1 21:59 ? 00:00:02 mongod -f /etc/mongodb/shard3.conf root 12023 1 1 22:01 ? 00:00:00 mongos -f /etc/mongodb/mongos.conf root 12056 10913 0 22:01 pts/0 00:00:00 grep mongo
#只有再啟動mongos路由分片是會報錯,其他都正常,可刪除authorization: enabled配置即可
[root@localhost ~]# mongos -f /etc/mongodb/mongos.conf Unrecognized option: security.authorization try 'mongos --help' for more information
登陸驗證
[root@localhost ~]# mongo 192.168.214.214:20000 MongoDB shell version v3.6.6 connecting to: mongodb://192.168.214.214:20000/test MongoDB server version: 3.6.6 mongos> show dbs 2019-04-25T13:50:32.370+0800 E QUERY [thread1] Error: listDatabases failed:{ "ok" : 0, "errmsg" : "not authorized on admin to execute command { listDatabases: 1.0, $clusterTime: { clusterTime: Timestamp(1556171350, 1), signature: { hash: BinData(0, 3BB007B2D5254E4F53B248B18D9020A666CFEC91), keyId: 6680724571257045010 } }, $db: \"admin\" }", "code" : 13, "codeName" : "Unauthorized", "$clusterTime" : { "clusterTime" : Timestamp(1556171425, 2), "signature" : { "hash" : BinData(0,"p35bBX0K45G/nrC4Q0Q288vpBmM="), "keyId" : NumberLong("6680724571257045010") } }, "operationTime" : Timestamp(1556171425, 2) } : _getErrorWithCode@src/mongo/shell/utils.js:25:13 Mongo.prototype.getDBs@src/mongo/shell/mongo.js:65:1 shellHelper.show@src/mongo/shell/utils.js:849:19 shellHelper@src/mongo/shell/utils.js:739:15 @(shellhelp2):1:1 mongos> use admin switched to db admin mongos> db.auth('root','root') 1 mongos> show dbs admin 0.000GB config 0.002GB test 0.014GB wsy 0.005GB
至此,mongodb分片集群加用戶驗證都已完成。
參考文獻
https://www.cnblogs.com/liqing1009/p/8611954.html
https://www.cnblogs.com/clsn/p/8214345.html#auto_id_28
http://www.cnblogs.com/zhoujinyi/p/4635444.html