TDengine集群搭建

1.准備工作

1.1集群規划

節點	操作系統版本	TDengine版本	HostName	IP	上傳目錄	安裝目錄
td1	Centos7	2.0.20.12	td1.test.com	192.168.137.101	/opt/software	/opt/programFiles
td2			td2.test.com	192.168.137.102
td3			td3.test.com	192.168.137.103

1.2 修改主機名和hosts文件

為什么要修改主機名和hosts文件？

為后面配置文件中的FQDN參數做准備。FQDN(fully qualified domain name,完全限定域名)是internet上特定計算機或主機的完整域名。

集群的每個節點是由End Point來唯一標識的，End Point是由FQDN外加Port組成，比如 h1.taosdata.com:6030。配置FQDN是為了標識集群中的每個節點。

在集群中通常不會使用IP，而會使用主機名，是因為通常IP容易發生變化，而主機名不會。這樣即使IP發生變化，也不會影響集群的使用。

(1)修改主機名

centos7修改主機名(不同的操作系統，命令有細微的差別)，命令如下(分別在3台主機上執行)：

hostnamectl set-hostname td1.test.com

hostnamectl set-hostname td2.test.com

hostnamectl set-hostname td3.test.com

查看是否修改成功命令如下：

hostname -f #（注意：hostname需要比localhost字符長度更長，否則hostname –f 查詢結果為localhost,將影響數據庫使用）

(2)修改hosts文件

修改hosts文件命令：

vim /etc/hosts

添加如下信息：

192.168.137.101 td1.test.com

192.168.137.102 td2.test.com

192.168.137.103 td3.test.com

 1.3關閉防火牆

 為什么要關閉防火牆？

 數據庫使用到了如下端口，如果端口未打開，則會影響數據庫使用。所以可以選擇兩種方式的任意一種：(1)打開以下端口；(2)關閉防火牆

注意：如果數據庫部署在雲服務器(阿里雲，華為雲)需要在安全設置中入站規則，將上面端口添加進去，否則會影響數據庫遠程訪問。

關閉防火牆命令如下：

systemctl stop firewalld  # 臨時生效

systemctl status firewalld  # 查看狀態

systemctl disable firewalld  # 永久生效

 1.4時間同步

為什么要進行集群節點時間同步？

不同計算設備之間控制、計算、處理、應用等數據或操作都具有時序性，若計算機時間不同步，這些應用或操作或將無法正常進行。如果時間不一致，集群安裝會報錯。

(1)簡單時間同步

date -s "2021-08-19 10:32:10" #具體的yyyy-MM-dd hh:mm:ss 不推薦使用

(2)同步服務器時間

ntpdate cn.pool.ntp.org #同步時間服務器

如果沒有安裝ntpdate工具，安裝命令是：

yum -y install ntp ntpdate

(3)設置時間服務器

可以參考網址：Linux配置ntp時間服務器（全）https://www.cnblogs.com/quchunhui/p/7658853.html

2.軟件安裝

2.1下載包

從下載網址：https://www.taosdata.com/cn/all-downloads/，下載包【TDengine-server-2.0.20.12-Linux-x64.tar.gz】

2.2安裝包

(1)解壓

cd /opt/software
tar zvxf TDengine-server-2.0.20.12-Linux-x64.tar.gz -C /opt/programFiles

(2)安裝

cd /opt/programFiles/TDengine-server-2.0.20.12
./install.sh 

(注意：第一個節點需要回車，第二個節點和第三個節點需要設置 td1.test.com:6030)

提示如何編輯配置文件；如何啟動服務；如何連接數據庫；如何創建節點

 2.3配置

cd /usr/local/taos/cfg
vim taos.cfg

修改td1配置文件taos.cfg如下（td2和td3只需修改對應的fqdn即可，標紅的部分都需要修改）：

########################################################
#                                                      #
#                  TDengine Configuration              #
#   Any questions, please email [e-mail]   #
#                                                      #
########################################################

# first fully qualified domain name (FQDN) for TDengine system
 firstEp                   td1.test.com:6030
 secondEq                  td2.test.com
# local fully qualified domain name (FQDN) 如果是td2，則fqdn配置成td2.test.com;如果是td3，則fqdn配置成td3.test.com
 fqdn                      td1.test.com

# first port number for the connection (12 continuous UDP/TCP port number are used) 
# serverPort                6030

# log file's directory
# logDir                    /var/log/taos

# data file's directory
# dataDir                   /var/lib/taos

# temporary file's directory
# tempDir                   /tmp/

# the arbitrator's fully qualified domain name (FQDN) for TDengine system, for cluster only   
# arbitrator                arbitrator_hostname:6042     

# number of threads per CPU core
# numOfThreadsPerCore       1.0

# number of threads to commit cache data
# numOfCommitThreads        4

# the proportion of total CPU cores available for query processing
# 2.0: the query threads will be set to double of the CPU cores.
# 1.0: all CPU cores are available for query processing [default].
# 0.5: only half of the CPU cores are available for query.
# 0.0: only one core available.
# ratioOfQueryCores        1.0

# the last_row/first/last aggregator will not change the original column name in the result fields
# keepColumnName            0

# number of management nodes in the system
 numOfMnodes               3

# enable/disable backuping vnode directory when removing vnode
# vnodeBak                  1

# enable/disable installation / usage report
# telemetryReporting        1

# enable/disable load balancing
# balance                   1

# role for dnode. 0 - any, 1 - mnode, 2 - dnode
# role                      0

# max timer control blocks
# maxTmrCtrl                512

# time interval of system monitor, seconds 
# monitorInterval           30

# number of seconds allowed for a dnode to be offline, for cluster only 
# offlineThreshold          864000

# RPC re-try timer, millisecond
# rpcTimer                  300

# RPC maximum time for ack, seconds. 
# rpcMaxTime                600

# time interval of dnode status reporting to mnode, seconds, for cluster only 
# statusInterval            1

# time interval of heart beat from shell to dnode, seconds
# shellActivityTimer        3

# minimum sliding window time, milli-second
# minSlidingTime            10

# minimum time window, milli-second
# minIntervalTime           10

# maximum delay before launching a stream computation, milli-second
# maxStreamCompDelay        20000

# maximum delay before launching a stream computation for the first time, milli-second
# maxFirstStreamCompDelay   10000

# retry delay when a stream computation fails, milli-second
# retryStreamCompDelay      10

# the delayed time for launching a stream computation, from 0.1(default, 10% of whole computing time window) to 0.9
# streamCompDelayRatio      0.1

# max number of vgroups per db, 0 means configured automatically
# maxVgroupsPerDb           0

# max number of tables per vnode
# maxTablesPerVnode         1000000

# cache block size (Mbyte)
# cache                     16

# number of cache blocks per vnode
# blocks                    6

# number of days per DB file
# days                  10

# number of days to keep DB file
# keep                  3650

# minimum rows of records in file block
# minRows               100

# maximum rows of records in file block
# maxRows               4096

# the number of acknowledgments required for successful data writing 至少需要寫入2個副本,才能成功
 quorum                2     

# enable/disable compression
# comp                  2

# write ahead log (WAL) level, 0: no wal; 1: write wal, but no fysnc; 2: write wal, and call fsync
# walLevel              1

# if walLevel is set to 2, the cycle of fsync being executed, if set to 0, fsync is called right away
# fsync                 3000

# number of replications, for cluster only 3個副本
 replica               3

# the compressed rpc message, option:
#  -1 (no compression)
#   0 (all message compressed),
# > 0 (rpc message body which larger than this value will be compressed)
# compressMsgSize       -1

# max length of an SQL
# maxSQLLength          65480

# the maximum number of records allowed for super table time sorting
# maxNumOfOrderedRes    100000

# system time zone
# timezone              Asia/Shanghai (CST, +0800)

# system locale
# locale                en_US.UTF-8

# default system charset
# charset               UTF-8

# max number of connections allowed in dnode
# maxShellConns         5000

# max number of connections allowed in client
# maxConnections        5000

# stop writing logs when the disk size of the log folder is less than this value
# minimalLogDirGB       0.1

# stop writing temporary files when the disk size of the tmp folder is less than this value
# minimalTmpDirGB       0.1

# if disk free space is less than this value, taosd service exit directly within startup process
# minimalDataDirGB      0.1

# One mnode is equal to the number of vnode consumed
# mnodeEqualVnodeNum    4

# enbale/disable http service
# http                  1

# enable/disable system monitor 
# monitor               1

# enable/disable recording the SQL statements via restful interface
# httpEnableRecordSql   0

# number of threads used to process http requests
# httpMaxThreads        2

# maximum number of rows returned by the restful interface
# restfulRowLimit       10240

# The following parameter is used to limit the maximum number of lines in log files.
# max number of lines per log filters
# numOfLogLines         10000000

# enable/disable async log
# asyncLog              1

# time of keeping log files, days
# logKeepDays           0


# The following parameters are used for debug purpose only.
# debugFlag 8 bits mask: FILE-SCREEN-UNUSED-HeartBeat-DUMP-TRACE_WARN-ERROR
# 131: output warning and error 
# 135: output debug, warning and error
# 143: output trace, debug, warning and error to log
# 199: output debug, warning and error to both screen and file
# 207: output trace, debug, warning and error to both screen and file

# debug flag for all log type, take effect when non-zero value
# debugFlag             0

# debug flag for meta management messages
# mDebugFlag            135

# debug flag for dnode messages
# dDebugFlag            135

# debug flag for sync module
# sDebugFlag            135

# debug flag for WAL
# wDebugFlag            135

# debug flag for SDB
# sdbDebugFlag          135

# debug flag for RPC 
# rpcDebugFlag          131

# debug flag for TAOS TIMER
# tmrDebugFlag          131

# debug flag for TDengine client 
# cDebugFlag            131

# debug flag for JNI
# jniDebugFlag          131

# debug flag for storage
# uDebugFlag            131

# debug flag for http server
# httpDebugFlag         131

# debug flag for monitor
# monDebugFlag          131

# debug flag for query
# qDebugFlag            131

# debug flag for vnode
# vDebugFlag            131

# debug flag for TSDB
# tsdbDebugFlag         131

# debug flag for continue query
# cqDebugFlag           131

# enable/disable recording the SQL in taos client
# enableRecordSql    0

# generate core file when service crash
# enableCoreFile        1

# maximum display width of binary and nchar fields in the shell. The parts exceeding this limit will be hidden
# maxBinaryDisplayWidth 30

# enable/disable stream (continuous query)
# stream                1

# in retrieve blocking model, only in 50% query threads will be used in query processing in dnode
# retrieveBlockingModel    0

# the maximum allowed query buffer size in MB during query processing for each data node
# -1 no limit (default)
# 0  no query allowed, queries are disabled
# queryBufferSize         -1

2.4.啟動

啟動3個節點

sudo systemctl start taosd  # 啟動服務

sudo systemctl status taosd  # 查看狀態

taos # 進入數據庫

2.5創建dnode節點

在td1或td2或td3上創建dnode節點，只需要在一個節點上創建即可。

CREATE DNODE "td2.test.com:6030";
CREATE DNODE "td3.test.com:6030";

3.測試驗證

3.1查詢dnode和mnode

(1)查看dnode

show dnodes;

(2)查看mnode

show mnodes;

3.2執行taosdemo案例，命令查看

(1)執行taosdemo

taosdemo

(2)在三個節點查看數據

taos

use test;
select count(*) from meters;

3.3其中一台宕機后，查看dnode和mnode

(0)停止td1

systemctl stop taosd

systemctl status taosd

(1)查看dnode

show dnodes;

(2)查看mnode

show mnodes;

3.4其中一台宕機后，命令查看

查看數據

use test;

select count(*) from meters;

參考網址：

TDengine集群安裝、管理 https://www.taosdata.com/cn/documentation/cluster

使用 TDengine 進行報警監測 https://www.taosdata.com/blog/2020/04/14/1438.html

常見問題 https://www.taosdata.com/cn/documentation/faq#port

一篇文章說清楚TDengine的FQDN  https://www.cnblogs.com/taosdata/p/13690374.html

Linux配置ntp時間服務器（全）https://www.cnblogs.com/quchunhui/p/7658853.html