配置systemctl(zookeeper、hadoop、redis)


==背景==

給公司搭建了一個建議的大數據平台,組件包括zookeeper、hadoop、flink、influxdb、redis、mysql、rocketmq

因為zookeeper、hadoop是平台的基礎組件,打算配置systemctl守護進程,用來掛掉自動重啟,以減少運維負擔。

 

==規划==

1、每台機器設置service文件的管理文件夾:/home/radmin/rexel-dev

2、文件組成,為每個service生成兩個文件:xxx.service及xxx.service.run。.run是.service需要執行的腳本,負責環境依賴

3、軟連接:在systemctl目錄下建立軟連接,連接到rexel-dev下的具體文件

4、所有.run文件賦予777權限。

 

==准備==

1、創建pid文件存放路徑

for i in {1..3}; do ssh vm$i mkdir -p /home/radmin/data/pids/hadoop; done

 

2、創建service文件存放路徑

for i in {1..3}; do ssh vm$i mkdir -p /home/radmin/rexel-dev; done

 

3、配置系統環境變量

export HADOOP_PID_DIR=/home/radmin/data/pids/hadoop
export YARN_PID_DIR=/home/radmin/data/pids/hadoop

 

==zookeeper==

1、文件規划

rexel-zookeeper.service
rexel-zookeeper.service.run

 

2、配置run文件

文件:rexel-zookeeper.service.run

#!/bin/bash
source /etc/profile
/home/radmin/zookeeper-3.5.6/bin/zkServer.sh start

 

3、配置service文件

文件:rexel-zookeeper.service

[Unit]
Description=Zookeeper Service
After=network.target
ConditionPathExists=/home/radmin/zookeeper-3.5.6/conf/zoo.cfg

[Service]
Type=forking
PIDFile=/home/radmin/data/zk/dataDir/zookeeper_server.pid
ExecStart=/home/radmin/rexel-dev/rexel-zookeeper.service.run
Restart=always

[Install]
WantedBy=multi-user.target

 

4、配置軟連接

命令:

ln -s /home/radmin/rexel-dev/rexel-zookeeper.service /usr/lib/systemd/system/rexel-zookeeper.service

 

5、加載生效

命令:

systemctl daemon-reload

 

6、啟動服務

命令:

systemctl status rexel-zookeeper.service

 

【小貼士】

任何增加、修改或刪除systemctl,都需要執行systemctl daemon-reload命令,使修改生效。

 

【小插曲】

service啟動失敗,通過journalctl -xe命令查看日志,發現提示

Can't open PID file /home/radmin/data/pid/zookeeper.pid (yet?) after start: No such file or directory

發現原因是配置的pid文件路徑不正確,將pid文件的路徑和zoo.cfg中的dataDir路徑一致,重新啟動service之后,恢復正常。

 

==hadoop==

1、文件規划

rexel-hdfs-journalnode.service
rexel-hdfs-journalnode.service.run
rexel-hdfs-namenode.service
rexel-hdfs-namenode.service.run
rexel-hdfs-zkfc.service
rexel-hdfs-zkfc.service.run
rexel-hdfs-datanode.service
rexel-hdfs-datanode.service.run
rexel-yarn-nodemanager.service
rexel-yarn-nodemanager.service.run
rexel-yarn-resourcemanager.service
rexel-yarn-resourcemanager.service.run

 

2、配置run文件

文件1:rexel-hdfs-journalnode.service.run

#!/bin/bash
source /etc/profile
/home/radmin/hadoop-2.8.3/sbin/hadoop-daemon.sh start journalnode

 

文件2:rexel-hdfs-namenode.service.run

#!/bin/bash
source /etc/profile
/home/radmin/hadoop-2.8.3/sbin/hadoop-daemon.sh start namenode

 

文件3:rexel-hdfs-zkfc.service.run

#!/bin/bash
source /etc/profile
/home/radmin/hadoop-2.8.3/sbin/hadoop-daemon.sh start zkfc

 

文件4:rexel-hdfs-datanode.service.run

#!/bin/bash
source /etc/profile
/home/radmin/hadoop-2.8.3/sbin/hadoop-daemon.sh start datanode

 

文件5:rexel-yarn-resourcemanager.service.run

#!/bin/bash
source /etc/profile
/home/radmin/hadoop-2.8.3/sbin/yarn-daemon.sh start resourcemanager

 

文件6:rexel-yarn-nodemanager.service.run

#!/bin/bash
source /etc/profile
/home/radmin/hadoop-2.8.3/sbin/yarn-daemon.sh start nodemanager

 

3、配置service文件

文件1:rexel-hdfs-journalnode.service

[Unit]
Description=HDFS Journalnode Service
After=network.target

[Service]
Type=forking
PIDFile=/home/radmin/data/pids/hadoop/hadoop-root-journalnode.pid
ExecStart=/home/radmin/rexel-dev/rexel-hdfs-journalnode.service.run
Restart=always

[Install]
WantedBy=multi-user.target

 

文件2:rexel-hdfs-namenode.service

[Unit]
Description=HDFS Namenode Service
After=network.target

[Service]
Type=forking
PIDFile=/home/radmin/data/pids/hadoop/hadoop-root-namenode.pid
ExecStart=/home/radmin/rexel-dev/rexel-hdfs-namenode.service.run
Restart=always

[Install]
WantedBy=multi-user.target

 

文件3:rexel-hdfs-datanode.service

[Unit]
Description=HDFS Datanode Service
After=network.target

[Service]
Type=forking
PIDFile=/home/radmin/data/pids/hadoop/hadoop-root-datanode.pid
ExecStart=/home/radmin/rexel-dev/rexel-hdfs-datanode.service.run
Restart=always

[Install]
WantedBy=multi-user.target

 

文件4:rexel-hdfs-zkfc.service

[Unit]
Description=HDFS ZKFC Service
After=network.target

[Service]
Type=forking
PIDFile=/home/radmin/data/pids/hadoop/hadoop-root-zkfc.pid
ExecStart=/home/radmin/rexel-dev/rexel-hdfs-zkfc.service.run
Restart=always

[Install]
WantedBy=multi-user.target

 

文件5:rexel-yarn-resourcemanager.service

[Unit]
Description=Yarn Resourcemanager Service
After=network.target

[Service]
Type=forking
PIDFile=/home/radmin/data/pids/hadoop/yarn-root-resourcemanager.pid
ExecStart=/home/radmin/rexel-dev/rexel-yarn-resourcemanager.service.run
Restart=always

[Install]
WantedBy=multi-user.target

 

文件6:rexel-yarn-nodemanager.service

[Unit]
Description=Yarn Nodemanager Service
After=network.target

[Service]
Type=forking
PIDFile=/home/radmin/data/pids/hadoop/yarn-root-nodemanager.pid
ExecStart=/home/radmin/rexel-dev/rexel-yarn-nodemanager.service.run
Restart=always

[Install]
WantedBy=multi-user.target

 

4、配置軟連接

命令:

ln -s /home/radmin/rexel-dev/rexel-hdfs-journalnode.service /usr/lib/systemd/system/rexel-hdfs-journalnode.service
ln -s /home/radmin/rexel-dev/rexel-hdfs-namenode.service /usr/lib/systemd/system/rexel-hdfs-namenode.service
ln -s /home/radmin/rexel-dev/rexel-hdfs-zkfc.service /usr/lib/systemd/system/rexel-hdfs-zkfc.service
ln -s /home/radmin/rexel-dev/rexel-hdfs-datanode.service /usr/lib/systemd/system/rexel-hdfs-datanode.service
ln -s /home/radmin/rexel-dev/rexel-yarn-resourcemanager.service /usr/lib/systemd/system/rexel-yarn-resourcemanager.service
ln -s /home/radmin/rexel-dev/rexel-yarn-nodemanager.service /usr/lib/systemd/system/rexel-yarn-nodemanager.service

 

5、加載生效

命令:

systemctl daemon-reload

 

6、啟動服務

命令:

systemctl start rexel-hdfs-journalnode.service
systemctl start rexel-hdfs-namenode.service
systemctl start rexel-hdfs-zkfc.service
systemctl start rexel-hdfs-datanode.service
systemctl start rexel-yarn-resourcemanager.service
systemctl start rexel-yarn-nodemanager.service

 

【小插曲】

和zookeeper一樣,提示了找不到pid文件的錯誤。

解決辦法:在系統環境變量中配置HADOOP_PID_DIR,YARN_PID_DIR然后修改一下service文件中pid文件的路徑,重新啟動服務即可。

 

==Redis==

1、文件規划

rexel-redis.service
rexel-redis.service.run

 

2、配置run文件

文件:rexel-redis.service.run

#!/bin/bash
source /etc/profile
/home/radmin/redis-5.0.8/src/redis-server /home/radmin/redis-5.0.8/redis.conf

 

3、配置service文件

文件:rexel-redis.service

[Unit]
Description=Redis Service
After=network.target

[Service]
Type=forking
PIDFile=/home/radmin/data/pids/redis_8793.pid
ExecStart=/home/radmin/rexel-dev/rexel-redis.service.run
Restart=always

[Install]
WantedBy=multi-user.target

 

【小貼士】

redis設置了默認的pid文件路徑:/var/run/redis_6379.pid,為了方便管理,我重新定義了pidfile的配置。

相應的在rexel-redis.service文件中的

 

4、配置軟連接

命令:

ln -s /home/radmin/rexel-dev/rexel-redis.service /usr/lib/systemd/system/rexel-redis.service

 

5、加載生效

命令:

systemctl daemon-reload

 

6、啟動服務

命令:

systemctl start rexel-redis.service

 

--END--

 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2026 CODEPRJ.COM