==背景==
給公司搭建了一個建議的大數據平台,組件包括zookeeper、hadoop、flink、influxdb、redis、mysql、rocketmq
因為zookeeper、hadoop是平台的基礎組件,打算配置systemctl守護進程,用來掛掉自動重啟,以減少運維負擔。
==規划==
1、每台機器設置service文件的管理文件夾:/home/radmin/rexel-dev
2、文件組成,為每個service生成兩個文件:xxx.service及xxx.service.run。.run是.service需要執行的腳本,負責環境依賴
3、軟連接:在systemctl目錄下建立軟連接,連接到rexel-dev下的具體文件
4、所有.run文件賦予777權限。
==准備==
1、創建pid文件存放路徑
for i in {1..3}; do ssh vm$i mkdir -p /home/radmin/data/pids/hadoop; done
2、創建service文件存放路徑
for i in {1..3}; do ssh vm$i mkdir -p /home/radmin/rexel-dev; done
3、配置系統環境變量
export HADOOP_PID_DIR=/home/radmin/data/pids/hadoop
export YARN_PID_DIR=/home/radmin/data/pids/hadoop
==zookeeper==
1、文件規划
rexel-zookeeper.service
rexel-zookeeper.service.run
2、配置run文件
文件:rexel-zookeeper.service.run
#!/bin/bash source /etc/profile /home/radmin/zookeeper-3.5.6/bin/zkServer.sh start
3、配置service文件
文件:rexel-zookeeper.service
[Unit] Description=Zookeeper Service After=network.target ConditionPathExists=/home/radmin/zookeeper-3.5.6/conf/zoo.cfg [Service] Type=forking PIDFile=/home/radmin/data/zk/dataDir/zookeeper_server.pid ExecStart=/home/radmin/rexel-dev/rexel-zookeeper.service.run Restart=always [Install] WantedBy=multi-user.target
4、配置軟連接
命令:
ln -s /home/radmin/rexel-dev/rexel-zookeeper.service /usr/lib/systemd/system/rexel-zookeeper.service
5、加載生效
命令:
systemctl daemon-reload
6、啟動服務
命令:
systemctl status rexel-zookeeper.service
【小貼士】
任何增加、修改或刪除systemctl,都需要執行systemctl daemon-reload命令,使修改生效。
【小插曲】
service啟動失敗,通過journalctl -xe命令查看日志,發現提示
Can't open PID file /home/radmin/data/pid/zookeeper.pid (yet?) after start: No such file or directory
發現原因是配置的pid文件路徑不正確,將pid文件的路徑和zoo.cfg中的dataDir路徑一致,重新啟動service之后,恢復正常。
==hadoop==
1、文件規划
rexel-hdfs-journalnode.service rexel-hdfs-journalnode.service.run rexel-hdfs-namenode.service rexel-hdfs-namenode.service.run rexel-hdfs-zkfc.service rexel-hdfs-zkfc.service.run rexel-hdfs-datanode.service rexel-hdfs-datanode.service.run rexel-yarn-nodemanager.service rexel-yarn-nodemanager.service.run rexel-yarn-resourcemanager.service rexel-yarn-resourcemanager.service.run
2、配置run文件
文件1:rexel-hdfs-journalnode.service.run
#!/bin/bash source /etc/profile /home/radmin/hadoop-2.8.3/sbin/hadoop-daemon.sh start journalnode
文件2:rexel-hdfs-namenode.service.run
#!/bin/bash source /etc/profile /home/radmin/hadoop-2.8.3/sbin/hadoop-daemon.sh start namenode
文件3:rexel-hdfs-zkfc.service.run
#!/bin/bash source /etc/profile /home/radmin/hadoop-2.8.3/sbin/hadoop-daemon.sh start zkfc
文件4:rexel-hdfs-datanode.service.run
#!/bin/bash source /etc/profile /home/radmin/hadoop-2.8.3/sbin/hadoop-daemon.sh start datanode
文件5:rexel-yarn-resourcemanager.service.run
#!/bin/bash source /etc/profile /home/radmin/hadoop-2.8.3/sbin/yarn-daemon.sh start resourcemanager
文件6:rexel-yarn-nodemanager.service.run
#!/bin/bash source /etc/profile /home/radmin/hadoop-2.8.3/sbin/yarn-daemon.sh start nodemanager
3、配置service文件
文件1:rexel-hdfs-journalnode.service
[Unit] Description=HDFS Journalnode Service After=network.target [Service] Type=forking PIDFile=/home/radmin/data/pids/hadoop/hadoop-root-journalnode.pid ExecStart=/home/radmin/rexel-dev/rexel-hdfs-journalnode.service.run Restart=always [Install] WantedBy=multi-user.target
文件2:rexel-hdfs-namenode.service
[Unit] Description=HDFS Namenode Service After=network.target [Service] Type=forking PIDFile=/home/radmin/data/pids/hadoop/hadoop-root-namenode.pid ExecStart=/home/radmin/rexel-dev/rexel-hdfs-namenode.service.run Restart=always [Install] WantedBy=multi-user.target
文件3:rexel-hdfs-datanode.service
[Unit] Description=HDFS Datanode Service After=network.target [Service] Type=forking PIDFile=/home/radmin/data/pids/hadoop/hadoop-root-datanode.pid ExecStart=/home/radmin/rexel-dev/rexel-hdfs-datanode.service.run Restart=always [Install] WantedBy=multi-user.target
文件4:rexel-hdfs-zkfc.service
[Unit] Description=HDFS ZKFC Service After=network.target [Service] Type=forking PIDFile=/home/radmin/data/pids/hadoop/hadoop-root-zkfc.pid ExecStart=/home/radmin/rexel-dev/rexel-hdfs-zkfc.service.run Restart=always [Install] WantedBy=multi-user.target
文件5:rexel-yarn-resourcemanager.service
[Unit] Description=Yarn Resourcemanager Service After=network.target [Service] Type=forking PIDFile=/home/radmin/data/pids/hadoop/yarn-root-resourcemanager.pid ExecStart=/home/radmin/rexel-dev/rexel-yarn-resourcemanager.service.run Restart=always [Install] WantedBy=multi-user.target
文件6:rexel-yarn-nodemanager.service
[Unit] Description=Yarn Nodemanager Service After=network.target [Service] Type=forking PIDFile=/home/radmin/data/pids/hadoop/yarn-root-nodemanager.pid ExecStart=/home/radmin/rexel-dev/rexel-yarn-nodemanager.service.run Restart=always [Install] WantedBy=multi-user.target
4、配置軟連接
命令:
ln -s /home/radmin/rexel-dev/rexel-hdfs-journalnode.service /usr/lib/systemd/system/rexel-hdfs-journalnode.service ln -s /home/radmin/rexel-dev/rexel-hdfs-namenode.service /usr/lib/systemd/system/rexel-hdfs-namenode.service ln -s /home/radmin/rexel-dev/rexel-hdfs-zkfc.service /usr/lib/systemd/system/rexel-hdfs-zkfc.service ln -s /home/radmin/rexel-dev/rexel-hdfs-datanode.service /usr/lib/systemd/system/rexel-hdfs-datanode.service ln -s /home/radmin/rexel-dev/rexel-yarn-resourcemanager.service /usr/lib/systemd/system/rexel-yarn-resourcemanager.service ln -s /home/radmin/rexel-dev/rexel-yarn-nodemanager.service /usr/lib/systemd/system/rexel-yarn-nodemanager.service
5、加載生效
命令:
systemctl daemon-reload
6、啟動服務
命令:
systemctl start rexel-hdfs-journalnode.service systemctl start rexel-hdfs-namenode.service systemctl start rexel-hdfs-zkfc.service systemctl start rexel-hdfs-datanode.service systemctl start rexel-yarn-resourcemanager.service systemctl start rexel-yarn-nodemanager.service
【小插曲】
和zookeeper一樣,提示了找不到pid文件的錯誤。
解決辦法:在系統環境變量中配置HADOOP_PID_DIR,YARN_PID_DIR然后修改一下service文件中pid文件的路徑,重新啟動服務即可。

==Redis==
1、文件規划
rexel-redis.service
rexel-redis.service.run
2、配置run文件
文件:rexel-redis.service.run
#!/bin/bash source /etc/profile /home/radmin/redis-5.0.8/src/redis-server /home/radmin/redis-5.0.8/redis.conf
3、配置service文件
文件:rexel-redis.service
[Unit] Description=Redis Service After=network.target [Service] Type=forking PIDFile=/home/radmin/data/pids/redis_8793.pid ExecStart=/home/radmin/rexel-dev/rexel-redis.service.run Restart=always [Install] WantedBy=multi-user.target
【小貼士】
redis設置了默認的pid文件路徑:/var/run/redis_6379.pid,為了方便管理,我重新定義了pidfile的配置。
相應的在rexel-redis.service文件中的

4、配置軟連接
命令:
ln -s /home/radmin/rexel-dev/rexel-redis.service /usr/lib/systemd/system/rexel-redis.service
5、加載生效
命令:
systemctl daemon-reload
6、啟動服務
命令:
systemctl start rexel-redis.service
--END--
