死磕salt系列-salt 故障匯總

這里將salt使用過程中遇到的所有的故障進行一個匯總。

grains 匹配后多了一個列表

salt-master中配置jinja模板來匹配自定義的grins。

vim /etc/salt/minion
grains:
  roles:
    - elastic-minion1
#配置文件中引用
network.host: {{ grains['roles']}}
#匹配后的結果如下：
network.host: ['elastic-minion1']

解決辦法

#修改配置文件中的引用
network.host: {{ grains['roles'][0] }}
重新執行salt同步的命令：
結果如下：
network.host: elastic-minion1

報錯一：

----------
          ID: limit-conf-config
    Function: file.managed
        Name: /etc/securilty/limits.conf
      Result: False
     Comment: Parent directory not present
     Started: 17:05:44.606040
    Duration: 11.448 ms
     Changes:   

解決辦法：

出現這個的原因是Minion上沒有name參數定義的目錄. 在minion上定義對應的目錄就可以了

報錯：salt 管理配置文件時，報錯。

[root@master files]# salt '*' state.highstate 
agent.niu.com:
    Data failed to compile:
----------
    Rendering SLS init.limit failed, render error: expected '<document start>', but found '<block mapping start>'
  in "<unicode string>", line 2, column 1:
    limit-conf-config:
    ^
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/salt/state.py", line 2509, in render_state
    sls, rendered_sls=mods
  File "/usr/lib/python2.7/site-packages/salt/template.py", line 79, in compile_template
    ret = render(input_data, saltenv, sls, **render_kwargs)
▽ File "/usr/lib/python2.7/site-packages/salt/renderers/yaml.py", line 50, in render
    data = load(yaml_data, Loader=get_yaml_loader(argline))
  File "/usr/lib64/python2.7/site-packages/yaml/__init__.py", line 71, in load
    return loader.get_single_data()
  File "/usr/lib64/python2.7/site-packages/yaml/constructor.py", line 37, in get_single_data
    node = self.get_single_node()
  File "/usr/lib64/python2.7/site-packages/yaml/composer.py", line 39, in get_single_node
    if not self.check_event(StreamEndEvent):
  File "/usr/lib64/python2.7/site-packages/yaml/parser.py", line 98, in check_event
    self.current_event = self.state()
  File "/usr/lib64/python2.7/site-packages/yaml/parser.py", line 174, in parse_document_start
    self.peek_token().start_mark)
ParserError: expected '<document start>', but found '<block mapping start>'
  in "<unicode string>", line 2, column 1:
    limit-conf-config:
    ^

解決辦法：

配置文件的路徑配置錯誤。導致報錯。修改配置文件，問題解決。

報錯：saltstack 執行后CPU占用率為百分之百。

[root@10-10-121-200 ~]# salt 'linux-backup-saltminion' test.ping 
Salt request timed out. The master is not responding. If this error persists after verifying the master is up, worker_threads may need to be increased.

解決辦法：

使用top 查看CPU占用率為百分之百。
降低salt的線程數。

報錯：配置完return后，不能向數據庫寫入數據：

mysql> select * from salt_returns;
Empty set (0.00 sec)

解決思路：

當遇到問題是可以將log的級別更改為debug模式來調試。

解決辦法：

因為是客戶端直接返回給數據庫，所有客戶端需要數據庫的連接權限。顧所有數據庫都需要添加如下內容：

vim /etc/salt/minion
mysql.host: '192.168.2.150'
mysql.user: 'salt'
mysql.pass: 'salt'
mysql.db: 'salt'
mysql.port: 3306
[root@centos6 ~]# /etc/init.d/salt-minion restart 
Stopping salt-minion daemon:                               [  OK  ]
Starting salt-minion daemon:                               [  OK  ]
[root@centos6 ~]# 

報錯： 執行state.sls 報錯

root@10.10.121.200 files]# salt '*.service' state.sls ssh.config env=prod
chuye.backup1.service:
    Data failed to compile:
----------
    Requisite declaration file.sshd-files in SLS ssh.config is not formed as a single key dictionary
----------
    Requisite declaration file.sshd-files in SLS ssh.config is not formed as a single key dictionary
chuye.logstash1.service:
    Data failed to compile:
----------
    Requisite declaration file.sshd-files in SLS ssh.config is not formed as a single key dictionary
----------
    Requisite declaration file.sshd-files in SLS ssh.config is not formed as a single key dictionary
chuye.logstash2.service:
    Data failed to compile:
----------
    Requisite declaration file.sshd-files in SLS ssh.config is not formed as a single key dictionary
----------
    Requisite declaration file.sshd-files in SLS ssh.config is not formed as a single key dictionary

解決辦法：

 - require:
      - file.sshd-files
    - watch:
      - file.sshd-files

上面的值配置錯誤

    - require:
      - file: sshd-files
    - watch:
      - file: sshd-files

報錯：在文件中調用grains報錯

----------
          ID: zabbix-agent-install
    Function: file.managed
        Name: /etc/zabbix_agentd.conf.bak
      Result: False
     Comment: Unable to manage file: Jinja variable 'item' is undefined
     Started: 16:12:39.454923
    Duration: 17.407 ms
     Changes:   

解決辦法：

Hostname={{ grains[fqdn] }}
沒有加引號
Hostname={{ grains['fqdn'] }}

報錯：調用jinjia模板的時候多了一對中括號

#!/bin/sh
 -IPADDR=['10.10.86.159']
 +IPADDR=chuye.logstash2.service

解決辦法：

Solution:

Address {{ grains['fqdn_ip4'][0] }}
Result:
Address 111.111.111.111

file.recurse 推送文件時，源文件刪除后， 報錯。

----------
          ID: prometheus_rules
    Function: file.recurse
        Name: /data/app/prometheus/rules
      Result: False
     Comment: #### /data/app/prometheus/rules/custom.rules ####
              Source file salt://prometheus/files/rules/custom.rules?saltenv=prod not found
              
              #### /data/app/prometheus/rules/mysqld.rules ####
              Source file salt://prometheus/files/rules/mysqld.rules?saltenv=prod not found
              
              #### /data/app/prometheus/rules/kubelet.rules ####
              Source file salt://prometheus/files/rules/kubelet.rules?saltenv=prod not found

臨時解決辦法（清除緩存）

rm -rf /var/cache/salt/master/file_lists/roots/*