Zabbix4.x 歷史數據存儲到Elasticsearch7.x

一、簡介

Zabbix 3.4.6 版本開始支持歷史數據存儲到 Elasticsearch, 早就想測試這個功能，最近有個需求需保存 zabbix 的歷史數據上達十億條，因此決定測試這功能的實用性，事實證明確實效果挺好。從今以后 zabbix 也支持大量的歷史數據。

二、安裝 ELK（這里不再講解，不懂的可以參考：https://www.cnblogs.com/liugp/p/11789933.html）

三、添加 Elasticsearch mapping

Elasticsearch 支持 Zabbix 的監控項類型：uint，dbl，str，log，text，對應如下：

Zabbix 監控項數據類型	對應 Zabbix 表	對應 Elasticsearch 類型
Numeric(unsigned)(無符號整型)	history_uint	uint
Numeric(float)（浮點型）	history	dbl
Character（字符）	history_str	str
Log（日志）	history_log	log
Text	history_text	text

ES7.x創建五中類型數據索引

PUT /uint
{
    "settings":{
        "number_of_replicas":1,
        "number_of_shards":5
    },
    "mappings":{
        "properties":{
            "itemid":{
                "type":"long"
            },
            "clock":{
                "format":"epoch_second",
                "type":"date"
            },
            "value":{
                "type":"long"
            }
        }
    }
}


PUT /dbl
{
    "settings":{
        "number_of_replicas":1,
        "number_of_shards":5
    },
    "mappings":{
        "properties":{
            "itemid":{
                "type":"long"
            },
            "clock":{
                "format":"epoch_second",
                "type":"date"
            },
            "value":{
                "type":"double"
            }
        }
    }
}

PUT /log
{
    "settings":{
        "number_of_replicas":1,
        "number_of_shards":5
    },
    "mappings":{
        "properties":{
            "itemid":{
                "type":"long"
            },
            "clock":{
                "format":"epoch_second",
                "type":"date"
            },
            "value":{
                "fields":{
                    "analyzed":{
                        "index":true,
                        "type":"text",
                        "analyzer":"standard"
                    }
                },
                "index":false,
                "type":"text"
            }
        }
    }
}

PUT /text
{
    "settings":{
        "number_of_replicas":1,
        "number_of_shards":5
    },
    "mappings":{
        "properties":{
            "itemid":{
                "type":"long"
            },
            "clock":{
                "format":"epoch_second",
                "type":"date"
            },
            "value":{
                "fields":{
                    "analyzed":{
                        "index":true,
                        "type":"text",
                        "analyzer":"standard"
                    }
                },
                "index":false,
                "type":"text"
            }
        }
    }
}

PUT /str
{
    "settings":{
        "number_of_replicas":1,
        "number_of_shards":5
    },
    "mappings":{
        "properties":{
            "itemid":{
                "type":"long"
            },
            "clock":{
                "format":"epoch_second",
                "type":"date"
            },
            "value":{
                "fields":{
                    "analyzed":{
                        "index":true,
                        "type":"text",
                        "analyzer":"standard"
                    }
                },
                "index":false,
                "type":"text"
            }
        }
    }
}

注意：通過上面創建的索引，zabbix-server無法添加數據，因為ES7移除了type類型導致的，但是可以讓zabbix自動創建

ES6.x創建五中類型數據索引

PUT /uint
{
    "settings":{
        "index":{
            "number_of_replicas":1,
            "number_of_shards":5
        }
    },
    "mappings":{
        "values":{
            "properties":{
                "itemid":{
                    "type":"long"
                },
                "clock":{
                    "format":"epoch_second",
                    "type":"date"
                },
                "value":{
                    "type":"long"
                }
            }
        }
    }
}


PUT /dbl
{
    "settings":{
        "index":{
            "number_of_replicas":1,
            "number_of_shards":5
        }
    },
    "mappings":{
        "values":{
            "properties":{
                "itemid":{
                    "type":"long"
                },
                "clock":{
                    "format":"epoch_second",
                    "type":"date"
                },
                "value":{
                    "type":"double"
                }
            }
        }
    }
}

PUT /log
{
    "settings":{
        "index":{
            "number_of_replicas":1,
            "number_of_shards":5
        }
    },
    "mappings":{
        "values":{
            "properties":{
                "itemid":{
                    "type":"long"
                },
                "clock":{
                    "format":"epoch_second",
                    "type":"date"
                },
                "value":{
                    "fields":{
                        "analyzed":{
                            "index":true,
                            "type":"text",
                            "analyzer":"standard"
                        }
                    },
                    "index":false,
                    "type":"text"
                }
            }
        }
    }
}

PUT /text
{
    "settings":{
        "index":{
            "number_of_replicas":1,
            "number_of_shards":5
        }
    },
    "mappings":{
        "values":{
            "properties":{
                "itemid":{
                    "type":"long"
                },
                "clock":{
                    "format":"epoch_second",
                    "type":"date"
                },
                "value":{
                    "fields":{
                        "analyzed":{
                            "index":true,
                            "type":"text",
                            "analyzer":"standard"
                        }
                    },
                    "index":false,
                    "type":"text"
                }
            }
        }
    }
}

PUT /str
{
    "settings":{
        "index":{
            "number_of_replicas":1,
            "number_of_shards":5
        }
    },
    "mappings":{
        "values":{
            "properties":{
                "itemid":{
                    "type":"long"
                },
                "clock":{
                    "format":"epoch_second",
                    "type":"date"
                },
                "value":{
                    "fields":{
                        "analyzed":{
                            "index":true,
                            "type":"text",
                            "analyzer":"standard"
                        }
                    },
                    "index":false,
                    "type":"text"
                }
            }
        }
    }
}

es6.x curl創建方式

curl -H "Content-Type:application/json" -XPUT http://192.168.182.132:9200/uint -d ' { "settings" : { "index" : { "number_of_replicas" : 1, "number_of_shards" : 5 } }, "mappings" : { "values" : { "properties" : { "itemid" : { "type" : "long" }, "clock" : { "format" : "epoch_second", "type" : "date" }, "value" : { "type" : "long" } } } } } '

curl -H "Content-Type:application/json" -XPUT http://192.168.182.132:9200/dbl -d ' { "settings" : { "index" : { "number_of_replicas" : 1, "number_of_shards" : 5 } }, "mappings" : { "values" : { "properties" : { "itemid" : { "type" : "long" }, "clock" : { "format" : "epoch_second", "type" : "date" }, "value" : { "type" : "double" } } } } } '

curl -H "Content-Type:application/json" -XPUT http://192.168.182.132:9200/log -d ' { "settings" : { "index" : { "number_of_replicas" : 1, "number_of_shards" : 5 } }, "mappings" : { "values" : { "properties" : { "itemid" : { "type" : "long" }, "clock" : { "format" : "epoch_second", "type" : "date" }, "value" : { "fields" : { "analyzed" : { "index" : true, "type" : "text", "analyzer" : "standard" } }, "index" : false, "type" : "text" } } } } } '

curl -H "Content-Type:application/json" -XPUT http://192.168.182.132:9200/text -d ' { "settings" : { "index" : { "number_of_replicas" : 1, "number_of_shards" : 5 } }, "mappings" : { "values" : { "properties" : { "itemid" : { "type" : "long" }, "clock" : { "format" : "epoch_second", "type" : "date" }, "value" : { "fields" : { "analyzed" : { "index" : true, "type" : "text", "analyzer" : "standard" } }, "index" : false, "type" : "text" } } } } } '

curl -H "Content-Type:application/json" -XPUT http://192.168.182.132:9200/str -d ' { "settings" : { "index" : { "number_of_replicas" : 1, "number_of_shards" : 5 } }, "mappings" : { "values" : { "properties" : { "itemid" : { "type" : "long" }, "clock" : { "format" : "epoch_second", "type" : "date" }, "value" : { "fields" : { "analyzed" : { "index" : true, "type" : "text", "analyzer" : "standard" } }, "index" : false, "type" : "text" } } } } } '

注意：zabbix的大坑，如果是es7，es7上有數據，但是圖形會顯示無數據，這個后期有待解決。如果有哪位大神已經解決了，可以在評論區回復我，Thank you!!!!。

 

 

四、配置zabbix服務器

Zabbix 安裝過程忽略

配置 zabbix_server.conf 文件

在 /etc/zabbix/zabbix_server.conf 文件下添加 elasticsearch 配置，指定歷時數據使用 elasticsearch。

# vim /etc/zabbix/zabbix_server.conf

HistoryStorageURL=http://192.168.182.132:9200
HistoryStorageTypes=uint,dbl,str,log,text

配置zabbix.conf.php文件

在/home/wwwroot/default/zabbix/conf/zabbix.conf.php文件下添加elasticsearch配置

# vim /home/wwwroot/default/zabbix/conf/zabbix.conf.php

// Elasticsearch url (can be string if same url is used for all types).
$HISTORY['url'] = 'http://192.168.182.132:9200';
// Value types stored in Elasticsearch.
$HISTORY['types'] = ['uint', 'text', 'log', 'str', 'dbl'];

多台 elasticsearch 集群可按以下格式配置

$HISTORY['url'] = [ 'uint' => 'http://192.168.182.132:9200 ', 'text' => 'http://192.168.182.133:9200', 'log' => 'http://192.168.182.134:9200'];
$HISTORY['types'] = ['uint', 'text','log'];

五、驗證數據

啟動 zabbix 后，使用 kibana 查看 es 集群是否有相關索引，方法這里就忽略。