jmxtrans監控kafka

我們知道jmx可以將程序內部的信息暴露出來，但是要想監控這些信息的話，就還需要自己寫java程序調用jmx接口去獲取數據，並按照某種格式發送到其他地方（如監控程序zabbix,ganglia）。這時jmxtrans就派上用場了，jmxtrans的作用是自動去jvm中獲取所有jmx格式數據，並按照某種格式（json文件配置格式）輸出到其他應用程序（常用的有ganglia）

安裝

主頁：https://github.com/jmxtrans/jmxtrans（這里面也有一個下載地址，貌似版本更高）
下載地址：https://github.com/jmxtrans/jmxtrans/downloads
sudo yum install jmxtrans-20121016.145842.6a28c97fbb-0.noarch.rpm
jmxtrans安裝目錄：/usr/share/jmxtrans
jmxtrans配置文件 ：/etc/sysconfig/jmxtrans
json文件默認目錄：/var/lib/jmxtrans/
日志路徑：/var/log/jmxtrans/jmxtrans.log

啟動

sudo /usr/share/jmxtrans/jmxtrans.sh start
sudo /usr/share/jmxtrans/jmxtrans.sh start /var/lib/jmxtrans/kafka.json # 也可以指定json文件路徑

停止

sudo /usr/share/jmxtrans/jmxtrans.sh stop

監控jvm

jvm需要加入以下參數

-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=9999
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false

監控kafka示例

配置kafka參數

第一步: 在kafka集群的所有機器上安裝JMXTrans第二步: 編輯 “kafka-run-class.sh”
　　　　KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false "第三步: 編輯 “kafka-server-start.sh” 
　　　　export JMX_PORT=${JMX_PORT:-9999}    

 

監控kafka信息發送到ganglia,/var/lib/jmxtrans/kafka.json

{
    "servers": [
        {
            "port": "9999",  # jmx 端口
            "host": "127.0.0.1",   # jmx 的主機
            "queries": [
                {
                    "outputWriters": [
                        {
                            "@class": "com.googlecode.jmxtrans.model.output.GangliaWriter",
                            "settings": {
                                "groupName": "jvmheapmemory",
                                "port": 8649,
                                "host": "127.0.0.1"
                            }
                        }
                    ],
                    "obj": "java.lang:type=Memory",
                    "resultAlias": "heap",
                    "attr": [
                        "HeapMemoryUsage",
                        "NonHeapMemoryUsage"
                    ]
                },
        {
                    "outputWriters": [
                        {
                            "@class": "com.googlecode.jmxtrans.model.output.GangliaWriter",
                            "settings": {
                                "groupName": "kafka topic stats",
                                "port": 8649,   # ganglia的gmond端口
                                "host": "127.0.0.1",  # ganglia 主機
                "typeNames":[
                    "name"
                ]
                            }
                        }
                    ],
                    "obj": "kafka.server:type=BrokerTopicMetrics,name=*",
                    "resultAlias": "Kafka",
                    "attr": [
                        "Count",
                        "OneMinuteRate"
                    ]
                },
                {
                    "outputWriters": [
                        {
                            "@class": "com.googlecode.jmxtrans.model.output.GangliaWriter",
                            "settings": {
                                "groupName": "kafka server request",
                                "port": 8649,
                                "host": "127.0.0.1"
                            }
                        }
                    ],
                    "obj":"kafka.server:type=OffsetManager,name=NumGroups",
                    "resultAlias": "OffsetManager NumGroups",
                    "attr": [
                        "Value"
                    ]
                },
                {
                    "outputWriters": [
                        {
                            "@class": "com.googlecode.jmxtrans.model.output.GangliaWriter",
                            "settings": {
                                "groupName": "jvmGC",
                                "port": 8649,
                                "host": "127.0.0.1",
                                "typeNames": [
                                    "name"
                                ]
                            }
                        }
                    ],
                    "obj": "java.lang:type=GarbageCollector,name=*",
                    "resultAlias": "GC",
                    "attr": [
                        "CollectionCount",
                        "CollectionTime"
                    ]
                }
            ],
            "numQueryThreads": 2
        }
    ]
}