使用python logging 模塊將日志打印到Logstash

目的：將腳本運行時的日志打印到ES，通過kibana展示，特別是后台運行時，可知道腳本運行狀態。

0.安裝logstash並配置

input {
    tcp {
        port => 5044
        codec => "json"
    }
}

output {
  elasticsearch { 
       hosts => ["10.39.40.2:9200","10.39.40.5:9200","10.39.40.6:9200"]
       manage_template => false
       index => "djangologs_%{+YYYY.MM.dd}"
    }
  stdout { codec => rubydebug }
}

1.安裝python-logstash模塊

pip install python-logstash

2.示例代碼

import logging
import logstash

host = '10.39.46.88'

test_logger = logging.getLogger('python-logstash-logger')
test_logger.setLevel(logging.DEBUG)
test_logger.addHandler(logstash.TCPLogstashHandler(host, 5044, version=1))

test_logger.error('python-logstash: test logstash error message.')
test_logger.info('python-logstash: test logstash info message.')
test_logger.warning('python-logstash: test logstash warning message.')

3.通過kibana查看

 4.python-logstash 集成到django 中

LOGGING = {
  ...
  'handlers': {
      'logstash': {
          'level': 'DEBUG',
          'class': 'logstash.LogstashHandler',
          'host': 'localhost',
          'port': 5959, # Default value: 5959
          'version': 1, # Version of logstash event schema. Default value: 0 (for backward compatibility of the library)
          'message_type': 'logstash',  # 'type' field in logstash message. Default value: 'logstash'.
          'fqdn': False, # Fully qualified domain name. Default value: false.
          'tags': ['tag1', 'tag2'], # list of tags. Default: None.
      },
  },
  'loggers': {
      'django.request': {
          'handlers': ['logstash'],
          'level': 'DEBUG',
          'propagate': True,
      },
  },
  ...
}

參考:

https://pypi.org/project/python-logstash/

https://pypi.org/project/python-logstash-async/1.0.0/