momo's Blog.

将Python日志发送到ES中

字数统计: 259阅读时长: 1 min
2021/07/01 Share

前言

某些时候,我们需要实时的去查看日志,但是却没有方法直接登录服务器,那为什么不直接发送到日志处理的平台呢.

自定义Hander

es.py

1
2
3
4
5
6
7
8
9
10
from elasticsearch import Elasticsearch
import datetime


class EsProducer:
def __init__(self, es_host, es_port):
self.es = Elasticsearch(hosts=es_host, port=es_port, timeout=10000)

def send(self, body):
self.es.index(index=f"es-monitor-{datetime.datetime.utcnow().strftime('%Y-%m-%d')}", body=body)
1
2
3
4
5
6
7
8
9
10
11
from config import es_host, es_port
import logging

class EsHandler(logging.StreamHandler):
def __init__(self):
self.es = es.EsProducer(es_host, es_port)
super().__init__()

def emit(self, record):
msg = self.format(record)
self.es.send(msg)

更改配置文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# 在handlers中追加
[handlers]
keys=consoleHandler,fileHandler,esHandler


# 在handlers中追加
[logger_jsonmodule]
level=INFO
handlers=fileHandler,esHandler
qualname=jsonmodule


# 添加handler
[handler_esHandler]
class=log.log_handers.EsHandler
level=INFO
formatter=json

完整配置如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
[loggers]
keys=root,jsonmodule

[handlers]
keys=consoleHandler,fileHandler,esHandler

[formatters]
keys=simpleFormatter,json

[logger_root]
level=DEBUG
handlers=consoleHandler

[logger_jsonmodule]
level=INFO
handlers=fileHandler,esHandler
qualname=jsonmodule

[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=simpleFormatter
args=(sys.stdout,)

[handler_fileHandler]
class=log.log_handers.CustomFileHandler
level=DEBUG
formatter=json
args=("app.log", 'd', 1, 15, 'utf-8')

[handler_esHandler]
class=log.log_handers.EsHandler
level=INFO
formatter=json


[formatter_json]
class=log.formatter.CustomJsonFormatter
format=%(asctime)s %(threadName)s %(thread)d %(lineno)d %(module)s %(funcName)s %(message)s

[formatter_simpleFormatter]
format=%(asctime)s %(funcName)s %(filename)s [line: %(lineno)d] %(levelname)s:%(message)s
CATALOG
  1. 1. 前言
  2. 2. 自定义Hander
  3. 3. 更改配置文件