Nginx 配置 Json 格式日志

修改 Nginx 配置文件,定义输出 json 格式的日志,便于 filebeat 和 logstash 收集

http {
    log_format  main  '{"@timestamp": "$time_iso8601", '
                        '"clientRealIp": "$remote_addr", '
                        '"scheme": "$scheme", '
                        '"method": "$request_method", '
                        '"host": "$host", '
                        '"url": "$request_uri", '
                        '"size": $body_bytes_sent, '
                        '"referrer": "$http_referer", '
                        '"agent": "$http_user_agent", '
                        '"upstream_addr": "$upstream_addr", '
                        '"request_time": $request_time, '
                        '"request_length": $request_length, '
                        '"upstream_connect_time": "$upstream_connect_time", '
                        '"upstream_response_time": "$upstream_response_time", '
                        '"upstream_status": "$upstream_status", '
                        '"status": "$status"}';
}

Filebeat 配置文件

编写 filebeat 配置文件,收集 Nginx 的 access.log 和 error.log,并且添加自定义字段和标签存储到 redis

cat /etc/filebeat/filebeat-nginx.yml 
filebeat.inputs:
- type: log
  enabled: true
  json.keys_under_root: true
  paths:
    - /usr/local/nginx/logs/access.log
  tags: ["access"]
  fields:
    server: nginx
    type: nginx-access
  fields_under_root: true

- type: log
  enabled: true
  json.keys_under_root: true
  paths:
    - /usr/local/nginx/logs/error.log
  tags: ["error"]
  fields:
    server: nginx
    type: nginx-error
  fields_under_root: true

processors:
- drop_fields:
    fields: ["input_type", "ecs.version", "host.name", "agent", "log.offset"]

#output.console:

output.redis:
  hosts: ["10.10.110.194:56379"]
  password: "123456"
  key: "nginx"
  db: 0
  datatype: list

控制台调试 Filebeat 输出的日志数据

通过 drop_fields 去控制我们想要输出的字段,得到精简的日志数据

{
    "@timestamp": "2020-09-07T18:08:49.000Z",
    "@metadata": {
        "beat": "filebeat",
        "type": "_doc",
        "version": "7.9.0"
    },
    "server": "nginx",
    "ecs": {},
    "host": {},
    "log": {
        "file": {
            "path": "/usr/local/nginx/logs/access.log"
        }
    },
    "json": {},
    "input": {
        "type": "log"
    },
    "type": "nginx-access",
    "message": "10.10.110.194 - - [08/Sep/2020:02:08:41 +0800] \"GET /848dd HTTP/1.1\" 404 153 \"-\" \"curl/7.29.0\"",
    "tags": ["access"]
}

Logstash 读取 Redis 中的日志数据

logstash 读取 redis 中的日志数据,并且在 Kibana展示 Nginx 日志

# logstash配置文件通过我们定义的fields字段和标签匹配数据,将不同的数据存储到不同的index
input {
  redis {
    host => "10.10.110.194"
    port => 56379
    password => "123456"
    db => "0"
    data_type => "list"
    key => "nginx"
  }
}

output { # 通过字段和标签判断日志数据,存储到不同的index
  if [type] == "nginx-access" {
    if [tags][0] == "access" {
      elasticsearch {
        hosts  => ["http://10.10.110.191:9200","http://10.10.110.192:9200","http://10.10.110.193:9200"]
        index  => "filebeat-nginx-access%{+YYYY.MM.dd}"
      }
      stdout { codec=> rubydebug }
    }
  }
  if [type] == "nginx-error" {
    if [tags][0] == "error" {
      elasticsearch {
        hosts  => ["http://10.10.110.191:9200","http://10.10.110.192:9200","http://10.10.110.193:9200"]
        index  => "filebeat-nginx-error%{+YYYY.MM.dd}"
      }
      stdout { codec=> rubydebug }
    }
  }
}

Kibana 展示 Nginx 日志

我们可以在 kibana 上创建索引,查看 Nginx 日志,通过字段去统计和展示日志数据