安装 Tomcat

Tomcat 属于 Java 应用,这里收集 Tomcat 日志作为示例

# 下载软件包
wget -P /server/tools/https://mirror.bit.edu.cn/apache/tomcat/tomcat-8/v8.5.53/bin/apache-tomcat-8.5.53.tar.gz

# 解压
tar xf apache-tomcat-8.5.53.tar.gz -C /usr/local/ && mv /usr/local/apache-tomcat-8.5.53/ /usr/local/tomcat

# 启动 tomcat
/usr/local/tomcat/bin/startup.sh

编写 Filebeat pipeline

filebeat 获取所有不以 “[“ 开头的行,并将它们合并到上一行以 “[“ 开头的行之后

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /usr/local/tomcat/logs/catalina.out
  tags: ["catalina"]
  fields:
    server: tomcat
    type: tomcat-catalina
  fields_under_root: true
  multiline:
    pattern: '^\['
    negate: true
    match: after

#output.console:

output.redis:
  hosts: ["10.10.110.194:56379"]
  password: "123456"
  key: "tomcat"
  db: 0
  datatype: list

模拟 Tomcat 报错日志

往 Tomcat 的日志写入错误信息,模拟报错信息

cat > /usr/local/tomcat/logs/catalina.out << EOF
Sep 09, 2020 5:50:33 PM org.apache.catalina.startup.Catalina stopServer
SEVERE: Catalina.stop: 
org.xml.sax.SAXParseException; systemId: file:/usr/local/tomcat/conf/server.xml; lineNumber: 22; columnNumber: 45; Attribute name "dda" associated with an 
element type "Server" must be followed by the ' = ' character.
        at java.xml/com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1243)
        at java.xml/com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:635)
        at org.apache.tomcat.util.digester.Digester.parse(Digester.java:1495)
        at org.apache.catalina.startup.Catalina.stopServer(Catalina.java:485)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.apache.catalina.startup.Bootstrap.stopServer(Bootstrap.java:389)
        at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:479)
EOF

编写 Logstash pipeline

input {
  redis {
    host => "10.10.110.194"
    port => 56379
    password => "123456"
    db => "0"
    data_type => "list"
    key => "tomcat"
  }
}

output {
  if [type] == "tomcat-catalina" {
    if [tags][0] == "catalina" {
      elasticsearch {
        hosts  => ["http://10.10.110.191:9200","http://10.10.110.192:9200","http://10.10.110.193:9200"]
        index  => "filebeat-tomcat-%{+YYYY.MM.dd}"
      }
      stdout { codec=> rubydebug }
    }
  }
}

Kibana 展示数据

这里展示数据是不显示完全的,我们可以指定字段查看更详细的信息

指定 message 字段,查看被合并成一行的 Tomcat 报错日志