logstash 配置
input {
file {
path => "/logs/*.log" #日志路径
codec => multiline {
pattern => "^%{TIMESTAMP_ISO8601}"
negate => true
what => "previous"
}
}
}
filter {
if [path] =~ "access" {
mutate { replace => { type => "access" } }
grok {
match => {
"message" => "%{TIMESTAMP_ISO8601:timestamp} %{INT:cost} %{IP:remoteIp}:%{POSINT:remotePort} %{IP:localIp}:%{POSINT:localPort} %{PATH:uri} %{INT:httpCode}"
}
remove_field => ["message"]
}
} else if [path] =~ "server" {
mutate { replace => { type => "server" } }
grok {
match => {
"message" => "%{TIMESTAMP_ISO8601:timestamp} \[%{LOGLEVEL:level}\] \[%{JAVACLASS:class}\] \[%{DATA:thread}\] - %{GREEDYDATA:content}"
}
remove_field => ["message"]
}
} else {
mutate { replace => { type => "random_logs" } }
}
date {
match => [ "timestamp", "yyyy-MM-dd HH:mm:ss.SSS" ]
}
}
output {
elasticsearch {
hosts => ["http://192.168.201.37:9200"]
index => "local_test"
}
}
说明：1 . multiline 处理一个事件由多行日志构成的情况，用时间戳标记新事件。2 . =~ 正则匹配日志名。3 . mutate 替换默认属性type的值4 . remove_field 删除原日志5 . date 用业务时间戳替换日志写入时间戳
日志举例
1. access-log
2017-04-13 09:23:52.725 6 127.0.0.1:53289 127.0.0.1:9092 /user/item/11 200
2. server-log
2017-04-13 11:13:33.766 [ERROR] [com.chengying.web.UserController] [http-nio-9092-exec-7] - item id 11
com.netflix.hystrix.exception.HystrixRuntimeException: ResourceQuery#queryResourceItem(String) failed and no fallback available.
at com.netflix.hystrix.AbstractCommand$22.call(AbstractCommand.java:805)
at com.netflix.hystrix.AbstractCommand$22.call(AbstractCommand.java:790)
at com.netflix.hystrix.AbstractCommand$DeprecatedOnFallbackHoo
at rx.observers.Subscribers$5.onError(Subscribers.java:230)
