SpringBoot集成分布式日志管理平台graylog

GrayLog是一个轻量型的分布式日志管理平台，一个开源的日志聚合、分析、审计、展示和预警工具。
功能上来说，和ELK类似，但又比ELK要简单轻量许多。

GrayLog包含Elasticsearch、MongoDb 和Graylog三个模块

环境搭建

采用docker-compose来搭建graylog测试环境，具体配置如下

# docker-compose配置参考：https://docs.graylog.org/en/3.3/pages/installation/docker.html
version: '3'

services:

  mongo:
    image: mongo:3
    container_name: graylog_demo_mongo
    restart: unless-stopped
    #    volumes:
    #      - ./graylog/mongo_data:/data/db
    networks:
      - graylog

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch-oss:6.8.10
    container_name: graylog_demo_elasticsearch
    restart: unless-stopped
    #    volumes:
    #      - ./graylog/es_data:/usr/share/elasticsearch/data
    environment:
      - http.host=0.0.0.0
      - transport.host=localhost
      - network.host=0.0.0.0
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    deploy:
      resources:
        limits:
          memory: 1g
    networks:
      - graylog

  graylog:
    image: graylog/graylog:3.3
    container_name: graylog_demo_graylog
    restart: unless-stopped
    #    volumes:
    #      - ./graylog/graylog_data_journal:/usr/share/graylog/data/journal
    environment:
      - GRAYLOG_PASSWORD_SECRET=somepasswordpepper # CHANGE ME (must be at least 16 characters)!
      - GRAYLOG_ROOT_PASSWORD_SHA2=8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918 # Password: admin
      - GRAYLOG_HTTP_EXTERNAL_URI=http://127.0.0.1:9000/ # TODO 这里填写自己的ip地址
    ports:
      # Graylog web interface and REST API
      - 9000:9000
      # Syslog TCP
      - 1514:1514
      # Syslog UDP
      - 1514:1514/udp
      # GELF TCP
      - 12201:12201
      # GELF UDP
      - 12201:12201/udp
    networks:
      - graylog
    depends_on:
      - mongo
      - elasticsearch

# 网桥graylog -> 方便相互通讯
networks:
  graylog:
    driver: bridge
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68

启动服务

docker-compose -f docker-compose-graylog.yml -p graylog_demo up -d

访问地址：http://ip地址:9001 默认登录账号密码：admin/admin

①、新建input http input

测试是否正常发送

curl -XPOST http://localhost:12201/gelf -p0 -d '{"message":"hello这是一条消息", "host":"127.0.0.1", "facility":"test", "topic": "meme"}'
1

②、创建gref udp input
由于http发送效率比较低，所以后面springboot会采用UDP方法来采集日志

一、依赖

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>springboot-demoartifactId>
        <groupId>com.etgroupId>
        <version>1.0-SNAPSHOTversion>
    parent>
    <modelVersion>4.0.0modelVersion>

    <artifactId>grayLogartifactId>

    <properties>
        <maven.compiler.source>8maven.compiler.source>
        <maven.compiler.target>8maven.compiler.target>
    properties>
    <dependencies>

        <dependency>
            <groupId>org.springframework.bootgroupId>
            <artifactId>spring-boot-starter-webartifactId>
        dependency>

        <dependency>
            <groupId>org.springframework.bootgroupId>
            <artifactId>spring-boot-autoconfigureartifactId>
        dependency>

        <dependency>
            <groupId>biz.paluch.logginggroupId>
            <artifactId>logstash-gelfartifactId>
            <version>1.11.1version>
        dependency>

        <dependency>
            <groupId>org.projectlombokgroupId>
            <artifactId>lombokartifactId>
        dependency>

    dependencies>
project>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

二、配置文件和启动类

server:
  port: 8088
groylog:
  ip: 127.0.0.1
  port: 12201
  maximumMessageSize: 8192
1
2
3
4
5
6
7

@SpringBootApplication
public class DemoApplication {

   public static void main(String[] args) {
      SpringApplication.run(DemoApplication.class, args);
   }
}
1
2
3
4
5
6
7

logback.xml

<configuration>

    <appender name="GELF" class="biz.paluch.logging.gelf.logback.GelfLogbackAppender">
        <host>udp:127.0.0.1host>
        <port>12201port>
        <version>1.1version>
        <facility>springboot-with-graylogfacility>
        <extractStackTrace>trueextractStackTrace>
        <filterStackTrace>truefilterStackTrace>
        <mdcProfiling>truemdcProfiling>
        <timestampPattern>yyyy-MM-dd HH:mm:ss,SSSStimestampPattern>
        <maximumMessageSize>8192maximumMessageSize>

        
        <mdcFields>mdcField1,mdcField2mdcFields>
        <dynamicMdcFields>mdc.*,(mdc|MDC)fieldsdynamicMdcFields>
        <includeFullMdc>trueincludeFullMdc>
    appender>
    <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d{HH:mm:ss.SSS} %-5level %logger{36} - %msg%npattern>
        encoder>
    appender>

    <root level="INFO">
        <appender-ref ref="GELF"/>
        <appender-ref ref="CONSOLE"/>
    root>

configuration>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

三、Controller

@Slf4j
@Controller
public class HelloWorldController {
    @Autowired
    DemoService demoService;

    @RequestMapping("/hello")
    @ResponseBody
    public Map<String, Object> showHelloWorld(){
        Map<String, Object> map = new HashMap<>();
        map.put("msg", "HelloWorld");
        log.info("my name is HelloWorldController");
        demoService.hello();
        return map;
    }

    @GetMapping("log")
    public String log(){

        log.error("error");
        log.warn("warn");
        log.info("info");
        log.debug("debug");
        int i = 1/0;
        return "success";
    }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

测试：

启动服务
登录graylog，查看日志是否已经发送过去