FlinkCDC介绍及使用

CDC简介

        什么是CDC？

                cdc是Change Data Capture(变更数据获取)的简称。核心思想是，监测并捕获数据库的

变动(包括数据或数据表的插入，更新以及删除等)，将这些变更按发生的顺序完整记录下来，写入到消息中间件以供其它服务进行订阅及消费

        CDC的种类？

CDC主要分为基于查询和基于Binlog两种方式，咱们主要了解一下这两种的区别：

FlinkCDC案例实操

        开启MySQL Binlog并重启MySQL

sudo vim/etc/my.cnf

把需要监控的数据库名写入到里面

重启mysql

sudo systemctl restart mysqld

DataStream方式的应用

导入依赖

    8

    8

    UTF-8

    1.17.1

    2.4.0

    3.3.4

    

        org.apache.flink

        flink-clients

        ${flink.version}

        provided

    

    

        org.apache.flink

        flink-connector-kafka

        ${flink.version}

    

    

        org.apache.flink

        flink-json

        ${flink.version}

        provided

    

    

        com.alibaba

        fastjson

        1.2.83

    

    

    

        org.apache.hadoop

        hadoop-client-api

        ${hadoop.version}

        

    

    

        org.apache.hadoop

        hadoop-client-runtime

        ${hadoop.version}

        

    

    

        org.apache.logging.log4j

        log4j-to-slf4j

        2.14.0

        provided

    

    

        org.apache.logging.log4j

        log4j-api

        2.14.0

    

    

        org.slf4j

        slf4j-log4j12

        1.7.36

    

    

    

        com.ververica

        flink-connector-mysql-cdc

        ${flink-cdc.vesion}

    

    

    

        org.apache.flink

        flink-table-api-java-bridge

        ${flink.version}

        provided

    

    

        org.apache.flink

        flink-table-planner-loader

        ${flink.version}

        provided

    

    

        org.apache.flink

        flink-table-runtime

        ${flink.version}

        provided

    

编写代码