• flink cdc笔记(二):flink cdc的两种开发方式


    支持两种开发方式:SQLDataStream的方式

    1,sql的方式

     Mysqloracle在cdc sql中的数据类型映射

    MySQL CDC Connector — Flink CDC documentation

    MySQL type

    Flink SQL type

    TINYINT

    TINYINT

    SMALLINT
    TINYINT UNSIGNED

    SMALLINT

    INT
    MEDIUMINT
    SMALLINT UNSIGNED

    INT

    BIGINT
    INT UNSIGNED

    BIGINT

    BIGINT UNSIGNED

    DECIMAL(20, 0)

    REAL
    FLOAT

    FLOAT

    DOUBLE

    DOUBLE

    NUMERIC(p, s)
    DECIMAL(p, s)
    where p <= 38

    DECIMAL(p, s)

    NUMERIC(p, s)
    DECIMAL(p, s)
    where 38 < p <= 65

    STRING

    BOOLEAN
    TINYINT(1)
    BIT(1)

    BOOLEAN

    DATE

    DATE

    TIME [(p)]

    TIME [(p)]

    TIMESTAMP [(p)]
    DATETIME [(p)]

    TIMESTAMP [(p)]

    CHAR(n)

    CHAR(n)

    VARCHAR(n)

    VARCHAR(n)

    BIT(n)

    BINARY(⌈n/8⌉)

    Oracle type

    Flink SQL type

    NUMBER(p, s <= 0), p - s < 3

    TINYINT

    NUMBER(p, s <= 0), p - s < 5

    SMALLINT

    NUMBER(p, s <= 0), p - s < 10

    INT

    NUMBER(p, s <= 0), p - s < 19

    BIGINT

    NUMBER(p, s <= 0), 19 <= p - s <= 38

    DECIMAL(p - s, 0)

    NUMBER(p, s > 0)

    DECIMAL(p, s)

    NUMBER(p, s <= 0), p - s > 38

    STRING

    FLOAT
    BINARY_FLOAT

    FLOAT

    DOUBLE PRECISION
    BINARY_DOUBLE

    DOUBLE

    NUMBER(1)

    BOOLEAN

    DATE
    TIMESTAMP [(p)]

    TIMESTAMP [(p)] [WITHOUT TIMEZONE]

    TIMESTAMP [(p)] WITH TIME ZONE

    TIMESTAMP [(p)] WITH TIME ZONE

    TIMESTAMP [(p)] WITH LOCAL TIME ZONE

    TIMESTAMP_LTZ [(p)]

    CHAR(n)
    NCHAR(n)
    NVARCHAR2(n)
    VARCHAR(n)
    VARCHAR2(n)
    CLOB
    NCLOB
    XMLType

    STRING

    BLOB
    ROWID

    BYTES

    SQL读出来的内容是的Tuple2<Boolean,Row>的格式

     

     

    2,DataStream的方式

     DataStream读出来的内容可以设置成JSON的格式,方便解析

     {"before":null,"after":{"C_COLLECTORID":"W2","S_TIME":1655497161000,"S_NUM":{"scale":0,"value":"BA=="},"S_LASTESTTIME":1655916525000,"S_INITTIME":1655497161000,"W_WORKSHOP":null,"W_WORKCLASSID":null,"C_FACTORY":"2720"},"source":{"version":"1.5.4.Final","connector":"oracle","name":"oracle_logminer","ts_ms":1657262696008,"snapshot":"last","db":"ORCL.TEST","sequence":null,"schema":"FLINKUSER","table":"GD_WCOLLECTCURRENTSTAT","txId":null,"scn":"1135085","commit_scn":null,"lcr_position":null},"op":“c","ts_ms":1657262696008,"transaction":null}

     

     before就是数据改变前的内容,after就是数据改变后的内容,op为c代表新增的,下面是op的对应数据改变类型

     

  • 相关阅读:
    Vue3最佳实践 第六章 Pinia,Vuex与axios,VueUse 3(VueUse )
    果脯干货经营小程序商城的效果如何
    复习中的一些疑点细节
    线性调频雷达回波仿真+脉冲压缩仿真
    RediSearch CentOS7编译安装
    Java单表实现评论回复功能
    视频监控系统/视频汇聚平台EasyCVR如何反向代理进行后端保活?
    猿创征文| 怎么提高 Java 开发效率?使用 Idea 和它的插件
    CET4汉译英part
    Android中如何知道我使用的是 AndroidX 库还是 Support Library
  • 原文地址:https://blog.csdn.net/YiRan_Zhao/article/details/126956600