mbinlogmq 一個(gè)基于MySQL binlog協(xié)議的數(shù)據(jù)同步中間件

什么是mbinlogmq?

mbinlogmq?是一個(gè)使用C語(yǔ)言開(kāi)發(fā)的基于 MySQL?binlog?協(xié)議的一個(gè)中間件，通過(guò)模擬Slave 來(lái)實(shí)時(shí)獲取 MySQL binlog日志，并將數(shù)據(jù)變更信息以及?DB?語(yǔ)句發(fā)送到?RabbitMQ，中間通過(guò)監(jiān)聽(tīng)機(jī)制與校驗(yàn)機(jī)制來(lái)確保不宕機(jī)的情況下的100%抵達(dá)?RMQ

設(shè)計(jì)架構(gòu)

安裝&編譯

• mbinlog依賴(lài)于一些第三方庫(kù)：

請(qǐng)先安裝依賴(lài)軟件：?cURL?、mysql、rabbitmq，安裝完成后記住其安裝的路徑，進(jìn)入下面的步驟

• 下載mbinlogmq源碼

git clone https://gitee.com/josinli/mbinlogmq.git

• 修改?CMakeLists.txt，將如下的幾行變?yōu)槟厦姘惭b的路徑：

set(MYSQL_INCLUDE     /usr/local/mysql/include)      # 您系統(tǒng)的MySQL頭文件路徑
set(MYSQL_LIBRARY_DIR /usr/local/mysql/lib)          # 您系統(tǒng)的MySQL庫(kù)文件路徑
set(CURL_DIR          /usr/local/Cellar/curl/7.58.0) # 您系統(tǒng)的curl安裝路徑

• 執(zhí)行編譯安裝命令

mkdir build
cd build
cmake .. && make && sudo make install	

• 修改?/etc/mbinlogmq/binlog.xml?您的配置信息

    rabbitmq 
    127.0.0.1
    root
    3333
    3306
    6
    1




    127.0.0.1
    guest
    guest
    15672
    1
    amq.default
    Invalid
    fanout
    /

• 啟動(dòng)您的?minblogmq

mbinlogmq -k start

此時(shí)進(jìn)入?MySQL?操作，同時(shí)登錄?RabbitMQ?查看消息是否成功抵達(dá)，目前的版本存在一些或多或少的問(wèn)題，切勿生產(chǎn)環(huán)境使用。

mbinlogmq會(huì)針對(duì)不同的binlog日志生成四種類(lèi)型的消息，消息類(lèi)型如下

• 消息全部為?JSON?格式
• 格式如下

1、UPDATE?語(yǔ)句，etype等于 1，包含一個(gè)pre對(duì)象 和?new對(duì)象，分別對(duì)應(yīng)修改之前的數(shù)據(jù)和新的數(shù)據(jù)

{
    "etype": 1,
    "data" : {
        "pre": {
            "id":1,
            "ad":2
        },
        "new": {
            "id":1,
            "ad":3
        }
    }
}

2、INSERT?消息， etype 等于 2

{
    "etype": 1,
    "data" : {
        "id":1,
        "ad":3
    }
}

3、DELETE?消息， etype 等于 3

{
    "etype": 3,
    "data" : {
        "id":1,
        "ad":3
    }
}

4、SQL 語(yǔ)句?消息， etype 等于 4

{
    etype: 4,
    data: "ALTER TABLE `books`.`test_at` MODIFY COLUMN `cc` datetime(0) NULL DEFAULT NULL AFTER `bb`"
}

示例消息如下：

 [x] Received {"etype":4,"data":"ALTER TABLE `books`.`test_at` \nMODIFY COLUMN `bb` date NULL DEFAULT NULL AFTER `aa`"}
 [x] Received {"etype":4,"data":"ALTER TABLE `books`.`test_at` \nMODIFY COLUMN `bb` date NOT NULL AFTER `aa`"}
 [x] Received {"etype":4,"data":"BEGIN"}
 [x] Received {"etype":1,"data":{"pre":{"id":1,"book_name":"傲世丹神1","book_cover":"http://img.c0m.io/quanben.io/upload/thumbnail/book_0_999/book_1.jpg","add_time":"2019-7-23 14:28:43","author_name":"寂小賊111","introduction":"廢柴少年得無(wú)上傳承,獲逆天神脈,學(xué)絕世神功,掌握超絕丹術(shù),這使他...","test_date":"2019-7-29"},"new":{"id":1,"book_name":"傲世丹神1","book_cover":"http://img.c0m.io/quanben.io/upload/thumbnail/book_0_999/book_1.jpg","add_time":"2019-7-23 21:47:36","author_name":"寂小賊","introduction":"廢柴少年得無(wú)上傳承,獲逆天神脈,學(xué)絕世神功,掌握超絕丹術(shù),這使他...","test_date":"2019-7-29"}}}
 [x] Received {"etype":4,"data":"COMMIT"}
 [x] Received {"etype":4,"data":"BEGIN"}
 [x] Received {"etype":1,"data":{"pre":{"id":1,"book_name":"傲世丹神1","book_cover":"http://img.c0m.io/quanben.io/upload/thumbnail/book_0_999/book_1.jpg","add_time":"2019-7-23 21:47:36","author_name":"寂小賊","introduction":"廢柴少年得無(wú)上傳承,獲逆天神脈,學(xué)絕世神功,掌握超絕丹術(shù),這使他...","test_date":"2019-7-29"},"new":{"id":1,"book_name":"傲世丹神","book_cover":"http://img.c0m.io/quanben.io/upload/thumbnail/book_0_999/book_1.jpg","add_time":"2019-7-23 21:47:54","author_name":"寂小賊","introduction":"廢柴少年得無(wú)上傳承,獲逆天神脈,學(xué)絕世神功,掌握超絕丹術(shù),這使他...","test_date":"2019-7-29"}}}
 [x] Received {"etype":4,"data":"COMMIT"}

有問(wèn)題可以及時(shí)通過(guò) ISSUE 反饋，反饋地址：GITEE Go Go Go!!!