canal-go |中文
canal-go is a golang Client of alibaba's open source component Canal,and it provides golang developer a much easier way to use Canal.
Cancal is a incremental Publish&Subscription component based on mysql's binlog.To learn more a bout Cancal,please visit https://github.com/alibaba/canal/wiki
log based incremental pub&sub model can be applied to many business cases:
- Database Mirroring
- Realtime Database backup
- Multi-level index
- search build
- Busienss cache refresh
- Notification on price change(in e-commerce site) and so on
canal-go as a golang client of canal,it can be used in situations where Canal is being used.We have listed some using scenarios above,here are some more detailed use cases:
-
As a substitution of polling Database to monitor Database changes,it is much more effective than polling.
-
Realtime Update data in search engine when data in mysql database changed.For example,in a E-commerce site,when Product information changed,it can be updated to Elasticsearch、solr,etc in a realtime way.
-
Realtime update data in cache,Also take E-commerce site as an example,when price,inventory and other product information changed,it can be updated to Redis in a realtime way.
-
Database Backup,synchronization.
-
To trigger some other business logics on data change,for example,when a customer place an order without payment in a certain period of time and so the order is cancelled automatically,we get the changed status of the order and then we can push a notification to the customer(or take some other actions)
-
Push changed data to RabbitMq,kafak or other message queues for consuming.
As a golang client of Canal,it use Socket for communication,the transfer Protocol is TCP,and the data exchange Protocol is Google's Protocol Buffer 3.0
-
Canal connect to Mysql database ,pretented to be a slave of mysql
-
canal-go(as mentioned it is a client of Cancal) connect to Canal
-
When data in the database changed,it write the changes to binlog
-
Canal send dump request to mysql database,get binlog and then parse it.
-
canal-go send request to Canal for data.
-
Canal send parsed data to canal-go
-
canal-go get data successfully and then send an acknowledgement to Canal(optional)
-
Canal record the consumption position
The following picture shows how it works:
To Install and configure Canal,please visit https://github.com/alibaba/canal/wiki/QuickStart for information
go get github.com/CanalSharp/canal-goor you can install it from nuget manager(a graphic ui) in visual studio
connector := client.NewSimpleCanalConnector("192.168.199.17", 11111, "", "", "example", 60000, 60*60*1000)
connector.Connect()
connector.Subscribe(".*\\\\..*")
for {
message := connector.Get(100, nil, nil)
batchId := message.Id
if batchId == -1 || len(message.Entries) <= 0 {
time.Sleep(300 * time.Millisecond)
fmt.Println("===没有数据了===")
continue
}
printEntry(message.Entries)
}for more detailed information please visit Sample
git clone https://github.com/CanalClient/canal-go.git
cd canal-go
cd docker
docker-compose up -dip：the ip address of the server where docker running
mysql username：root
mysql password：000000
mysql port：4406
There is a default database named test,which contains a table named test
Execute the following sql:
insert into test values(1000,'111');
update test set name='222' where id=1000;
delete from test where id=1000;we can see that after executing insrt,update,delete sql,Our canal-go get the changed data.
canal-go clustering support
We gladly accept community contributions.
1.fork the canal-go Project
2.make changes to it
3.make a pull request
Please do not hesitate to make a pull request,your effort will not in vain.