8000 DML construct error and changefeed stop caused by 'rename tables' · Issue #5059 · pingcap/tiflow · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

DML construct error and changefeed stop caused by 'rename tables' #5059

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
maxshuang opened this issue Mar 29, 2022 · 0 comments · Fixed by #5068
Closed

DML construct error and changefeed stop caused by 'rename tables' #5059

maxshuang opened this issue Mar 29, 2022 · 0 comments · Fixed by #5068
Assignees
@maxshuang
Labels
affects-5.4 This bug affects the 5.4.x(LTS) versions. affects-6.0 area/ticdc Issues or PRs related to TiCDC. severity/major type/bug The issue is confirmed as a bug.

Comments

@maxshuang
Copy link
Contributor

What did you do?

  1. Setup upstream and downstream cluster
  2. Test tidb-mysql-test bigint test case, need change all drop table to rename table
  3. check data consistency

What did you expect to see?

changefeed work normal

What did you see instead?

Error DML rewrite and cause changefeed stop by error.
It may cause by incorrect rename table implemement.

sql sequence:

create table t1 (
 value64  bigint unsigned  not null,
 value32  integer          not null,
 primary key(value64, value32)
);

create table t2 (
 value64  bigint unsigned  not null,
 value32  integer          not null,
 primary key(value64, value32)
);

insert into t1 values(17156792991891826145, 1);
insert into t1 values( 9223372036854775807, 2);
insert into t2 values(17156792991891826145, 3);
insert into t2 values( 9223372036854775807, 4);

select * from t1;
select * from t2;

select * from t1, t2 where t1.value64=17156792991891826145 and
t2.value64=17156792991891826145;
select * from t1, t2 where t1.value64=17156792991891826145 and
t2.value64=t1.value64;

select * from t1, t2 where t1.value64= 9223372036854775807 and
t2.value64=9223372036854775807;
select * from t1, t2 where t1.value64= 9223372036854775807 and
t2.value64=t1.value64;

rename table t1 to t1_7, t2 to t2_7;

log:

[2022/03/29 06:33:43.930 +00:00] [DEBUG] [mysql.go:768] ["prepare DMLs"] [rows="[{\"start-ts\":432153712015441934,\"commit-ts\":432153712015441938,\"row-id\":1,\"table\":{\"db-name\":\"bigint\",\"tbl-name\":\"t2_7\",\"tbl-id\":84,\"is-partition\":false},\"column-infos\":[{\"ID\":1,\"IsPKHandle\":false,\"VirtualGenCol\":false,\"Ft\":{\"Tp\":8,\"Flag\":4131,\"Flen\":20,\"Decimal\":0,\"Charset\":\"binary\",\"Collate\":\"binary\",\"Elems\":null}},{\"ID\":2,\"IsPKHandle\":false,\"VirtualGenCol\":false,\"Ft\":{\"Tp\":3,\"Flag\":4099,\"Flen\":11,\"Decimal\":0,\"Charset\":\"binary\",\"Collate\":\"binary\",\"Elems\":null}}],\"table-info-version\":432153712028549131,\"replica-id\":0,\"columns\":[{\"name\":\"value64\",\"type\":8,\"charset\":\"binary\",\"flag\":187,\"value\":17156792991891826145},{\"name\":\"value32\",\"type\":3,\"charset\":\"binary\",\"flag\":59,\"value\":3}],\"pre-columns\":null}]"] [sqls="[\"REPLACE INTO `bigint`.`t2_7`(`value64`,`value32`) VALUES (?,?)\"]"] [values="[[17156792991891826145,3]]"]
[2022/03/29 06:33:43.932 +00:00] [DEBUG] [mysql.go:614] ["exec row"] [sql="REPLACE INTO `bigint`.`t2_7`(`value64`,`value32`) VALUES (?,?)"] [args="[17156792991891826145,3]"]
[2022/03/29 06:33:44.293 +00:00] [ERROR] [feed_state_manager.go:344] ["processor report an error"] [changefeed=tidb-mysql-test] [captureID=87ffe707-65f7-4fae-a0c8-67c545fbe9bd] [error="{\"addr\":\"127.0        .0.1:8300\",\"code\":\"CDC:ErrProcessorUnknown\",\"message\":\"[CDC:ErrMySQLTxnError]MySQL txn error: Error 1146: Table 'bigint.t2_7' doesn't exist\"}"]

cdc 5.log

Versions of the cluster

Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

Release Version: v6.1.0-alpha-77-gcd0d6b85a
Edition: Community
Git Commit Hash: cd0d6b85a92ad2ee5da735563603b1588c1d1463
Git Branch: master
UTC Build Time: 2022-03-28 11:12:19
GoVersion: go1.18
Race Enabled: false

Upstream TiKV version (execute tikv-server --version):

Release Version:   6.0.0-alpha
Edition:           Community
Git Commit Hash:   4f3e720e1ec2e4e2f8d1ce2c0151a0879b01b885
Git Commit Branch: master
UTC Build Time:    2022-03-28 11:09:04
Rust Version:      rustc 1.60.0-nightly (1e12aef3f 2022-02-13)
Enable Features:   jemalloc mem-profiling portable sse test-engines-rocksdb cloud-aws cloud-gcp cloud-azure
Profile:           dist_release

TiCDC version (execute cdc version):

Release Version: v6.0.0-master-dirty
@maxshuang maxshuang added type/bug The issue is confirmed as a bug. area/ticdc Issues or PRs related to TiCDC. severity/major labels Mar 29, 2022
@maxshuang maxshuang changed the title Error DDL/DML schedule， which cause error DML rewrite DML rewrite error and changefeed stop caused by 'rename table' Mar 29, 2022
@maxshuang maxshuang changed the title DML rewrite error and changefeed stop caused by 'rename table' DML construct error and changefeed stop caused by 'rename table' Mar 29, 2022
@maxshuang maxshuang mentioned this issue Mar 29, 2022
Open
57 tasks
@maxshuang maxshuang self-assigned this Mar 29, 2022
@maxshuang maxshuang changed the title DML construct error and changefeed stop caused by 'rename table' DML construct error and changefeed stop caused by 'rename tables' Mar 30, 2022
@maxshuang maxshuang mentioned this issue Mar 30, 2022
Merged
overvenus added a commit that referenced this issue Mar 30, 2022
@hicqu @overvenus @ti-chi-bot
schema(cdc): fix DML construct error caused by 'rename tables' DDL (#…
a9d5d5a 
…5068)

* fix(schema): fix DML construct error caused by 'rename tables' DDL

close  #5059

Co-authored-by: qupeng <qupeng@pingcap.com>
Co-authored-by: Neil Shen <overvenus@gmail.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
@ti-chi-bot ti-chi-bot mentioned this issue Mar 30, 2022
Merged
ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this issue Mar 30, 2022
@ti-chi-bot
fix(schema): fix DML construct error caused by 'rename tables' DDL
d0e85d5 
close  pingcap#5059
@ti-chi-bot ti-chi-bot mentioned this issue Mar 30, 2022
Merged
overvenus added a commit that referenced this issue Mar 31, 2022
@ti-chi-bot @overvenus
schema(cdc): fix DML construct error caused by 'rename tables' DDL (#…
2f363f6 
…5068) (#5080)

* fix(schema): fix DML construct error caused by 'rename tables' DDL

close  #5059

Co-authored-by: maxshuang <huangguohao@pingcap.com>
Co-authored-by: Neil Shen <overvenus@gmail.com>
This was referenced May 7, 2022
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@maxshuang maxshuang

Labels
affects-5.4 This bug affects the 5.4.x(LTS) versions. affects-6.0 area/ticdc Issues or PRs related to TiCDC. severity/major type/bug The issue is confirmed as a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

schema(cdc): fix DML construct error caused by 'rename tables' DDL
2 participants
@maxshuang @ti-chi-bot
0