We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CCR: 3cb46f9 Doris:apache/doris@be11b48
job error and stop sync data
error message:ERROR need rollback, err: normal: partitionId: 20050 not found github.com/selectdb/ccr_syncer/xerror.Errorf /root/ccr-syncer/xerror/xerror.go:123 github.com/selectdb/ccr_syncer/ccr.(*Meta).UpdateIndexes /root/ccr-syncer/ccr/meta.go:580 github.com/selectdb/ccr_syncer/ccr.(*Meta).getIndexes /root/ccr-syncer/ccr/meta.go:654 github.com/selectdb/ccr_syncer/ccr.(*Meta).GetIndexIdMap /root/ccr-syncer/ccr/meta.go:665 github.com/selectdb/ccr_syncer/ccr.(*IngestBinlogJob).preparePartition /root/ccr-syncer/ccr/ingest_binlog_job.go:355 github.com/selectdb/ccr_syncer/ccr.(*IngestBinlogJob).prepareTable /root/ccr-syncer/ccr/ingest_binlog_job.go:467 github.com/selectdb/ccr_syncer/ccr.(*IngestBinlogJob).prepareTabletIngestJobs /root/ccr-syncer/ccr/ingest_binlog_job.go:495 github.com/selectdb/ccr_syncer/ccr.(*IngestBinlogJob).Run /root/ccr-syncer/ccr/ingest_binlog_job.go:522 github.com/selectdb/ccr_syncer/ccr.(*Job).ingestBinlog /root/ccr-syncer/ccr/job.go:593 github.com/selectdb/ccr_syncer/ccr.(*Job).handleUpsert /root/ccr-syncer/ccr/job.go:718 github.com/selectdb/ccr_syncer/ccr.(*Job).handleUpsert /root/ccr-syncer/ccr/job.go:807 github.com/selectdb/ccr_syncer/ccr.(*Job).handleUpsert /root/ccr-syncer/ccr/job.go:807 github.com/selectdb/ccr_syncer/ccr.(*Job).handleBinlog /root/ccr-syncer/ccr/job.go:1078 github.com/selectdb/ccr_syncer/ccr.(*Job).handleBinlogs /root/ccr-syncer/ccr/job.go:1030 github.com/selectdb/ccr_syncer/ccr.(*Job).incrementalSync /root/ccr-syncer/ccr/job.go:1170 github.com/selectdb/ccr_syncer/ccr.(*Job).dbSync /root/ccr-syncer/ccr/job.go:1229 github.com/selectdb/ccr_syncer/ccr.(*Job).sync /root/ccr-syncer/ccr/job.go:1243 github.com/selectdb/ccr_syncer/ccr.(*Job).run /root/ccr-syncer/ccr/job.go:1292 github.com/selectdb/ccr_syncer/ccr.(*Job).Run /root/ccr-syncer/ccr/job.go:1358 github.com/selectdb/ccr_syncer/ccr.(*JobManager).runJob.func1 /root/ccr-syncer/ccr/job_manager.go:156 runtime.goexit /usr/lib/golang/src/runtime/asm_amd64.s:1594
step 1: TRUNCATE TABLE step 2: DELETE from table WHERE where_cond stetp 3: insert into table
i check the ccr log,and found step 1 and step 2 are correct sync. logs: [2023-10-13 09:42:18] INFO handle truncate table binlog job=ccr_test [2023-10-13 09:42:18] INFO truncateTableSql: TRUNCATE TABLE z_mid_po_gr_ekbe job=ccr_test [2023-10-13 09:42:18] INFO table z_mid_po_gr_ekbe not found, skip clear job=ccr_test [2023-10-13 09:42:18] INFO table z_mid_po_gr_ekbe not found, skip clear job=ccr_test [2023-10-13 09:43:00] INFO handle upsert binlog, sub sync state: Done job=ccr_test [2023-10-13 09:43:00] INFO handle upsert binlog, sub sync state: BeginTransaction job=ccr_test [2023-10-13 09:43:00] INFO handle upsert binlog, sub sync state: IngestBinlog job=ccr_test [2023-10-13 09:43:00] INFO ingestBinlog, txnId: 3008 job=ccr_test [2023-10-13 09:43:00] INFO handle upsert binlog, sub sync state: CommitTransaction job=ccr_test [2023-10-13 09:43:00] INFO commit TxnId: 3008 resp: TCommitTxnResult_({Status:TStatus({StatusCode:OK ErrorMsgs:[]})}) job=ccr_test
The text was updated successfully, but these errors were encountered:
pls use the latest ccr version
Sorry, something went wrong.
No branches or pull requests
Search before asking
Version
CCR: 3cb46f9
Doris:apache/doris@be11b48
What's Wrong?
job error and stop sync data
error message:ERROR need rollback, err: normal: partitionId: 20050 not found
github.com/selectdb/ccr_syncer/xerror.Errorf
/root/ccr-syncer/xerror/xerror.go:123
github.com/selectdb/ccr_syncer/ccr.(*Meta).UpdateIndexes
/root/ccr-syncer/ccr/meta.go:580
github.com/selectdb/ccr_syncer/ccr.(*Meta).getIndexes
/root/ccr-syncer/ccr/meta.go:654
github.com/selectdb/ccr_syncer/ccr.(*Meta).GetIndexIdMap
/root/ccr-syncer/ccr/meta.go:665
github.com/selectdb/ccr_syncer/ccr.(*IngestBinlogJob).preparePartition
/root/ccr-syncer/ccr/ingest_binlog_job.go:355
github.com/selectdb/ccr_syncer/ccr.(*IngestBinlogJob).prepareTable
/root/ccr-syncer/ccr/ingest_binlog_job.go:467
github.com/selectdb/ccr_syncer/ccr.(*IngestBinlogJob).prepareTabletIngestJobs
/root/ccr-syncer/ccr/ingest_binlog_job.go:495
github.com/selectdb/ccr_syncer/ccr.(*IngestBinlogJob).Run
/root/ccr-syncer/ccr/ingest_binlog_job.go:522
github.com/selectdb/ccr_syncer/ccr.(*Job).ingestBinlog
/root/ccr-syncer/ccr/job.go:593
github.com/selectdb/ccr_syncer/ccr.(*Job).handleUpsert
/root/ccr-syncer/ccr/job.go:718
github.com/selectdb/ccr_syncer/ccr.(*Job).handleUpsert
/root/ccr-syncer/ccr/job.go:807
github.com/selectdb/ccr_syncer/ccr.(*Job).handleUpsert
/root/ccr-syncer/ccr/job.go:807
github.com/selectdb/ccr_syncer/ccr.(*Job).handleBinlog
/root/ccr-syncer/ccr/job.go:1078
github.com/selectdb/ccr_syncer/ccr.(*Job).handleBinlogs
/root/ccr-syncer/ccr/job.go:1030
github.com/selectdb/ccr_syncer/ccr.(*Job).incrementalSync
/root/ccr-syncer/ccr/job.go:1170
github.com/selectdb/ccr_syncer/ccr.(*Job).dbSync
/root/ccr-syncer/ccr/job.go:1229
github.com/selectdb/ccr_syncer/ccr.(*Job).sync
/root/ccr-syncer/ccr/job.go:1243
github.com/selectdb/ccr_syncer/ccr.(*Job).run
/root/ccr-syncer/ccr/job.go:1292
github.com/selectdb/ccr_syncer/ccr.(*Job).Run
/root/ccr-syncer/ccr/job.go:1358
github.com/selectdb/ccr_syncer/ccr.(*JobManager).runJob.func1
/root/ccr-syncer/ccr/job_manager.go:156
runtime.goexit
/usr/lib/golang/src/runtime/asm_amd64.s:1594
How to Reproduce?
step 1: TRUNCATE TABLE
step 2: DELETE from table WHERE where_cond
stetp 3: insert into table
Anything Else?
i check the ccr log,and found step 1 and step 2 are correct sync.
logs:
[2023-10-13 09:42:18] INFO handle truncate table binlog job=ccr_test
[2023-10-13 09:42:18] INFO truncateTableSql: TRUNCATE TABLE z_mid_po_gr_ekbe job=ccr_test
[2023-10-13 09:42:18] INFO table z_mid_po_gr_ekbe not found, skip clear job=ccr_test
[2023-10-13 09:42:18] INFO table z_mid_po_gr_ekbe not found, skip clear job=ccr_test
[2023-10-13 09:43:00] INFO handle upsert binlog, sub sync state: Done job=ccr_test
[2023-10-13 09:43:00] INFO handle upsert binlog, sub sync state: BeginTransaction job=ccr_test
[2023-10-13 09:43:00] INFO handle upsert binlog, sub sync state: IngestBinlog job=ccr_test
[2023-10-13 09:43:00] INFO ingestBinlog, txnId: 3008 job=ccr_test
[2023-10-13 09:43:00] INFO handle upsert binlog, sub sync state: CommitTransaction job=ccr_test
[2023-10-13 09:43:00] INFO commit TxnId: 3008 resp: TCommitTxnResult_({Status:TStatus({StatusCode:OK ErrorMsgs:[]})}) job=ccr_test
The text was updated successfully, but these errors were encountered: