[Enhancement] Try to cleanup lingering transactions when restoring in exactly-once mode #271
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
7ebbde1 to
fd46059
Compare
|
This PREPARED transaction is because the import encountered an exception. So when the import encountered an exception,Should the PREPARED transaction be closed and then restarted ? Each task needs to be set with sink.label-prefix, which makes it easy to repeat |
|
@dyp12 Thanks for your comments
|
Signed-off-by: PengFei Li <lpengfei2016@gmail.com>
Signed-off-by: PengFei Li <lpengfei2016@gmail.com>
Signed-off-by: PengFei Li <lpengfei2016@gmail.com>
Signed-off-by: PengFei Li <lpengfei2016@gmail.com>
fd46059 to
e85f108
Compare
Signed-off-by: PengFei Li <lpengfei2016@gmail.com>
xlfjcg
approved these changes
Aug 31, 2023
banmoy
added a commit
to banmoy/starrocks-connector-for-apache-flink
that referenced
this pull request
Sep 2, 2023
… exactly-once mode (StarRocks#271) Signed-off-by: PengFei Li <lpengfei2016@gmail.com>
banmoy
added a commit
to banmoy/starrocks-connector-for-apache-flink
that referenced
this pull request
Sep 2, 2023
… exactly-once mode (StarRocks#271) Signed-off-by: PengFei Li <lpengfei2016@gmail.com>
banmoy
added a commit
that referenced
this pull request
Sep 11, 2023
… exactly-once mode (#271) Signed-off-by: PengFei Li <lpengfei2016@gmail.com>
banmoy
added a commit
that referenced
this pull request
Sep 11, 2023
… exactly-once mode (#271) Signed-off-by: PengFei Li <lpengfei2016@gmail.com>
|
For "Some of those PREPARED transactions may be in a successful checkpoint, and will be committed when the job restores from the checkpoint", if fe restore when this txn is PREPARED, the callback instance in fe will lose, and this txn will never be commited, but is also cannot be rollback by connector because of exactly-once. how to solve this problem? is this a bug in sr or connector? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What type of PR is this：
Which issues of this PR fixes ：
Fixes #
Problem Summary(Required) ：
What's the problem
When using exactly-once, the connector will not abort the
PREPAREDtransactions when the flink job failovers or exits because it's 2PC mechanism. Some of those PREPARED transactions may be in a successful checkpoint, and will be committed when the job restores from the checkpoint, but some of them are just useless, and should be aborted, otherwise they will be lingering in StarRocks until timeout which maybe make StarRocks unstable. We should try to abort those lingering transactions when restoringHow to solve it
When flink job restores, the connector will try to find those lingering transactions, and abort them. The key is how to find those transactions because the labels of them are not stored in the checkpoint. Here we design a label generator (ExactlyOnceLabelGenerator) to solve it
sink.label-prefixwhich is used as the prefix of the labels, and it must be unique across all the ingestions, including flink connector, broker load and routine load, running on the same StarRocks cluster{labelPrefix}-{tableName}-{subtaskIndex}-{id}.subtaskIndexwill make the label unique across subtasks if the sink writes parallelidis incremental, and it will make the label unique across different transactions in a subtaskPREPAREDstate, and should abort itChecklist: