feat(UTXO): balance event streaming for Electrum clients #2013

onur-ozkan · 2023-11-23T20:22:33Z

This PR aims to:

Implement balance events for UTXOs.
Fix electrum notification receiving bug.
Support broadcasting multiple balance events.
Implement subscribing to electrum scripthash.

Can be tested in a same way of #1978 PR.

Signed-off-by: onur-ozkan <work@onurozkan.dev>

mm2src/coins/utxo/utxo_balance_events.rs

shamardy

Thank you for the PR! Only one note at this stage of the review.

mm2src/coins/utxo/utxo_balance_events.rs

artemii235

I also have few notes 🙂

mm2src/coins/utxo/utxo_balance_events.rs

mm2src/mm2_core/src/mm_ctx.rs

Signed-off-by: onur-ozkan <work@onurozkan.dev>

artemii235

One more note.

mm2src/coins/utxo/utxo_balance_events.rs

artemii235

Next iteration.

mm2src/coins/utxo/utxo_balance_events.rs

artemii235 · 2023-12-01T07:51:30Z

mm2src/coins/utxo/utxo_balance_events.rs

+                    log::error!("{e}");
+                    continue;


It's worth triggering an event here to notify GUI/user that balance is likely changed, but its value is unknown at this point due to error. They will need to request it using RPC.

True, however, I think GUI needs to tell what kind of error(like the format of error structure) they want to handle from event streaming. Right now GUI not using this feature so it's not clear what kind of errors we should send. I will leave a TODO here.

mm2src/coins/utxo/utxo_balance_events.rs

Signed-off-by: onur-ozkan <work@onurozkan.dev>

artemii235

Looks great, thanks for fixes! 🙂

shamardy

Thank you for the fixes! Next review iteration!

mm2src/coins/utxo/utxo_common.rs

mm2src/coins/rpc_command/init_scan_for_new_addresses.rs

mm2src/coins/utxo.rs

mm2src/coins/utxo/utxo_balance_events.rs

mm2src/coins/utxo/rpc_clients.rs

Signed-off-by: onur-ozkan <work@onurozkan.dev>

…alanceOps` trait Signed-off-by: onur-ozkan <work@onurozkan.dev>

Signed-off-by: onur-ozkan <work@onurozkan.dev>

shamardy

🔥

smk762 · 2023-12-09T16:18:18Z

Working here in CLI testing, though it seems every event is logged to the page twice

onur-ozkan · 2023-12-09T16:48:13Z

Working here in CLI testing, though it seems every event is logged to the page twice

Just checked, it's the electrum servers sending notification twice. It shouldn't cause any problem anyway I think.

ca333

secure code reviewed

onur-ozkan · 2023-12-21T07:56:08Z

I need this to be merged for fixing UTXO TODOs related to error events. @shamardy

* dev: (22 commits) chore(config): remove vscode launchjson (KomodoPlatform#2040) feat(trading-proto-upgrade): wasm DB, kickstart, refund states, v2 RPCs (KomodoPlatform#2015) feat(UTXO): balance event streaming for Electrum clients (KomodoPlatform#2013) feat(tx): add new sign_raw_transaction rpc for UTXO and EVM coins (KomodoPlatform#1930) fix(p2p): handle encode_and_sign errors (KomodoPlatform#2038) chore(release): add changelog entries for v2.0.0-beta (KomodoPlatform#2037) chore(network): write network information to stdout (KomodoPlatform#2034) fix(price_endpoints): add cached url (KomodoPlatform#2032) deps(network): sync with upstream yamux (KomodoPlatform#2030) fix(config): accept a string as rpcport value (KomodoPlatform#2026) feat(nft): move db lock, add tx fee and confirmations (KomodoPlatform#1989) chore(network): update seednodes for netid 8762 (KomodoPlatform#2024) chore(network): add todo on peer storage behaviour (KomodoPlatform#2025) chore(network): exclude `168.119.236.249` from the seednode list (KomodoPlatform#2021) feat(network): deprecate 7777 network (KomodoPlatform#2020) chore(release): bump mm2 version to 2.0.0-beta (KomodoPlatform#2018) feat(UTXO swaps): kmd burn plan impl (KomodoPlatform#2006) chore(docs): fix the link to simple market maker in README.md (KomodoPlatform#2011) refactor(cli): cli dependency updates and warn on bad config perm (KomodoPlatform#1956) chore(containers and docs): update docs and container images (KomodoPlatform#2003) ... # Conflicts: # mm2src/mm2_main/tests/mm2_tests/mm2_tests_inner.rs # mm2src/mm2_test_helpers/src/for_tests.rs

* dev: (24 commits) chore(release): bump mm2 version to 2.1.0-beta (KomodoPlatform#2044) feat(trezor): add segwit support for withdraw with trezor (KomodoPlatform#1984) chore(config): remove vscode launchjson (KomodoPlatform#2040) feat(trading-proto-upgrade): wasm DB, kickstart, refund states, v2 RPCs (KomodoPlatform#2015) feat(UTXO): balance event streaming for Electrum clients (KomodoPlatform#2013) feat(tx): add new sign_raw_transaction rpc for UTXO and EVM coins (KomodoPlatform#1930) fix(p2p): handle encode_and_sign errors (KomodoPlatform#2038) chore(release): add changelog entries for v2.0.0-beta (KomodoPlatform#2037) chore(network): write network information to stdout (KomodoPlatform#2034) fix(price_endpoints): add cached url (KomodoPlatform#2032) deps(network): sync with upstream yamux (KomodoPlatform#2030) fix(config): accept a string as rpcport value (KomodoPlatform#2026) feat(nft): move db lock, add tx fee and confirmations (KomodoPlatform#1989) chore(network): update seednodes for netid 8762 (KomodoPlatform#2024) chore(network): add todo on peer storage behaviour (KomodoPlatform#2025) chore(network): exclude `168.119.236.249` from the seednode list (KomodoPlatform#2021) feat(network): deprecate 7777 network (KomodoPlatform#2020) chore(release): bump mm2 version to 2.0.0-beta (KomodoPlatform#2018) feat(UTXO swaps): kmd burn plan impl (KomodoPlatform#2006) chore(docs): fix the link to simple market maker in README.md (KomodoPlatform#2011) ...

* evm-hd-wallet: (27 commits) Fix todo comments Fix HDAddressOps::Address trait bounds fix(indexeddb): fix IDB cursor.continue_() call after drop (KomodoPlatform#2028) security bump for `h2` (KomodoPlatform#2062) fix(makerbot): allow more than one prices url in makerbot (KomodoPlatform#2027) fix(wasm worker env): refactor direct usage of `window` (KomodoPlatform#1953) feat(nft): nft abi in withdraw_nft RPC, clear_nft_db RPC (KomodoPlatform#2039) refactor(utxo): refactor utxo output script creation (KomodoPlatform#1960) feat(ETH): balance event streaming for ETH (KomodoPlatform#2041) chore(release): bump mm2 version to 2.1.0-beta (KomodoPlatform#2044) feat(trezor): add segwit support for withdraw with trezor (KomodoPlatform#1984) chore(config): remove vscode launchjson (KomodoPlatform#2040) feat(trading-proto-upgrade): wasm DB, kickstart, refund states, v2 RPCs (KomodoPlatform#2015) feat(UTXO): balance event streaming for Electrum clients (KomodoPlatform#2013) feat(tx): add new sign_raw_transaction rpc for UTXO and EVM coins (KomodoPlatform#1930) fix(p2p): handle encode_and_sign errors (KomodoPlatform#2038) chore(release): add changelog entries for v2.0.0-beta (KomodoPlatform#2037) chore(network): write network information to stdout (KomodoPlatform#2034) fix(price_endpoints): add cached url (KomodoPlatform#2032) deps(network): sync with upstream yamux (KomodoPlatform#2030) ...

onur-ozkan added 2 commits November 23, 2023 18:50

[WIP] save development state

18dc21b

Signed-off-by: onur-ozkan <work@onurozkan.dev>

implement scripthash channels between electrum and utxo coin

58f35c5

Signed-off-by: onur-ozkan <work@onurozkan.dev>

onur-ozkan added enhancement New feature or request in progress Changes will be made from the author labels Nov 23, 2023

subscribe to scripthashes

6ab27b6

Signed-off-by: onur-ozkan <work@onurozkan.dev>

onur-ozkan changed the title ~~feat(UTXO): balance event streaming for Electrum and Native clients~~ feat(UTXO): balance event streaming for Electrum clients Nov 24, 2023

onur-ozkan added 4 commits November 24, 2023 23:06

[wip] fetch balances when scripthash triggered

5866d3e

Signed-off-by: onur-ozkan <work@onurozkan.dev>

[wip] broadcast balances when they change

7f9cb9b

Signed-off-by: onur-ozkan <work@onurozkan.dev>

proper balance event errors on UTXO activations

1abd13f

Signed-off-by: onur-ozkan <work@onurozkan.dev>

create ScripthashNotification types

88894e7

Signed-off-by: onur-ozkan <work@onurozkan.dev>

onur-ozkan force-pushed the utxo-balance-stream-event branch from 0c425e7 to 88894e7 Compare November 27, 2023 08:46

onur-ozkan added 7 commits November 27, 2023 11:53

add doc-comment for scripthash_notification_sender

fb864df

Signed-off-by: onur-ozkan <work@onurozkan.dev>

avoid unwrap in fn scripthash_notification_sender

4d00657

Signed-off-by: onur-ozkan <work@onurozkan.dev>

do pass by ref for ScripthashNotificationSender

4831076

Signed-off-by: onur-ozkan <work@onurozkan.dev>

impl get_scripthash_notification_handlers for MmArc

a4563f8

Signed-off-by: onur-ozkan <work@onurozkan.dev>

fix WASM functions

6ab47b2

Signed-off-by: onur-ozkan <work@onurozkan.dev>

broadcast multiple balances at once

4bac4fa

Signed-off-by: onur-ozkan <work@onurozkan.dev>

move SubscriptionNotification to top in ElectrumRpcResponseEnum

37e8a8d

Signed-off-by: onur-ozkan <work@onurozkan.dev>

onur-ozkan added under review and removed in progress Changes will be made from the author labels Nov 27, 2023

onur-ozkan marked this pull request as ready for review November 27, 2023 11:54

onur-ozkan self-assigned this Nov 27, 2023

onur-ozkan commented Nov 27, 2023

View reviewed changes

mm2src/coins/utxo/utxo_balance_events.rs Outdated Show resolved Hide resolved

shamardy self-requested a review November 27, 2023 12:54

shamardy mentioned this pull request Nov 28, 2023

[bug] HD Wallet implementation #1838

Open

shamardy reviewed Nov 28, 2023

View reviewed changes

mm2src/coins/utxo/utxo_balance_events.rs Outdated Show resolved Hide resolved

artemii235 reviewed Nov 29, 2023

View reviewed changes

fix review notes

98d6553

Signed-off-by: onur-ozkan <work@onurozkan.dev>

artemii235 reviewed Nov 30, 2023

View reviewed changes

mm2src/coins/utxo/utxo_balance_events.rs Outdated Show resolved Hide resolved

onur-ozkan removed the under review label Nov 30, 2023

artemii235 requested changes Dec 1, 2023

View reviewed changes

handle subscriptions from rpc_client connections

681f039

Signed-off-by: onur-ozkan <work@onurozkan.dev>

artemii235 previously approved these changes Dec 4, 2023

View reviewed changes

shamardy reviewed Dec 4, 2023

View reviewed changes

create utxo common function address_to_scripthash

2c3d03c

Signed-off-by: onur-ozkan <work@onurozkan.dev>

onur-ozkan dismissed artemii235’s stale review via 2c3d03c December 5, 2023 07:49

onur-ozkan added 2 commits December 5, 2023 11:58

add prepare_addresses_for_balance_stream_if_enabled into `HDWalletB…

cc3dfac

…alanceOps` trait Signed-off-by: onur-ozkan <work@onurozkan.dev>

update utxo_prepare_addresses_for_balance_stream_if_enabled

8357033

Signed-off-by: onur-ozkan <work@onurozkan.dev>

shamardy mentioned this pull request Dec 5, 2023

feat(utxo): prioritize electrum connections #1966

Open

send scripthash event instead of calling rpcs

a290522

Signed-off-by: onur-ozkan <work@onurozkan.dev>

onur-ozkan force-pushed the utxo-balance-stream-event branch from 9b611cf to a290522 Compare December 5, 2023 12:43

shamardy approved these changes Dec 5, 2023

View reviewed changes

shamardy added 2.0.0-beta QA sec-review no-deps 2.1.0-beta and removed 2.0.0-beta labels Dec 5, 2023

ca333 approved these changes Dec 12, 2023

View reviewed changes

ca333 requested a review from DeckerSU December 12, 2023 06:49

shamardy merged commit 21ca5cc into dev Dec 21, 2023
29 of 36 checks passed

shamardy deleted the utxo-balance-stream-event branch December 21, 2023 11:47

harsenyan3 mentioned this pull request Jul 16, 2024

Add "balance event streaming" info to docs KomodoPlatform/komodo-docs-mdx#278

Closed

shamardy mentioned this pull request Aug 1, 2024

chore(release): v2.1.0-beta #2043

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(UTXO): balance event streaming for Electrum clients #2013

feat(UTXO): balance event streaming for Electrum clients #2013

onur-ozkan commented Nov 23, 2023 •

edited

Loading

shamardy left a comment

artemii235 left a comment

artemii235 left a comment

artemii235 left a comment

artemii235 Dec 1, 2023

onur-ozkan Dec 1, 2023

artemii235 left a comment

shamardy left a comment

shamardy left a comment

smk762 commented Dec 9, 2023

onur-ozkan commented Dec 9, 2023

ca333 left a comment

onur-ozkan commented Dec 21, 2023

feat(UTXO): balance event streaming for Electrum clients #2013

feat(UTXO): balance event streaming for Electrum clients #2013

Conversation

onur-ozkan commented Nov 23, 2023 • edited Loading

shamardy left a comment

Choose a reason for hiding this comment

artemii235 left a comment

Choose a reason for hiding this comment

artemii235 left a comment

Choose a reason for hiding this comment

artemii235 left a comment

Choose a reason for hiding this comment

artemii235 Dec 1, 2023

Choose a reason for hiding this comment

onur-ozkan Dec 1, 2023

Choose a reason for hiding this comment

artemii235 left a comment

Choose a reason for hiding this comment

shamardy left a comment

Choose a reason for hiding this comment

shamardy left a comment

Choose a reason for hiding this comment

smk762 commented Dec 9, 2023

onur-ozkan commented Dec 9, 2023

ca333 left a comment

Choose a reason for hiding this comment

onur-ozkan commented Dec 21, 2023

onur-ozkan commented Nov 23, 2023 •

edited

Loading