feat: support separate trie database #2021

flywukong · 2023-12-01T03:26:33Z

Description

This PR introduces the functionality to run node with a separate trie database on a distinct disk or directory. This proposal aims to alleviate the bandwidth strain on the current key-value database, while also streamlining and simplifying the log-structured merge (LSM) of the database. These enhancements are anticipated to improve the read performance of the database and subsequently enhance the performance of geth.

Specifically, the PR includes the following features:

Support for running geth with a separate trie database. Users can set the --separatetrie flag to enable a separate database for storing trie data. Both hash-based and path-based trie databases are supported.
In the event of running with PBSS, the ancient state directory is also stored in the separate trie directory.
If users intend to utilize this feature for running a fullnode or validator, they must either run --separatetrie from genesis or make use of the provided snapshot containing the separate trie database.
The separate trie db use the subdirectory of /geth/chaindata which called "state". User can move the state subdirectory to any disk or directory and make a soft link called "state" under the /geth/chaindata

The inspected data below pertains to the execution of a fullnode in PBSS+Pebble mode across two separate disks.

This one shows the separate trie db

This one the db without trie data and ancient state

Rationale

Blow is the test result using different disk resource for trie.

avg import cost geth with single db is 324 ms while the cost of geth with seprated db is 291 ms.
The performance using seprated db has improve 10% (95 percent )

Example

// init geth with the separate trie db
./geth --datadir node  --state.scheme path --db.engine pebble --triedir trie init genesis.json

// run geth node with the separate  trie db
./geth --config config.toml --datadir node --triedir trie --cache 10000 --rpc.allow-unprotected-txs --syncmode full --state.scheme path

Changes

Notable changes:

N/A

cmd/geth/dbcmd.go

core/blockchain.go

miner/stress/clique/main.go

cmd/utils/flags.go

cmd/geth/chaincmd.go

cmd/geth/dbcmd.go

core/blockchain.go

core/rawdb/database.go

sysvm · 2024-01-31T09:37:41Z

Can users merge separate trie db?

flywukong · 2024-01-31T09:53:24Z

Can users merge separate trie db?

not needed for now

cmd/geth/snapshot.go

cmd/utils/flags.go

core/blockchain.go

cmd/geth/dbcmd.go

core/rawdb/accessors_trie.go

fynnss · 2024-03-05T11:48:01Z

core/rawdb/database.go

@@ -659,14 +690,14 @@ func InspectDatabase(db ethdb.Database, keyPrefix, keyStart []byte) error {
 			bodies.Add(size)
 		case bytes.HasPrefix(key, blockReceiptsPrefix) && len(key) == (len(blockReceiptsPrefix)+8+common.HashLength):
 			receipts.Add(size)
+		case IsLegacyTrieNode(key, it.Value()):


Why move it from 668-669 to 693-694?

it is a known issue of inspect db , the precondition of the judgment conditions is used to avoid identifying some header metadata as hash trie data.

fynnss

LGTM

fynnss · 2024-03-06T11:15:41Z

core/rawdb/ancient_utils.go

@@ -91,7 +91,8 @@ func inspectFreezers(db ethdb.Database) ([]freezerInfo, error) {
 			infos = append(infos, info)

 		case StateFreezerName:
-			if ReadStateScheme(db) != PathScheme {
+			if
+			ReadStateScheme(db) != PathScheme {


Please fix the lint error.

fynnss · 2024-03-06T11:16:12Z

core/rawdb/database.go

@@ -659,14 +690,14 @@ func InspectDatabase(db ethdb.Database, keyPrefix, keyStart []byte) error {
 			bodies.Add(size)
 		case bytes.HasPrefix(key, blockReceiptsPrefix) && len(key) == (len(blockReceiptsPrefix)+8+common.HashLength):
 			receipts.Add(size)
+		case IsLegacyTrieNode(key, it.Value()):


RenRick

LGTM

fynnss

LGTM

flywukong force-pushed the support-speratedb branch from 25f8873 to c1b812c Compare December 1, 2023 03:27

flywukong marked this pull request as draft December 4, 2023 03:27

sysvm reviewed Dec 7, 2023

View reviewed changes

cmd/geth/dbcmd.go Outdated Show resolved Hide resolved

sysvm reviewed Dec 7, 2023

View reviewed changes

core/blockchain.go Outdated Show resolved Hide resolved

sysvm reviewed Dec 7, 2023

View reviewed changes

core/blockchain.go Outdated Show resolved Hide resolved

sysvm reviewed Dec 7, 2023

View reviewed changes

miner/stress/clique/main.go Outdated Show resolved Hide resolved

flywukong force-pushed the support-speratedb branch 3 times, most recently from 4fe0ab8 to 523ed87 Compare January 3, 2024 11:58

flywukong marked this pull request as ready for review January 3, 2024 13:14

flywukong changed the title ~~feat: support speratedb~~ feat: support sperated trie db Jan 3, 2024

flywukong force-pushed the support-speratedb branch 4 times, most recently from 8b18c3d to 2b7cb26 Compare January 18, 2024 02:22

fynnss reviewed Jan 18, 2024

View reviewed changes

cmd/utils/flags.go Outdated Show resolved Hide resolved

flywukong force-pushed the support-speratedb branch from 5f01167 to 5672be8 Compare January 18, 2024 08:20

sysvm reviewed Jan 19, 2024

View reviewed changes

cmd/utils/flags.go Outdated Show resolved Hide resolved

jingjunLi reviewed Jan 22, 2024

View reviewed changes

cmd/geth/chaincmd.go Outdated Show resolved Hide resolved

flywukong force-pushed the support-speratedb branch 2 times, most recently from 8a4198b to aa616b6 Compare January 29, 2024 13:53

flywukong changed the title ~~feat: support sperated trie db~~ feat: support separated trie database Jan 30, 2024

sysvm reviewed Jan 31, 2024

View reviewed changes

cmd/geth/dbcmd.go Outdated Show resolved Hide resolved

sysvm reviewed Jan 31, 2024

View reviewed changes

core/blockchain.go Outdated Show resolved Hide resolved

sysvm reviewed Jan 31, 2024

View reviewed changes

core/rawdb/database.go Outdated Show resolved Hide resolved

flywukong changed the title ~~feat: support separated trie database~~ feat: support separate trie database Jan 31, 2024

fynnss reviewed Feb 1, 2024

View reviewed changes

cmd/geth/snapshot.go Outdated Show resolved Hide resolved

cmd/utils/flags.go Outdated Show resolved Hide resolved

core/blockchain.go Outdated Show resolved Hide resolved

flywukong force-pushed the support-speratedb branch from e9f15b2 to 1622f54 Compare February 1, 2024 11:59

fynnss reviewed Mar 5, 2024

View reviewed changes

flywukong force-pushed the support-speratedb branch from 38e115d to 8c13f99 Compare March 6, 2024 10:17

fynnss previously approved these changes Mar 6, 2024

View reviewed changes

flywukong dismissed fynnss’s stale review via 1ff5ef1 March 7, 2024 02:26

flywukong force-pushed the support-speratedb branch from 1ff5ef1 to 53f32a1 Compare March 7, 2024 10:09

flywukong added 18 commits March 8, 2024 15:01

feat: support separate db for trie

79305da

fix: add log

50d1377

fix: fix trie dir init error

1dfa81d

fix: fix db inspect command

1eecb3c

refactor: delete the trie datadir flag and make it fixed

79143db

fix: fix db compaction

3b242e4

fix:optimize variable names

64029de

fix:fix separatedb namespace

480be65

fix:remove OpenStateDataBase function

d7d9c00

refactor: add stateStore to ethdb database

dd8ec3e

fix: fix state related commands

5d61f92

fix: remove unuse code

aaa1917

fix: fix inspect db

fc95001

fix: add parameter judgement for separate db

3e98700

refactor: support statestore reader

58645bd

fix: fix comments

c806e15

fix: fix inspect & dump commnd

474fcb4

fix: delete blank lines

32159eb

flywukong force-pushed the support-speratedb branch from 53f32a1 to 32159eb Compare March 8, 2024 07:18

RenRick approved these changes Mar 8, 2024

View reviewed changes

fynnss approved these changes Mar 8, 2024

View reviewed changes

flywukong merged commit def3512 into bnb-chain:develop Mar 8, 2024
7 checks passed

This was referenced Mar 12, 2024

release: prepare for release v1.4.2-alpha #2273

Merged

draft release v1.4.2-alpha #2274

Merged

cmd/geth: add dbDeleteAll #2275

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: support separate trie database #2021

feat: support separate trie database #2021

flywukong commented Dec 1, 2023 •

edited

Loading

sysvm commented Jan 31, 2024

flywukong commented Jan 31, 2024

fynnss Mar 5, 2024

flywukong Mar 6, 2024

fynnss Mar 6, 2024

fynnss left a comment

fynnss Mar 6, 2024

flywukong Mar 8, 2024

fynnss Mar 6, 2024

RenRick left a comment

fynnss left a comment

feat: support separate trie database #2021

feat: support separate trie database #2021

Conversation

flywukong commented Dec 1, 2023 • edited Loading

Description

Rationale

Example

Changes

sysvm commented Jan 31, 2024

flywukong commented Jan 31, 2024

fynnss Mar 5, 2024

Choose a reason for hiding this comment

flywukong Mar 6, 2024

Choose a reason for hiding this comment

fynnss Mar 6, 2024

Choose a reason for hiding this comment

fynnss left a comment

Choose a reason for hiding this comment

fynnss Mar 6, 2024

Choose a reason for hiding this comment

flywukong Mar 8, 2024

Choose a reason for hiding this comment

fynnss Mar 6, 2024

Choose a reason for hiding this comment

RenRick left a comment

Choose a reason for hiding this comment

fynnss left a comment

Choose a reason for hiding this comment

flywukong commented Dec 1, 2023 •

edited

Loading