Skip to content

Commit

Permalink
Small fixes for the database (#2015)
Browse files Browse the repository at this point in the history
- Fixed the name for historical columns - Metrics was working
incorrectly for historical columns.
- Added recommended setting for the RocksDB - the source of
recommendation is official documentation
https://github.com/facebook/rocksdb/wiki/Setup-Options-and-Basic-Tuning#other-general-options.
- Removed repairing since it could corrupt the database if fails -
several users reported about the corrupted state of the database after
having a "Too many descriptors" error where in logs, repairing of the
database also failed with this error creating a `lost` folder.

### Before requesting review
- [x] I have reviewed the code myself
  • Loading branch information
goldenpath1109 authored and rymnc committed Jul 5, 2024
1 parent 4bf048e commit 9784e6e
Show file tree
Hide file tree
Showing 3 changed files with 21 additions and 16 deletions.
21 changes: 11 additions & 10 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,16 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
- [#1994](https://github.com/FuelLabs/fuel-core/pull/1994): Added the actual implementation for the `AtomicView::latest_view`.
- [#1972](https://github.com/FuelLabs/fuel-core/pull/1972): Implement `AlgorithmUpdater` for `GasPriceService`
- [#1948](https://github.com/FuelLabs/fuel-core/pull/1948): Add new `AlgorithmV1` and `AlgorithmUpdaterV1` for the gas price. Include tools for analysis
- [#1676](https://github.com/FuelLabs/fuel-core/pull/1676): Added new CLI arguments:
- `graphql-max-depth`
- `graphql-max-complexity`
- `graphql-max-recursive-depth`

### Changed
- [#2015](https://github.com/FuelLabs/fuel-core/pull/2015): Small fixes for the database:
- Fixed the name for historical columns - Metrics was working incorrectly for historical columns.
- Added recommended setting for the RocksDB - The source of recommendation is official documentation https://github.com/facebook/rocksdb/wiki/Setup-Options-and-Basic-Tuning#other-general-options.
- Removed repairing since it could corrupt the database if fails - Several users reported about the corrupted state of the database after having a "Too many descriptors" error where in logs, repairing of the database also failed with this error creating a `lost` folder.
- [#2010](https://github.com/FuelLabs/fuel-core/pull/2010): Updated the block importer to allow more blocks to be in the queue. It improves synchronization speed and mitigate the impact of other services on synchronization speed.
- [#2006](https://github.com/FuelLabs/fuel-core/pull/2006): Process block importer events first under P2P pressure.
- [#2002](https://github.com/FuelLabs/fuel-core/pull/2002): Adapted the block producer to react to checked transactions that were using another version of consensus parameters during validation in the TxPool. After an upgrade of the consensus parameters of the network, TxPool could store invalid `Checked` transactions. This change fixes that by tracking the version that was used to validate the transactions.
Expand All @@ -35,6 +43,9 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
- [#2001](https://github.com/FuelLabs/fuel-core/pull/2001): Prevent GraphQL query body to be huge and cause OOM. The default body size is `1MB`. The limit can be changed by the `graphql-request-body-bytes-limit` CLI argument.
- [#1991](https://github.com/FuelLabs/fuel-core/pull/1991): Prepare the database to use different types than `Database` for atomic view.
- [#1989](https://github.com/FuelLabs/fuel-core/pull/1989): Extract `HistoricalView` trait from the `AtomicView`.
- [#1676](https://github.com/FuelLabs/fuel-core/pull/1676): New `fuel-core-client` is incompatible with the old `fuel-core` because of two requested new fields.
- [#1676](https://github.com/FuelLabs/fuel-core/pull/1676): Changed default value for `api-request-timeout` to be `30s`.
- [#1676](https://github.com/FuelLabs/fuel-core/pull/1676): Now, GraphQL API has complexity and depth limitations on the queries. The default complexity limit is `20000`. It is ~50 blocks per request with transaction IDs and ~2-5 full blocks.

### Fixed
- [#2000](https://github.com/FuelLabs/fuel-core/pull/2000): Use correct query name in metrics for aliased queries.
Expand All @@ -57,24 +68,14 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
## [Version 0.29.0]

### Added
- [#1676](https://github.com/FuelLabs/fuel-core/pull/1676): Added new CLI arguments:
- `graphql-max-depth`
- `graphql-max-complexity`
- `graphql-max-recursive-depth`
- [#1889](https://github.com/FuelLabs/fuel-core/pull/1889): Add new `FuelGasPriceProvider` that receives the gas price algorithm from a `GasPriceService`

### Changed
- [#1676](https://github.com/FuelLabs/fuel-core/pull/1676): Changed default value for `api-request-timeout` to be `30s`.
- [#1942](https://github.com/FuelLabs/fuel-core/pull/1942): Sequential relayer's commits.
- [#1952](https://github.com/FuelLabs/fuel-core/pull/1952): Change tip sorting to ratio between tip and max gas sorting in txpool
- [#1960](https://github.com/FuelLabs/fuel-core/pull/1960): Update fuel-vm to v0.53.0.
- [#1964](https://github.com/FuelLabs/fuel-core/pull/1964): Add `creation_instant` as second sort key in tx pool

#### Breaking

- [#1676](https://github.com/FuelLabs/fuel-core/pull/1676): Now, GraphQL API has complexity and depth limitations on the queries. The default complexity limit is `20000`. It is ~50 blocks per request with transaction IDs and ~2-5 full blocks.
- [#1676](https://github.com/FuelLabs/fuel-core/pull/1676): New `fuel-core-client` is incompatible with the old `fuel-core` because of two requested new fields.

### Fixed
- [#1962](https://github.com/FuelLabs/fuel-core/pull/1962): Fixes the error message for incorrect keypair's path.
- [#1950](https://github.com/FuelLabs/fuel-core/pull/1950): Fix cursor `BlockHeight` encoding in `SortedTXCursor`
Expand Down
6 changes: 3 additions & 3 deletions crates/fuel-core/src/state/historical_rocksdb/description.rs
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ where
fn name(&self) -> String {
match self {
Column::OriginalColumn(c) => c.name(),
Column::HistoricalDuplicateColumn(_) => {
format!("history_{}", Description::name())
Column::HistoricalDuplicateColumn(c) => {
format!("history_{}", c.name())
}
Column::HistoryColumn => "modifications_history".to_string(),
}
Expand Down Expand Up @@ -68,7 +68,7 @@ where
}

fn name() -> String {
format!("{}_history", Description::name())
Description::name()
}

fn metadata_column() -> Self::Column {
Expand Down
10 changes: 7 additions & 3 deletions crates/fuel-core/src/state/rocks_db.rs
Original file line number Diff line number Diff line change
Expand Up @@ -286,6 +286,7 @@ where
block_opts.disable_cache();
}
block_opts.set_bloom_filter(10.0, true);
block_opts.set_block_size(16 * 1024);

let mut opts = Options::default();
opts.create_if_missing(true);
Expand All @@ -304,6 +305,11 @@ where
let cache = Cache::new_lru_cache(row_cache_size);
opts.set_row_cache(&cache);
}
opts.set_max_background_jobs(6);
opts.set_bytes_per_sync(1048576);

#[cfg(feature = "test-helpers")]
opts.set_max_open_files(512);

let existing_column_families = DB::list_cf(&opts, &path).unwrap_or_default();

Expand All @@ -330,9 +336,7 @@ where
Ok(db)
},
Err(err) => {
tracing::error!("Couldn't open the database with an error: {}. \nTrying to repair the database", err);
DB::repair(&opts, &path)
.map_err(|e| DatabaseError::Other(e.into()))?;
tracing::error!("Couldn't open the database with an error: {}. \nTrying to reopen the database", err);

let iterator = cf_descriptors_to_open
.clone()
Expand Down

0 comments on commit 9784e6e

Please sign in to comment.