Merge branch 'master' into make-consensus-versionable

FuelLabs · Jan 19, 2024 · 6a8a2e4 · 6a8a2e4
2 parents 77640b4 + 97ab308
commit 6a8a2e4
Show file tree

Hide file tree

Showing 116 changed files with 5,140 additions and 3,117 deletions.
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -105,6 +105,8 @@ jobs:
             args: --all-targets --all-features
           - command: check
             args: --all-targets
+          - command: doc
+            args: --all-features --workspace
           - command: make
             args: check --locked
           - command: test

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -8,18 +8,124 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 
 Description of the upcoming release here.
 
-
 ### Changed
 
 - [#1591](https://github.com/FuelLabs/fuel-core/pull/1591): Simplify libp2p dependencies and not depend on all sub modules directly.
 - [#1585](https://github.com/FuelLabs/fuel-core/pull/1585): Let `NetworkBehaviour` macro generate `FuelBehaviorEvent` in p2p
+- [#1579](https://github.com/FuelLabs/fuel-core/pull/1579): The change extracts the off-chain-related logic from the executor and moves it to the GraphQL off-chain worker. It creates two new concepts - Off-chain and On-chain databases where the GraphQL worker has exclusive ownership of the database and may modify it without intersecting with the On-chain database.
 - [#1577](https://github.com/FuelLabs/fuel-core/pull/1577): Moved insertion of sealed blocks into the `BlockImporter` instead of the executor.
+- [#1601](https://github.com/FuelLabs/fuel-core/pull/1601): Fix formatting in docs and check that `cargo doc` passes in the CI.
 
 #### Breaking
 - [#1596](https://github.com/FuelLabs/fuel-core/pull/1596) Make `Consensus` type a version-able enum
 - [#1593](https://github.com/FuelLabs/fuel-core/pull/1593) Make `Block` type a version-able enum
+- [#1576](https://github.com/FuelLabs/fuel-core/pull/1576): The change moves the implementation of the storage traits for required tables from `fuel-core` to `fuel-core-storage` crate. The change also adds a more flexible configuration of the encoding/decoding per the table and allows the implementation of specific behaviors for the table in a much easier way. It unifies the encoding between database, SMTs, and iteration, preventing mismatching bytes representation on the Rust type system level. Plus, it increases the re-usage of the code by applying the same blueprint to other tables.
+
+    It is a breaking PR because it changes database encoding/decoding for some tables.
+
+    ### StructuredStorage
+
+    The change adds a new type `StructuredStorage`. It is a wrapper around the key-value storage that implements the storage traits(`StorageInspect`, `StorageMutate`, `StorageRead`, etc) for the tables with blueprint. This blueprint works in tandem with the `TableWithBlueprint` trait. The table may implement `TableWithBlueprint` specifying the blueprint, as an example:
+
+    ```rust
+    impl TableWithBlueprint for ContractsRawCode {
+        type Blueprint = Plain<Raw, Raw>;
+
+        fn column() -> Column {
+            Column::ContractsRawCode
+        }
+    }
+    ```
+
+    It is a definition of the blueprint for the `ContractsRawCode` table. It has a plain blueprint meaning it simply encodes/decodes bytes and stores/loads them into/from the storage. As a key codec and value codec, it uses a `Raw` encoding/decoding that simplifies writing bytes and loads them back into the memory without applying any serialization or deserialization algorithm.
+
+    If the table implements `TableWithBlueprint` and the selected codec satisfies all blueprint requirements, the corresponding storage traits for that table are implemented on the `StructuredStorage` type.
+
+    ### Codecs
+
+    Each blueprint allows customizing the key and value codecs. It allows the use of different codecs for different tables, taking into account the complexity and weight of the data and providing a way of more optimal implementation.
+
+    That property may be very useful to perform migration in a more easier way. Plus, it also can be a `no_std` migration potentially allowing its fraud proving.
+
+    An example of migration:
+
+    ```rust
+    /// Define the table for V1 value encoding/decoding.
+    impl TableWithBlueprint for ContractsRawCodeV1 {
+        type Blueprint = Plain<Raw, Raw>;
+
+        fn column() -> Column {
+            Column::ContractsRawCode
+        }
+    }
+
+    /// Define the table for V2 value encoding/decoding.
+    /// It uses `Postcard` codec for the value instead of `Raw` codec.
+    ///
+    /// # Dev-note: The columns is the same.
+    impl TableWithBlueprint for ContractsRawCodeV2 {
+        type Blueprint = Plain<Raw, Postcard>;
+
+        fn column() -> Column {
+            Column::ContractsRawCode
+        }
+    }
+
+    fn migration(storage: &mut Database) {
+        let mut iter = storage.iter_all::<ContractsRawCodeV1>(None);
+        while let Ok((key, value)) = iter.next() {
+            // Insert into the same table but with another codec.
+            storage.storage::<ContractsRawCodeV2>().insert(key, value);
+        }
+    }
+    ```
+
+    ### Structures
+
+    The blueprint of the table defines its behavior. As an example, a `Plain` blueprint simply encodes/decodes bytes and stores/loads them into/from the storage. The `SMT` blueprint builds a sparse merkle tree on top of the key-value pairs.
+
+    Implementing a blueprint one time, we can apply it to any table satisfying the requirements of this blueprint. It increases the re-usage of the code and minimizes duplication.
+
+    It can be useful if we decide to create global roots for all required tables that are used in fraud proving.
+
+    ```rust
+    impl TableWithBlueprint for SpentMessages {
+        type Blueprint = Plain<Raw, Postcard>;
+
+        fn column() -> Column {
+            Column::SpentMessages
+        }
+    }
+                     |
+                     |
+                    \|/
+
+    impl TableWithBlueprint for SpentMessages {
+        type Blueprint =
+            Sparse<Raw, Postcard, SpentMessagesMerkleMetadata, SpentMessagesMerkleNodes>;
+
+        fn column() -> Column {
+            Column::SpentMessages
+        }
+    }
+    ```
+
+    ### Side changes
+
+    #### `iter_all`
+    The `iter_all` functionality now accepts the table instead of `K` and `V` generics. It is done to use the correct codec during deserialization. Also, the table definition provides the column.
+
+    #### Duplicated unit tests
+
+    The `fuel-core-storage` crate provides macros that generate unit tests. Almost all tables had the same test like `get`, `insert`, `remove`, `exist`. All duplicated tests were moved to macros. The unique one still stays at the same place where it was before.
+
+    #### `StorageBatchMutate`
+
+    Added a new `StorageBatchMutate` trait that we can move to `fuel-storage` crate later. It allows batch operations on the storage. It may be more performant in some cases.
+
 - [#1573](https://github.com/FuelLabs/fuel-core/pull/1573): Remove nested p2p request/response encoding. Only breaks p2p networking compatibility with older fuel-core versions, but is otherwise fully internal.
 
+
 ## [Version 0.22.0]
 
 ### Added