Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Implement nested storage transactions #6269

Merged
merged 49 commits into from
Jun 23, 2020
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
5b3d0b5
Add transactional storage functionality to OverlayChanges
athei May 25, 2020
68e34f0
Add storage transactions runtime interface
athei Jun 5, 2020
0a70191
Add frame support for transactions
athei Jun 5, 2020
0e32850
Merge branch 'master' into at-storage-tx
athei Jun 6, 2020
c767eea
Fix committed typo
athei Jun 8, 2020
c48d417
Rename 'changes' variable to 'overlay'
athei Jun 8, 2020
98a89da
Fix renaming change
athei Jun 8, 2020
3fb2e05
Fixed strange line break
athei Jun 8, 2020
1acf7a4
Rename clear to clear_where
athei Jun 8, 2020
d4440a4
Add comment regarding delete value on mutation
athei Jun 8, 2020
14738f8
Merge remote-tracking branch 'origin/master' into at-storage-tx
athei Jun 8, 2020
3592246
Add comment which changes are covered by a transaction
athei Jun 8, 2020
7395a03
Do force the arg to with_transaction return a Result
athei Jun 8, 2020
81e5241
Use rust doc comments on every documentable place
athei Jun 9, 2020
ff52101
Fix wording of insert_diry doc
athei Jun 9, 2020
d789aab
Improve doc on start_transaction
athei Jun 9, 2020
5e65240
Rename value to overlayed in close_transaction
athei Jun 9, 2020
7eb9065
Inline negation
athei Jun 9, 2020
5dd6a85
Improve wording of close_transaction comments
athei Jun 9, 2020
1c8e7c1
Get rid of an expect by using get_or_insert_with
athei Jun 9, 2020
232b68f
Remove trailing whitespace
athei Jun 9, 2020
00b9ff3
Rename should to expected in tests
athei Jun 9, 2020
85617a5
Rolling back a transaction must mark the overlay as dirty
athei Jun 9, 2020
04471b4
Merge remote-tracking branch 'origin/master' into at-storage-tx
athei Jun 9, 2020
4e039a1
Protect client initiated storage tx from being droped by runtime
athei Jun 9, 2020
52888f2
Merge remote-tracking branch 'origin/master' into at-storage-tx
athei Jun 10, 2020
db24a2b
Merge remote-tracking branch 'origin/master' into at-storage-tx
athei Jun 10, 2020
8ee3696
Review nits
athei Jun 10, 2020
843c84a
Return Err when entering or exiting runtime fails
athei Jun 10, 2020
5c851f6
Merge remote-tracking branch 'origin/master' into at-storage-tx
athei Jun 10, 2020
c60a4c1
Documentation fixup
athei Jun 10, 2020
1ad2077
Remove close type
athei Jun 11, 2020
de8af5b
Move enter/exit runtime to excute_aux in the state-machine
athei Jun 11, 2020
552f611
Merge remote-tracking branch 'origin/master' into at-storage-tx
athei Jun 11, 2020
4cf20c9
Rename Discard -> Rollback
athei Jun 11, 2020
10022f6
Move child changeset creation to constructor
athei Jun 11, 2020
5c0c35f
Move child spawning into the closure
athei Jun 11, 2020
8db272c
Merge remote-tracking branch 'origin/master' into at-storage-tx
athei Jun 11, 2020
76ce5cf
Apply suggestions from code review
athei Jun 12, 2020
3d0f162
Fixup for code suggestion
athei Jun 12, 2020
1651f68
Unify re-exports
athei Jun 12, 2020
64f3c90
Rename overlay_changes to mod.rs and move into subdir
athei Jun 12, 2020
1568984
Change proof wording
athei Jun 16, 2020
96b6fd6
Merge branch 'master' into at-storage-tx
athei Jun 16, 2020
5878df6
Merge branch 'master' into at-storage-tx
athei Jun 19, 2020
6a8b3a7
Adapt a new test from master to storage-tx
athei Jun 19, 2020
46a56dc
Suggestions from the latest round of review
athei Jun 20, 2020
1d72286
Merge remote-tracking branch 'origin/master' into at-storage-tx
athei Jun 20, 2020
5145916
Fix warning message
athei Jun 21, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 20 additions & 8 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

18 changes: 18 additions & 0 deletions frame/support/src/storage/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,24 @@ pub mod child;
pub mod generator;
pub mod migration;

/// Execute the supplied function in a new storage transaction.
///
/// All changes to storage performed by the supplied function are discarded if an
/// error is returned and comitted on success.
///
/// Transactions can be nested to any depth. Commits happen to the parent transaction.
pub fn with_transaction<T, E, F>(f: F) -> Result<T, E> where
F: FnOnce() -> Result<T, E>
{
sp_io::storage::start_transaction();
let result = f();
match result {
Ok(_) => sp_io::storage::commit_transaction(),
Err(_) => sp_io::storage::rollback_transaction(),
}
result
}
athei marked this conversation as resolved.
Show resolved Hide resolved

/// A trait for working with macro-generated storage values under the substrate storage API.
///
/// Details on implementation can be found at
Expand Down
157 changes: 157 additions & 0 deletions frame/support/test/tests/storage_transaction.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
// This file is part of Substrate.

// Copyright (C) 2019-2020 Parity Technologies (UK) Ltd.
athei marked this conversation as resolved.
Show resolved Hide resolved
// SPDX-License-Identifier: Apache-2.0

// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use codec::{Encode, Decode, EncodeLike};
use frame_support::{StorageMap, StorageValue, storage::with_transaction};
use sp_io::TestExternalities;

pub trait Trait {
type Origin;
type BlockNumber: Encode + Decode + EncodeLike + Default + Clone;
}

frame_support::decl_module! {
pub struct Module<T: Trait> for enum Call where origin: T::Origin {}
}

frame_support::decl_storage!{
trait Store for Module<T: Trait> as StorageTransactions {
pub Value: u32;
pub Map: map hasher(twox_64_concat) String => u32;
}
}


#[test]
fn storage_transaction_basic_commit() {
TestExternalities::default().execute_with(|| {

assert_eq!(Value::get(), 0);
assert!(!Map::contains_key("val0"));

let _: Result<(), ()> = with_transaction(|| {
Value::set(99);
Map::insert("val0", 99);
assert_eq!(Value::get(), 99);
assert_eq!(Map::get("val0"), 99);
Ok(())
});

assert_eq!(Value::get(), 99);
assert_eq!(Map::get("val0"), 99);
});
}

#[test]
fn storage_transaction_basic_rollback() {
TestExternalities::default().execute_with(|| {

assert_eq!(Value::get(), 0);
assert_eq!(Map::get("val0"), 0);

let _: Result<(), ()> = with_transaction(|| {
Value::set(99);
Map::insert("val0", 99);
assert_eq!(Value::get(), 99);
assert_eq!(Map::get("val0"), 99);
Err(())
});

assert_eq!(Value::get(), 0);
assert_eq!(Map::get("val0"), 0);
});
}

#[test]
fn storage_transaction_rollback_then_commit() {
TestExternalities::default().execute_with(|| {
Value::set(1);
Map::insert("val1", 1);

let _: Result<(), ()> = with_transaction(|| {
Value::set(2);
Map::insert("val1", 2);
Map::insert("val2", 2);

let _: Result<(), ()> = with_transaction(|| {
Value::set(3);
Map::insert("val1", 3);
Map::insert("val2", 3);
Map::insert("val3", 3);

assert_eq!(Value::get(), 3);
assert_eq!(Map::get("val1"), 3);
assert_eq!(Map::get("val2"), 3);
assert_eq!(Map::get("val3"), 3);

Err(())
});

assert_eq!(Value::get(), 2);
assert_eq!(Map::get("val1"), 2);
assert_eq!(Map::get("val2"), 2);
assert_eq!(Map::get("val3"), 0);

Ok(())
});

assert_eq!(Value::get(), 2);
assert_eq!(Map::get("val1"), 2);
assert_eq!(Map::get("val2"), 2);
assert_eq!(Map::get("val3"), 0);
});
}

#[test]
fn storage_transaction_commit_then_rollback() {
TestExternalities::default().execute_with(|| {
Value::set(1);
Map::insert("val1", 1);

let _: Result<(), ()> = with_transaction(|| {
Value::set(2);
Map::insert("val1", 2);
Map::insert("val2", 2);

let _: Result<(), ()> = with_transaction(|| {
Value::set(3);
Map::insert("val1", 3);
Map::insert("val2", 3);
Map::insert("val3", 3);

assert_eq!(Value::get(), 3);
assert_eq!(Map::get("val1"), 3);
assert_eq!(Map::get("val2"), 3);
assert_eq!(Map::get("val3"), 3);

Ok(())
});

assert_eq!(Value::get(), 3);
assert_eq!(Map::get("val1"), 3);
assert_eq!(Map::get("val2"), 3);
assert_eq!(Map::get("val3"), 3);

Err(())
});

assert_eq!(Value::get(), 1);
assert_eq!(Map::get("val1"), 1);
assert_eq!(Map::get("val2"), 0);
assert_eq!(Map::get("val3"), 0);
});
}
8 changes: 6 additions & 2 deletions primitives/api/proc-macro/src/impl_runtime_apis.rs
Original file line number Diff line number Diff line change
Expand Up @@ -257,6 +257,7 @@ fn generate_runtime_api_base_structures() -> Result<TokenStream> {
&self,
map_call: F,
) -> std::result::Result<R, E> where Self: Sized {
self.changes.borrow_mut().start_transaction();
*self.commit_on_success.borrow_mut() = false;
let res = map_call(self);
*self.commit_on_success.borrow_mut() = true;
Expand Down Expand Up @@ -366,6 +367,9 @@ fn generate_runtime_api_base_structures() -> Result<TokenStream> {
&self,
call_api_at: F,
) -> std::result::Result<#crate_::NativeOrEncoded<R>, E> {
if *self.commit_on_success.borrow() {
self.changes.borrow_mut().start_transaction();
}
let res = call_api_at(
&self.call,
self,
Expand All @@ -383,9 +387,9 @@ fn generate_runtime_api_base_structures() -> Result<TokenStream> {
fn commit_on_ok<R, E>(&self, res: &std::result::Result<R, E>) {
if *self.commit_on_success.borrow() {
if res.is_err() {
self.changes.borrow_mut().discard_prospective();
self.changes.borrow_mut().rollback_transaction();
athei marked this conversation as resolved.
Show resolved Hide resolved
} else {
self.changes.borrow_mut().commit_prospective();
self.changes.borrow_mut().commit_transaction();
athei marked this conversation as resolved.
Show resolved Hide resolved
}
}
}
Expand Down
25 changes: 25 additions & 0 deletions primitives/externalities/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,31 @@ pub trait Externalities: ExtensionStore {
/// The returned hash is defined by the `Block` and is SCALE encoded.
fn storage_changes_root(&mut self, parent: &[u8]) -> Result<Option<Vec<u8>>, ()>;

/// Start a new nested transaction.
///
/// This allows to either commit or roll back all changes that are made after this call.
athei marked this conversation as resolved.
Show resolved Hide resolved
/// For every transaction there must be a matching call to either `storage_rollback_transaction`
/// or `storage_commit_transaction`.
///
/// Changes made without any open transaction are comitted immediatly
fn storage_start_transaction(&mut self);

/// Rollback the last transaction started by `start_transaction`.
///
/// Any changes made during that transaction are discarded.
///
/// Panics:
/// Will panic if there is no open transaction.
athei marked this conversation as resolved.
Show resolved Hide resolved
fn storage_rollback_transaction(&mut self);

/// Commit the last transaction started by `start_transaction`.
///
/// Any changes made during that transaction are committed.
///
/// Panics:
/// Will panic if there is no open transaction.
athei marked this conversation as resolved.
Show resolved Hide resolved
fn storage_commit_transaction(&mut self);

/// !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
/// Benchmarking related functionality and shouldn't be used anywhere else!
/// !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Expand Down
37 changes: 37 additions & 0 deletions primitives/io/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,43 @@ pub trait Storage {
fn next_key(&mut self, key: &[u8]) -> Option<Vec<u8>> {
self.next_storage_key(&key)
}

/// Start a new nested transaction.
///
/// This allows to either commit or roll back all changes that are made after this call.
/// For every transaction there must be a matching call to either `rollback_transaction`
/// or `commit_transaction`. This is also effective for all values manipulated using the
/// `DefaultChildStorage` API.
///
/// # Warning
///
/// This is a low level API that is potentially dangerous as it can easily result
/// in unbalanced transactions. FRAME users should use high level storage abstractions.
athei marked this conversation as resolved.
Show resolved Hide resolved
fn start_transaction(&mut self) {
self.storage_start_transaction();
}

/// Rollback the last transaction started by `start_transaction`.
///
/// Any changes made during that transaction are discarded.
///
/// # Panics
///
/// Will panic if there is no open transaction.
fn rollback_transaction(&mut self) {
self.storage_rollback_transaction();
}

/// Commit the last transaction started by `start_transaction`.
///
/// Any changes made during that transaction are committed.
///
/// # Panics
///
/// Will panic if there is no open transaction.
fn commit_transaction(&mut self) {
self.storage_commit_transaction();
}
}

/// Interface for accessing the child storage for default child trie,
Expand Down
Loading