WIP: Replace `wasm-gc` with `wasm-opt` #12280

athei · 2022-09-15T15:50:55Z

This replaces the outdated wasm-gc crate by the new wasm-opt crate for post processing of the runtime blob. The latter is a safe binding for the tried and true binaryen. For now we run with optimization level -O0 to just do simple garbage collection until we are confident enough in the applied optimizations.

Let's run some benchmarks.

Possible due to https://github.com/w3f/Grants-Program/blob/master/applications/wasm-opt-for-rust.md

cc @Robbepop
closes #1262

polkadot companion: paritytech/polkadot#6209
cumulus companion: paritytech/cumulus#1806

nazar-pc · 2022-09-17T00:59:17Z

This will close #1262 if merged

ggwpez

I dont see a performance regression here.

ggwpez · 2022-09-21T14:14:37Z

utils/wasm-builder/Cargo.toml

 sp-maybe-compressed-blob = { version = "4.1.0-dev", path = "../../primitives/maybe-compressed-blob" }
 filetime = "0.2.16"
+wasm-opt = "0.0.1-preview.3"


In case someone wonders (quote from Alex):

the wasm-opt binding is still in alpha but the underlying wasmopt is mature

athei · 2022-09-21T20:36:26Z

Rebased and removed changes to weight files in order to avoid merge conflicts.

koute · 2022-09-22T15:34:00Z

Some more numbers (release and production are for the runtime from substrate with benchmarking enabled; polkadot is for the polkadot runtime compiled under production; sorry, the table is kind of a chungus):

kind	functions	size	size_compressed	time	has name	has DWARF
polkadot	3005	5330k	1264k	0s	true	true
polkadot wasm-gc	3005	4884k	1203k	0s	true	false
polkadot wasm-gc --no-demangle	3005	4945k	1210k	0s	true	false
polkadot wasm-opt -O0	3005	4716k	1170k	0s	false	true
polkadot wasm-opt -O1	2936	4520k	1147k	1s	false	true
polkadot wasm-opt -O2	2597	4504k	1156k	6s	false	true
polkadot wasm-opt -O3	2592	4476k	1163k	9s	false	true
polkadot wasm-opt -O4	2593	4500k	1168k	17s	false	true
polkadot wasm-opt -Os	2594	4471k	1162k	7s	false	true
polkadot wasm-opt -Oz	2594	4472k	1161k	9s	false	true
polkadot wasm-opt -O0 --strip-dwarf	3005	4570k	1132k	0s	false	false
polkadot wasm-opt -O1 --strip-dwarf	2936	4375k	1109k	1s	false	false
polkadot wasm-opt -O2 --strip-dwarf	2597	4364k	1120k	6s	false	false
polkadot wasm-opt -O3 --strip-dwarf	2592	4335k	1125k	9s	false	false
polkadot wasm-opt -O4 --strip-dwarf	2593	4359k	1133k	17s	false	false
polkadot wasm-opt -Os --strip-dwarf	2594	4331k	1124k	7s	false	false
polkadot wasm-opt -Oz --strip-dwarf	2594	4331k	1124k	9s	false	false
polkadot wasm-opt -O0 -g	3005	5110k	1258k	2s	true	true
polkadot wasm-opt -O1 -g	3005	4939k	1237k	3s	true	true
polkadot wasm-opt -O2 -g	3005	4931k	1234k	3s	true	true
polkadot wasm-opt -O3 -g	3005	4919k	1234k	4s	true	true
polkadot wasm-opt -O4 -g	3005	4919k	1234k	4s	true	true
polkadot wasm-opt -Os -g	3005	4914k	1233k	3s	true	true
polkadot wasm-opt -Oz -g	3005	4915k	1233k	4s	true	true
polkadot wasm-opt -O0 -g --strip-dwarf	3005	4941k	1206k	0s	true	false
polkadot wasm-opt -O1 -g --strip-dwarf	3005	4771k	1185k	1s	true	false
polkadot wasm-opt -O2 -g --strip-dwarf	3005	4763k	1182k	1s	true	false
polkadot wasm-opt -O3 -g --strip-dwarf	3005	4751k	1183k	2s	true	false
polkadot wasm-opt -O4 -g --strip-dwarf	3005	4751k	1183k	2s	true	false
polkadot wasm-opt -Os -g --strip-dwarf	3005	4746k	1182k	2s	true	false
polkadot wasm-opt -Oz -g --strip-dwarf	3005	4747k	1182k	2s	true	false
production	3907	6843k	1694k	0s	true	true
production wasm-gc	3907	6617k	1636k	0s	true	false
production wasm-gc --no-demangle	3907	6696k	1645k	0s	true	false
production wasm-opt -O0	3907	6406k	1594k	0s	false	true
production wasm-opt -O1	3832	6140k	1566k	1s	false	true
production wasm-opt -O2	3463	6122k	1585k	7s	false	true
production wasm-opt -O3	3458	6070k	1591k	14s	false	true
production wasm-opt -O4	3458	6108k	1602k	23s	false	true
production wasm-opt -Os	3459	6063k	1591k	10s	false	true
production wasm-opt -Oz	3459	6063k	1590k	14s	false	true
production wasm-opt -O0 --strip-dwarf	3907	6228k	1547k	0s	false	false
production wasm-opt -O1 --strip-dwarf	3832	5962k	1517k	1s	false	false
production wasm-opt -O2 --strip-dwarf	3463	5949k	1538k	6s	false	false
production wasm-opt -O3 --strip-dwarf	3458	5896k	1548k	13s	false	false
production wasm-opt -O4 --strip-dwarf	3458	5933k	1558k	22s	false	false
production wasm-opt -Os --strip-dwarf	3459	5889k	1547k	10s	false	false
production wasm-opt -Oz --strip-dwarf	3459	5890k	1546k	13s	false	false
production wasm-opt -O0 -g	3907	6882k	1695k	3s	true	true
production wasm-opt -O1 -g	3907	6642k	1670k	4s	true	true
production wasm-opt -O2 -g	3907	6632k	1665k	4s	true	true
production wasm-opt -O3 -g	3907	6601k	1663k	5s	true	true
production wasm-opt -O4 -g	3907	6601k	1663k	5s	true	true
production wasm-opt -Os -g	3907	6594k	1662k	5s	true	true
production wasm-opt -Oz -g	3907	6595k	1661k	5s	true	true
production wasm-opt -O0 -g --strip-dwarf	3907	6691k	1640k	0s	true	false
production wasm-opt -O1 -g --strip-dwarf	3907	6452k	1614k	1s	true	false
production wasm-opt -O2 -g --strip-dwarf	3907	6442k	1610k	1s	true	false
production wasm-opt -O3 -g --strip-dwarf	3907	6412k	1608k	2s	true	false
production wasm-opt -O4 -g --strip-dwarf	3907	6412k	1608k	2s	true	false
production wasm-opt -Os -g --strip-dwarf	3907	6404k	1608k	2s	true	false
production wasm-opt -Oz -g --strip-dwarf	3907	6406k	1608k	2s	true	false
release	8752	9339k	1914k	0s	true	true
release wasm-gc	8752	8989k	1841k	0s	true	false
release wasm-gc --no-demangle	8752	9166k	1859k	0s	true	false
release wasm-opt -O0	8752	8347k	1738k	0s	false	true
release wasm-opt -O1	8044	7807k	1706k	1s	false	true
release wasm-opt -O2	3993	7658k	1862k	7s	false	true
release wasm-opt -O3	3895	7561k	1851k	45s	false	true
release wasm-opt -O4	3889	7601k	1873k	55s	false	true
release wasm-opt -Os	3951	7505k	1847k	35s	false	true
release wasm-opt -Oz	3951	7505k	1855k	37s	false	true
release wasm-opt -O0 --strip-dwarf	8752	8099k	1684k	0s	false	false
release wasm-opt -O1 --strip-dwarf	8044	7564k	1648k	1s	false	false
release wasm-opt -O2 --strip-dwarf	3993	7453k	1809k	7s	false	false
release wasm-opt -O3 --strip-dwarf	3895	7356k	1797k	44s	false	false
release wasm-opt -O4 --strip-dwarf	3889	7395k	1821k	55s	false	false
release wasm-opt -Os --strip-dwarf	3951	7300k	1794k	35s	false	false
release wasm-opt -Oz --strip-dwarf	3951	7300k	1804k	37s	false	false
release wasm-opt -O0 -g	8752	9421k	1918k	4s	true	true
release wasm-opt -O1 -g	8752	9174k	1901k	5s	true	true
release wasm-opt -O2 -g	8752	9162k	1896k	5s	true	true
release wasm-opt -O3 -g	8752	9075k	1895k	6s	true	true
release wasm-opt -O4 -g	8752	9075k	1895k	6s	true	true
release wasm-opt -Os -g	8752	9066k	1893k	6s	true	true
release wasm-opt -Oz -g	8752	9066k	1893k	6s	true	true
release wasm-opt -O0 -g --strip-dwarf	8752	9159k	1855k	0s	true	false
release wasm-opt -O1 -g --strip-dwarf	8752	8913k	1831k	1s	true	false
release wasm-opt -O2 -g --strip-dwarf	8752	8902k	1829k	1s	true	false
release wasm-opt -O3 -g --strip-dwarf	8752	8814k	1828k	2s	true	false
release wasm-opt -O4 -g --strip-dwarf	8752	8814k	1828k	2s	true	false
release wasm-opt -Os -g --strip-dwarf	8752	8805k	1826k	2s	true	false
release wasm-opt -Oz -g --strip-dwarf	8752	8805k	1826k	2s	true	false

Some observations:

When ran without -g and --strip-dwarf the name section is stripped, but the DWARF sections are kept.
When ran with -g the name section is kept.
The name section is used when decoding symbols so that e.g. instead of _wasm_function_1234 we get wasmi_validation::func::FunctionValidationContext::validate_unop::ha8ecb960f7924be1.
wasm-gc leaves the name section, but strips the DWARF sections.
Specifying -g disables most of wasm-opt's transformations.
I have no idea whether the DWARF sections are correct after wasm-opt's transformations; it's possible they are garbage. It seems strange that having wasm-opt emit the name section (which should be very easy to fixup after transformations) disables most of its transformations, but having it emit the DWARF sections (which should be harder to fixup) do not.
The smallest compressed size we get with wasm-opt -O1 --strip-dwarf
A direct replacement of our current use of wasm-gc would be wasm-opt -O0 -g --strip-dwarf, which is only 3k bigger compressed for polkadot.

Performance-wise it's a little tricky to compare. I tried to generate a matrix of number for one of the contracts' benchmarks, but that's not really possible due to wasmtime's loop misalignment bug randomly tanking the performance. One interesting thing I've found for those combinations which are not affected by the bug is this (lower is better):

pallet_contracts/seal_return_per_kb in release configuration: ~225
pallet_contracts/seal_return_per_kb in release + O2 and release + O3 configuration: ~215
pallet_contracts/seal_return_per_kb in production configuration: ~191
pallet_contracts/seal_return_per_kb in production + O3 configuration: ~211

So when built under the release profile we get a slightly a slight performance bump when using wasm-opt -O2, but when we build the runtime under the production profile we get a performance regression when using wasm-opt -O3. Of course, this is just one data point with one benchmark on one machine, but this does suggest that performance-wise it's not necessarily a slam dunk.

Also I'm a little nervous about is the correctness of the transformations that the wasm-opt is doing here, especially at more niche levels like -Oz. I'm not necessarily saying it's going to be buggy, but it does vastly more than wasm-gc so I'd like some stronger guarantee that it's not going to cause problems.

So if we decide to introduce this I think I'd like to run a full sync burn-in with wasm-opt hooked into the node. (Basically just hack the executor so that it preoptimizes any .wasm blob with wasm-opt that it loads before compiling it, and then sync a node from scratch and see if it's successful.) I can set such a burn-in up on one of my GCP machines provided we can come to consensus as to which optimization level we want to pick here.

Open questions

Do we want to keep the name section? Having this should make it possible to decode the symbols when a panic happens inside of WASM, but can cost up to ~80k compressed, and prohibits most of wasm-opt's transformations.
Do we want to keep the DWARF sections? This should mostly only be useful when hooking into the process with a debugger, I think?
Do we want to actually use wasm-opt with a non-zero optimization level? The benefits are kind-of dubious; it only nets us something like ~30k compressed while being potentially risky.

cc @pepyakin

athei · 2022-09-23T06:55:43Z

Do we want to actually use wasm-opt with a non-zero optimization level? The benefits are kind-of dubious; it only nets us something like ~30k compressed while being potentially risky.

According to @Robbepop newer wasmi versions depend on these optimizations (the wasmi code itself). Apart from that running with O0 as a first step sounds sensible. We can change the level in a follow up?

koute · 2022-09-23T07:00:49Z

According to @Robbepop newer wasmi versions depend on these optimizations (the wasmi code itself). Apart from that running with O0 as a first step sounds sensible. We can change the level in a follow up?

Ah, yes, right, I remember that. In which case can we just switch to a wasm-gc-equivalent configuration for now (which should be safe enough to do as-is), and potentially bump the optimization level after we upgrade wasmi?

athei · 2022-09-23T07:12:10Z

A direct replacement of our current use of wasm-gc would be wasm-opt -O0 -g --strip-dwarf, which is only 3k bigger compressed for polkadot

But do we really want to keep the name section for a production build?

koute · 2022-09-23T07:19:28Z

A direct replacement of our current use of wasm-gc would be wasm-opt -O0 -g --strip-dwarf, which is only 3k bigger compressed for polkadot

But do we really want to keep the name section for a production build?

This is a good question. Being able to decode the symbols in case of an e.g. panic is something that is nice to have, and the size difference compressed is not that big. Personally I'd probably keep it, but I'm willing to be convinced otherwise, however in that case I'd insist on updating our release process so that fully unstripped runtime blobs also be posted in our releases alongside the .compact.compressed.wasm. I don't want to end up in a situation where I have to debug a runtime issue and will have no way of knowing what is what without reverse engineering the bytecode to figure out which function is which.

kripken · 2022-10-17T22:16:48Z

The unneeded memory section issue has been fixed in binaryen: WebAssembly/binaryen#5145

brson · 2022-10-17T23:07:08Z

The unneeded memory section issue has been fixed in binaryen: WebAssembly/binaryen#5145

I can probably backport this to Binaryen 110, which wasm-opt-rs is already forking for another backport, and make a point release of the wasm-opt crate. It might take me some time though as I have a busy week.

brson/wasm-opt-rs#105

brson · 2022-10-27T20:06:27Z

The just-published 0.110.2 contains the backport to remove the memories section.

koute · 2022-10-28T13:30:09Z

Thanks @brson!

I'll do one last check, and if everything's good this should be good to go.

athei · 2022-10-28T13:33:13Z

I updated the PR to the new version. Hopefully the CI will be green now.

koute · 2022-10-28T13:51:11Z

Looks like this might need a Polkadot companion to update the deps.

athei · 2022-10-28T14:35:48Z

Problem is that the color-eyre dependency used there depends on an older version of cc. We can't really resolve this issue while we are using this dependency.

brson · 2022-10-28T15:08:07Z

I don't quite understand what the CI is saying about the dependencies (cc is locked to 1.0.72, which doesn't appear to be something strictly required by backtrace/color-eyre), but I can probably downgrade wasm-opt-sys's cc dependency to the even older version that cxx_build depends on and make a point release.

koute

Okay, I did one final thorough check comparing the wasm-gc produced blob and wasm-opt produced one.

It looks good to me, although there's still one last difference: wasm-gc demangles Rust symbols in the name section while wasm-opt doesn't. This results in the wasm-gc-produced blob being slightly smaller as the demangled symbols are slightly shorter. Besides the slight size difference this doesn't make much of a difference though as wasmtime will automatically demangle the symbols anyway when producing backtraces, so it's not really critical, although it'd be nice to have I guess.

koute · 2022-10-28T15:20:50Z

@athei @brson I think it should be possible to merge this without any downgrading. I see this problem all the time when we bump some dependency in substrate and the polkadot job fails with a cargo error like this. I'm not entirely sure why this happens (it's probably because the Cargo.toml in substrate has a newer required patch version, but the polkadot's Cargo.lock is locked to an older version), but the way to fix it is to make a companion PR in polkadot and/or cumulus and just bump the affected dependency in their Cargo.lock, and then the CI should pass.

athei · 2022-10-29T09:26:05Z

@ggwpez @koute Please also approve the companions. Otherwise can't merge.

ggwpez · 2022-10-29T10:02:26Z

@ggwpez @koute Please also approve the companions. Otherwise can't merge.

Okay done. I will re-run all benches and then open another MR or push here if it is still open.

athei · 2022-10-30T10:09:38Z

bot merge

athei · 2022-10-30T10:12:55Z

@ggwpez @koute Please also approve the companions. Otherwise can't merge.

Okay done. I will re-run all benches and then open another MR or push here if it is still open.

Not sure if necessary. We run without any optimizations in this PR. As a follow up we try to enable to optimizations. This is when we need to do some serious checking.

koute · 2022-10-30T10:14:40Z

Indeed, this shouldn't change anything performance-wise since this is essentially a wasm-gc-equivalent config which doesn't do any extra optimizations.

* BlockId removal: refactor: ProofProvider (#12519) * BlockId removal: refactor: ProofProvider It changes the arguments of methods of `ProofProvider` trait from: block: `BlockId<Block>` to: hash: `&Block::Hash` This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292) * LightClientRequestHandler: excessive BlockIdTo bound removed * imports cleanup * formatting * args tyeps cleanup * registrar: Avoid freebies in provide_judgement (#12465) * evaluate repatriate reserved error in pallet identity * fix benchmarks * add repatriate reserved error test * benchmark fix * undo lock * include balance to use for benchmarks * rename test * Update frame/identity/src/benchmarking.rs * Update frame/identity/src/benchmarking.rs Co-authored-by: Bastian Köcher <git@kchr.de> * EPM: allow duplicate submissions (#12237) * allow for duplicate signed submissions * Fix a bunch of things, seems all good now * fmt * Fix * Update frame/election-provider-multi-phase/src/signed.rs Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com> * Update frame/election-provider-multi-phase/src/signed.rs Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com> * add migratin * fmt * comment typo * some review comments * fix bench Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com> Co-authored-by: Ross Bulat <ross@parity.io> * CI check against Rust feature bleed (#12341) * CI check for rust feature bleed Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Cargo not available Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Handle missing programs Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Check for deps Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add doc Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use correct CI image Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Remove --offline Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Install cargo-workspaces Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Remove cargo-workspaces dep Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix try-runtime feature Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix features Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix features Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix more features... Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use pipeline-script Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * 🤡 Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Make stupid change to test the CI Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * This reverts commit ad2746aa117fa7cb473521113a9bec89aaf26484. * Use correct branch Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Allow failure Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Make stupid change to test the CI Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Revert "Make stupid change to test the CI" This reverts commit 16ec00e1675c7ec57c988315549ff71e832a3093. Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * contracts: Decrease the interation count on slow benchmarks (#12526) * Decrease amount of benchmark iterations for long slow ones * ".git/.scripts/bench-bot.sh" pallet dev pallet_contracts Co-authored-by: command-bot <> * BlockId removal: refactor: Finalizer (#12528) * BlockId removal: refactor: Finalizer It changes the arguments of methods of `Finalizer` trait from: block: `BlockId<Block>` to: hash: `&Block::Hash` This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292) * minor corrections * failing test corrected * minor rework * BlockId removal: refactor: BlockImportOperation+Bknd::finalize_block (#12535) * BlockId removal: refactor: BlockImportOperation+Bknd::finalize_block It changes the arguments of methods of `BlockImportOperation` trait from: block: `BlockId<Block>` to: hash: `&Block::Hash` `Backend::finalize_block` was also changed. This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292) * Review suggestion applied thx to @davxy * trigger CI job * Remove multiple DHTs support from `Discovery` (#12524) * CI: Enable debug assertions in Wasmer sandbox test (#12540) * CI: Enable debug assertions in Wasmer sandbox test Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix feature dependant import Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Force base weights to be the minimum only when the intercept is negative (#12482) * Force base weights to be the minimum only when the intercept is negative; emit minimum execution times * Add an `assert` making sure the intercept is zero when it's supposed to be zero * Fix template Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * ".git/.scripts/bench-bot.sh" pallet dev pallet_assets * ".git/.scripts/bench-bot.sh" pallet dev pallet_uniques Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: command-bot <> * Add `DefensiveTruncateFrom` (#12515) * Add DefensiveTruncateFrom Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add tests Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix tests Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Map_err in preimage Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Map_err in beefy Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Make test dependant in debug-assertions Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: parity-processbot <> * Refactor service tests in `sc-network` (#12517) * Refactor service tests in `sc-network` Create a separate directory for the tests and move common network instantion related code to `mod.rs` from where it can be used by both service and chainsync tests. Use the builder pattern when creating the `TestNetwork` object to reduce code duplication between the test files. * Update client/network/src/service/tests/mod.rs Co-authored-by: Dmitrii Markin <dmitry@markin.tech> Co-authored-by: Dmitrii Markin <dmitry@markin.tech> Co-authored-by: parity-processbot <> * Actually fix major sync detection (#12114) * Actually fix major sync detection * Introduce `SyncState::Importing` state * Add target to SyncState enum variants and add `is_major_syncing` method on it * Remove unnecessary duplicated `best_seen_block` from `SyncState` struct * Revert "Remove unnecessary duplicated `best_seen_block` from `SyncState` struct" This reverts commit bb8abd458c939881c049f69d59f3acba47c97c5c. * Add missing `websocket` feature to `libp2p` Co-authored-by: parity-processbot <> * BlockId removal: refactor: Backend::begin_state_operation (#12541) It changes the arguments of `Backend::begin_state_operation` from: block: `BlockId<Block>` to: hash: `&Block::Hash` This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292) * use headers on templates (#12546) * Make `NetworkService` callable for `ChainSync` (#12542) Introduce a middleware called `NetworkServiceProvider` which the `ChainSync` can use to communicate with `NetworkService`. `ChainSync` is given a `NetworkServiceHandle` which it uses to call `NetworkServiceProvider` which then dispatches the calls to `NetworkService` on behalf of `ChainSync`. This change will allow `ChainSync` to disconnect and report peers and in the future it'll be possible to send requests and notifications through the `NetworkServiceProvider`. `NetworkServiceProvider` is needed only until the `ChainSync` object has been removed from `Protocol`. After that, a normal `NetworkService` handle can be passed onto `ChainSync` and these changes can be deprecated. Co-authored-by: parity-processbot <> * Base Kademlia protocol name on genesis hash and fork ID (#12545) * contracts: Allow indeterministic instructions off-chain (#12469) * Allow indetermistic instructions off-chain * Apply suggestions from code review Co-authored-by: Sasha Gryaznov <hi@agryaznov.com> * fmt Co-authored-by: Sasha Gryaznov <hi@agryaznov.com> * Normalize keystore type and its usage across tests (#12553) * Normalize keystore type and usage across tests * Extract peer index from array index * Update template pallet to latest enum syntax (#12552) * feat: generalize some functions in sp-trie (#12376) * feat: add to_memory_db to StorageProof * feat: add iter method and generalize iter_nodes * fmt * feat: generalize `encode_compact` like `decode_compact`, add to_compact_proof to StorageProof * fix to_compact_proof * improve by suggestions * improve by suggestions Co-authored-by: Bastian Köcher <git@kchr.de> * Make Multisig Pallet Bounded (#12457) * Bounded multisig * just use u32 Co-authored-by: parity-processbot <> * Fix error during build: failed to run custom build command for sc-network-bitswap (#12494) * Update `pallet-multisig` benches (#12558) * Typo Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * ".git/.scripts/bench-bot.sh" pallet dev pallet_multisig * remove functions * ".git/.scripts/bench-bot.sh" pallet dev pallet_multisig Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: command-bot <> Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> * [ci] cargo-check-benches against different base branches (#12557) * Fix typo (#12571) * replaced println with log Closes #12338 (#12348) * replaced println with log Closes #12338 * fixed println * Apply suggestions from code review Co-authored-by: Bastian Köcher <git@kchr.de> * Update utils/frame/benchmarking-cli/src/pallet/command.rs * Apply suggestions from code review * ".git/.scripts/fmt.sh" 1 Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: command-bot <> * Aura: Adds some compatibility mode to support old chains (#12492) * Aura: Adds some compatibility mode to support old chains In https://github.com/paritytech/substrate/pull/9132 we changed the way how we get the authorities from the runtime. Before this mentioned pr we would call `initialize_block` before calling the authorities runtime function. The problem with this was that when you have a block X that would switch the authority set, it would already be signed by an authority of the new set. This was wrong, as a block should only be signed by the current authority set. As this change is a hard fork, this pr brings back the possibility for users that have a chain running with this old logic to upgrade. They will need to use: ``` CompatibilityMode::UseInitializeBlock { until: some_block_in_the_future } ``` Using this compatibility mode will make the node behave like the old nodes, aka calling `initialize_block` before doing the actual runtime call to `authorities`. Then when the given `until` block is being build/imported the node switches to the new behaviour of not calling `initialize_block` before. This is a hard fork, so the `until` block should be chosen wisely as a point where all nodes in the network have upgraded. * Fixes * Make docs ready * Update client/consensus/aura/src/lib.rs Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> * Update client/consensus/aura/src/lib.rs Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> * Update client/consensus/aura/src/lib.rs Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> * FMT Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> * bump ed25519-zebra; fixes `full_crypto` feature flag in `no_std` (#12576) * Utility: add more tests for batch/batchAll/forceBatch (#12506) * Utility: add more tests for batch/batchAll/forceBatch * remove unnecessary * batch works with council * add more tests * better call examples * shorter code * Update frame/utility/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update frame/utility/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update frame/utility/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * update TestBaseCallFilter * fix * fix? * fix Co-authored-by: Bastian Köcher <git@kchr.de> * Treat near-zero intercept values as zero when calculating weights (#12573) * Treat near-zero intercept values as zero when calculating weights * Simplify comparison Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update contract weights Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * [Enhancement] Convert fast-unstake to use StakingInterface, decouplin… (#12424) * [Enhancement] Convert fast-unstake to use StakingInterface, decoupling it from Staking * Update primitives/staking/src/lib.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Update primitives/staking/src/lib.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * fix validator comment * remove todo * ideas from Kian (#12474) Co-authored-by: kianenigma <kian@parity.io> * Rename StakingInterface -> Staking for nomination-pools * Staking fixes * StakingInterface changes * fix fast-unstake * fix nomination-pools * Fix fast-unstake tests * Fix benches for fast-unstake * fix is_unbonding * fix nomination pools * fix node code * add mock comments * fix imports * remove todo * more fixes * more fixes * bench fixes * more fixes * more fixes * import fix * more fixes * more bench fix * refix * refix * Update primitives/staking/src/lib.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * is_unbonding returns a result * fix * review fixes * more review fixes * more fixes * more fixes * Update frame/fast-unstake/src/benchmarking.rs Co-authored-by: Squirrel <gilescope@gmail.com> * remove redundant CurrencyBalance from nom-pools * remove CB * rephrase * Apply suggestions from code review * Update frame/nomination-pools/src/tests.rs * finish damn renamed * clippy fix * fix Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Co-authored-by: kianenigma <kian@parity.io> Co-authored-by: parity-processbot <> Co-authored-by: Squirrel <gilescope@gmail.com> * nomination-pools: allow pool-ids to be reused (#12407) * initial * logic check * do_create * cargo fmt * fixes * ensure_signed * update * cargo fmt * remove unused import * WIP: Replace `wasm-gc` with `wasm-opt` (#12280) * Use wasm-opt on runtime * Optimize for size * Simplify fn compact_wasm_file * Run a lighter pass for non production builds * Disable optimizations and keep name section * Update wasm-opt * Remove dward sections * Update wasm-opt * Update wasm-opt * Use minimum_nominator_bond instead of nominator_bond (#12585) Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * BlockId removal: refactor: Backend::append_justification (#12551) * BlockId removal: refactor: Backend::append_justification It changes the arguments of `Backend::append_justification` from: block: `BlockId<Block>` to: hash: `&Block::Hash` This PR is part of `BlockId::Number` refactoring analysis (paritytech/substrate#11292) * Error message improved Co-authored-by: Adrian Catangiu <adrian@parity.io> * single error message in beefy::finalize * println removed Co-authored-by: Adrian Catangiu <adrian@parity.io> * fix some typos (#12584) * Added test for Client::block (#12590) * client/beefy: fix incorrect BEEFY justifications import test (#12593) Signed-off-by: Adrian Catangiu <adrian@parity.io> * BlockId removal: refactor: Backend::body (#12587) It changes the arguments of `Backend::body` method from: `BlockId<Block>` to: `&Block::Hash` This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292) Co-authored-by: parity-processbot <> * fix: construct_runtime multiple features (#12594) * fix: construct_runtime multiple features * Update frame/support/procedural/src/construct_runtime/mod.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Fix fungible unbalanced trait (#12569) * Fix fungible unbalanced trait * Add simple decrease_balance test Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix decrease_balance_at_most * Fix decrease_balance_at_most in fungibles * Rename free_balanceto balance_on_free * Use reducible_balance instead of balance_on_free Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * [ci] allow fail skip-if-draft job (#12604) * BlockId removal: refactor: Backend::justifications (#12602) * BlockId removal: refactor: Backend::justifications It changes the arguments of `Backend::justifications` method from: `BlockId<Block>` to: `&Block::Hash` This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292) * trigger CI job * trigger CI job * bug fix * match -> if Co-authored-by: Adrian Catangiu <adrian@parity.io> Co-authored-by: Adrian Catangiu <adrian@parity.io> * use associated iterator types for InspectEnumerable (#12389) * use associated iterator types for InspectEnumerable * Update frame/uniques/src/impl_nonfungibles.rs Co-authored-by: parity-processbot <> Co-authored-by: Bastian Köcher <git@kchr.de> * Add map and try_map methods (#12581) Co-authored-by: parity-processbot <> * stabilize 4 storage host funcs (#12611) * Collective benchmark respects DefaultVote configuration (#12612) * Collective benchmark respects DefaultVote configuration * ".git/.scripts/bench-bot.sh" pallet dev pallet_collective Co-authored-by: command-bot <> * BlockId removal: refactor: Backend::block_indexed_body (#12609) * BlockId removal: refactor: Backend::block_indexed_body It changes the arguments of `Backend::block_indexed_body` method from: `BlockId<Block>` to: `&Block::Hash` This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292) * trigger CI job * Introduce DefensiveMin and DefensiveMax (#12554) * traits for defensive min and defensive max * defensive min and strict min with tests * defensive max and strict max with tests * include docs * implement partial ord on defensive min and max * Update frame/support/src/traits/misc.rs Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * wrap lines * Fix traits Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update frame/support/src/traits/misc.rs * Update frame/support/src/traits/misc.rs Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: parity-processbot <> * pallet-sudo: add `CheckOnlySudoAccount` signed extension (#12496) * pallet-sudo: add `CheckSudoKey` signed extension Signed-off-by: koushiro <koushiro.cqx@gmail.com> * Rename CheckSudoKey => CheckOnlySudo Signed-off-by: koushiro <koushiro.cqx@gmail.com> * Rename extension name and Add some docs * Apply review suggestions * Update frame/sudo/src/extension.rs Co-authored-by: Jegor Sidorenko <5252494+jsidorenko@users.noreply.github.com> * Update frame/sudo/src/extension.rs Signed-off-by: koushiro <koushiro.cqx@gmail.com> Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: Jegor Sidorenko <5252494+jsidorenko@users.noreply.github.com> * Move Throughput into `sc-sysinfo` (#12368) * move Throughput to sc-sysinfo * replace u64 * fix in tests * change Throughput * refactored Throughput * fixes * moved tests & fixes * custom serializer * note * fix serializer * forgot to remove * deserialize * functioning deserialization :) * try to make clipply happy * Serialize as function * test HwBench * rename * fix serialization * deserialize as function * unused import * move serialize/deserialize * don't serialize none * remove nonsense * remove nonsense comment :P * fixes * remove all the todos * return enum * fixes * fix nit * improve docs & readability * Update client/sysinfo/src/sysinfo.rs Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * fix all the nits * rename * fix * Update client/sysinfo/src/sysinfo.rs Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * remove unit from serialization * Update utils/frame/benchmarking-cli/src/machine/hardware.rs Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Bump `k256` from `0.10.4` to `0.11.4` (#12085) * Bump `k256` from `0.10.4` to `0.11.4` * Update Cargo.lock * Update Co-authored-by: Bastian Köcher <info@kchr.de> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Guard some invalid node for proof decoding. (#12417) * Guard some invalid node for proof decoding. * only forbid bitmap with no children. * change format * scale error. * small test Co-authored-by: parity-processbot <> * Bump regex from 1.5.5 to 1.6.0 (#12117) Bumps [regex](https://github.com/rust-lang/regex) from 1.5.5 to 1.6.0. - [Release notes](https://github.com/rust-lang/regex/releases) - [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md) - [Commits](https://github.com/rust-lang/regex/compare/1.5.5...1.6.0) --- updated-dependencies: - dependency-name: regex dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: parity-processbot <> * Make `--db` case insensitive again (#12630) This was broken in the switch to Clap v4. * txpool: enactment state forced update (#12632) * txpool: enactment state forced update When `tree_route` computation fails, we still need to update the `enactment_state` to be aligned with last known finalized/best block. We do not execute enactment phase of maintain procedure, but we do update the state. * error -> debug * test added * Add pallet dev mode (#12536) * stub for construct_dev_runtime! * revert * stub for dev_mode proc macro * preliminary docs for pallet::dev_mode (attribute) proc macro * add dev_mode to pallet_macros module * add docs item for dev_mode to frame_support * parsing of #[pallet(dev_mode)] * strip out dev_mode stub since it will be an arg for pallet instead * make pallet Def struct aware of dev mode * WIP * revert changes to call.rs * pass dev_mode to pallet parsing code * auto-specify default weights when in dev mode if not specified * add proof / expect for syn::parse in dev mode weight processing * set all storages to unbounded when in dev mode * just use 0 Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> * add invalid pallet arg test * add passing dev mode pallet test * add test confirming that dev mode features only work in dev mode * cargo fmt + clean up * bump CI * fix pallet ui test * add docs for dev mode * add warning about using dev mode in production circumstances * remove comment about no other attributes being supported * fix unneeded assignment * make warning more explicit * more explicit warning about using dev mode in production * simpler assignment for dev_mode boolean Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * add note about MEL requirement Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * add comment specifying why weights can be omitted in example Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * tweak wording of comments * bump ci Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * BlockId removal: &Hash to Hash (#12626) It changes &Block::Hash argument to Block::Hash. This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292) * Do not update peer information if ancestor search is in progress (#12631) * Do not update peer information if ancestor search is in progress If block announcement is received from a peer while ancestor search for that same peer is still in progress, do not update the peer's best hash and best number as that causes the ancestor search to yield different information from what was expected and can cause, for example, a fork of lower height not be be downloaded. * Block until peers are in sync * Pipeline with ci image with rust 1.65 (#12628) * Pipeline with ci image with rust 1.65 * fix tests * use image with sha * `sp_trie::Recorder`: Fix recording the same key for different tries (#12636) With `StateVersion::V1` values over a certain size are not inlined and being put into the backend with their own hash. When accessing a value in the trie with a recorder, we check if the value is maybe already recorded and thus, we can check the cache. To check if a value is already recorded, we use the key of the value to differentiate them. The problem is when there are multiple tries, like multiple child tries that all have different values under the same key. Before this pull request we didn't have differentiated for which trie we already had recorded a (key, value) pair. This is now done by also taking the storage root into account in the recorder to differentiate the different (key, value) pair in the tries. * Fix UI tests (#12642) * `payment_queryInfo`: Make it work with `WeightV2` (#12633) * `payment_queryInfo`: Make it work with `WeighV2` The runtime api for querying the payment info depends on the `Weight` type and broke for old runtimes that still use the `WeighV1`. This pull requests fixes this by: 1. Bumping the version of the runtime api. 2. Making the node side code use the correct runtime api function depending on the version of the runtime api. 3. Make the RPC always return `WeighV1`. Users of the api should switch to `state_call` and decide based on the version of the runtime api which `Weight` type is being returned. * Fix tests * Review comment * State-db refactoring (#12239) * Prune discarded blocks immediately * state-db refactoring part 1 * Some renames * Get rid of pending state * Revert "Prune discarded blocks immediately" This reverts commit 790f54038b52ff379a573ed0806f38d09af098ec. * Cleanup * Make clippy happy * Minor changes * Remove duplicate units (#12634) Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add batching to fast-unstake pallet (#12394) * implement a brand new batch with all tests passing. * fix benchmarks as well * make benchmarks more or less work * fix migration * add some testing * Update frame/fast-unstake/src/benchmarking.rs Co-authored-by: Roman Useinov <roman.useinov@gmail.com> * review comments * some fixes * fix review comments * fix build * fmt * fix benchmarks * fmt * update Co-authored-by: Roman Useinov <roman.useinov@gmail.com> * [ci] Use ci-linux:production image in ci (#12648) * New Weights for All Pallets (#12325) * new weights for everything * fix * fmt * new batch * fmt * new batch * Update run_all_benchmarks.sh * add headers * update weights * Update lib.rs * block and extrinsic weight * Remove partial key size limit from trie codec (#12566) * remove size limit from trie codec * test previous upper limit is not enforced anymore * fmt * restore test * Keep the same name (#12616) Co-authored-by: x <x@localhost.localdomain> * Do not finalize parent twice (#12653) If the parent block is alread finalized, we don't need to do this again. * update paritydb and remove dev deps on rocksdb (#12641) * update paritydb and remove dev deps on rocksdb * feature rocksdb for node testing * feature decl in node-bench * revert change to rocksdb inclusion logic * Update bin/node/bench/Cargo.toml Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: parity-processbot <> Co-authored-by: Bastian Köcher <git@kchr.de> * Epoch-Changes tree pruning was lagging by one epoch (#12567) * Remove all not required nodes from the epoch-changes tree Some outdated nodes were left there because of the predicate * Test to excercise the fix * Add a fork on genesis to the test * Fix typo in comments * Bound Election and Staking by MaxActiveValidators (#12436) * bounding election provider with kian * multi phase implement bounded election provider * election provider blanket implementation * staking compiles * fix test for election provider support * fmt * fixing epmp tests, does not compile yet * fix epmp tests * fix staking tests * fmt * fix runtime tests * fmt * remove outdated wip tags * add enum error * sort and truncate supports * comment * error when unsupported number of election winners * compiling wip after kian's suggestions * fix TODOs * remove,fix tags * ensure validator count does not exceed maxwinners * clean up * some more clean up and todos * handle too many winners * rename parameter for mock * todo * add sort and truncate rule if there are too many winners * fmt * fail, not swallow emergency result bound not met * remove too many winners resolution as it can be guaranteed to be bounded * fix benchmark * give MaxWinners more contextual name * make ready solution generic over T * kian feedback * fix stuff * Kian's way of solvign this * comment fix * fix compile * remove use of BoundedExecution * fmt * comment out failing integrity test * cap validator count increment to max winners * dont panic * add test for bad data provider * Update frame/staking/src/pallet/impls.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * fix namespace conflict and add test for onchain max winners less than desired targets * defensive unwrap * early convert to bounded vec * fix syntax * fmt * fix doc * fix rustdoc * fmt * fix maxwinner count for benchmarking * add instant election for noelection * fmt * fix compile * pr feedbacks * always error at validator count exceeding max winners * add useful error message * pr comments * import fix * add checked_desired_targets * fmt * fmt * fix rust doc Co-authored-by: parity-processbot <> Co-authored-by: kianenigma <kian@parity.io> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Update some dependencies to prune duplicated crates with different version (#12560) * sc-client-babe/sp-arithmetic-fuzzer: update num-bigint and num-rational to v0.4 * update lru 0.7.5 ==> v0.8.1 * pallet-example-offchain-worker: update lite-json v0.1.3 ==> v0.2.0 * update hyper 0.14.16 ==> 0.14.20, num-fromat 0.4.0 ==> 0.4.3 * pallet-mmr: update ckb-merkle-mountain-range v0.3.2 ==> v0.5.2 * update handlebars v4.2.2 ==> v4.3.5 * `runtime_cache_size` must always be at least 1 Signed-off-by: koushiro <koushiro.cqx@gmail.com> * default cache size with .min(1) Signed-off-by: koushiro <koushiro.cqx@gmail.com> * update hyper 0.14.20 ==> 0.14.22 Signed-off-by: koushiro <koushiro.cqx@gmail.com> * update lru 0.8.0 ==> 0.8.1 Signed-off-by: koushiro <koushiro.cqx@gmail.com> * Apply suggestions from code review * Apply suggestions from code review * Fix Cargo.lock Signed-off-by: koushiro <koushiro.cqx@gmail.com> Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: Bastian Köcher <info@kchr.de> * Consolidate and deduplicate MMR API methods (#12530) * histor. batch proof: make best block arg optional * correct testing range * make generate_batch_proof stub for historical * merge generate_{historical_}batch_proof functions * merge generate_{batch_}proof functions * merge verify_{batch_}proof functions * merge verify_{batch_}proof_stateless functions * remove {Leaf}Proof Not utilized by API anymore, so superfluous. Removal consistent with prior changes to just use "batch" proof API. * rename BatchProof->Proof no need to qualify if only one universal proof type. * cleanup * expose verify_proof rpc api * document verify_proof * expose verify_proof_stateless rpc api * add optional BlockHash to mmr_root rpc api * fixup! expose verify_proof rpc api * fix documentation phrasing Co-authored-by: Adrian Catangiu <adrian@parity.io> * documentation grammar Co-authored-by: Adrian Catangiu <adrian@parity.io> * define mmr error msgs together with error enum Co-authored-by: Serban Iorga <serban@parity.io> * fixup! define mmr error msgs together with error enum * map decoding errors to CallError::InvalidParams Co-authored-by: Serban Iorga <serban@parity.io> * fixup! map decoding errors to CallError::InvalidParams Co-authored-by: Adrian Catangiu <adrian@parity.io> Co-authored-by: parity-processbot <> Co-authored-by: Serban Iorga <serban@parity.io> * Bump ss58-registry from 1.29.0 to 1.34.0 (#12659) Bumps [ss58-registry](https://github.com/paritytech/ss58-registry) from 1.29.0 to 1.34.0. - [Release notes](https://github.com/paritytech/ss58-registry/releases) - [Changelog](https://github.com/paritytech/ss58-registry/blob/main/CHANGELOG.md) - [Commits](https://github.com/paritytech/ss58-registry/compare/v1.29.0...v1.34.0) --- updated-dependencies: - dependency-name: ss58-registry dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Add `CreateOrigin` to Assets Pallet (#12586) * add CreateOrigin to Assets pallet * fix asset-tx-payment test * use AccountId > u64 in test * Update frame/assets/src/benchmarking.rs Co-authored-by: Jegor Sidorenko <5252494+jsidorenko@users.noreply.github.com> Co-authored-by: Jegor Sidorenko <5252494+jsidorenko@users.noreply.github.com> * `sp-runtime`: make `parity-util-mem` dependency optional (#12657) * `sp-runtime`: make `parity-util-mem` dependency optional * Use default-features = false for sp-runtime in sp-keyring * Remove parity-util-mem from sp-core * Cargo.lock * Restore default-features for keyring dependency * GrandpaJustification: Feature gate `Debug` (#12664) The grandpa crate is deriving `Debug` only when the `std` feature is enabled. `RuntimeDebug` can be forced to derive `Debug` also in `no_std` and that doesn't work together. So, we should feature gate `Debug` on `no_std`. * More testing and fuzzing and docs for pools (#12624) * move pools fuzzing to hongfuzz * merge more small fixes * fix all tests * Update frame/nomination-pools/fuzzer/src/call.rs Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com> * remove transactional * fmt * fix CI * fmt * fix build again * fix CI Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com> * Remove `sp_tasks::spawn` API and related code + host functions (#12639) * Remove `sp_tasks::spawn` API and related code * Remove `RuntimeTasks::{spawn, join}` host functions * remove unused * Remove a few more tests that I forgot to remove Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> * Contracts pallet: Bump Runtime API (#12677) * Fix typo (#12680) * Move `WeightCounter` to `sp-weights` (#12603) * Move WeightCounter to sp_weights Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Rename to WeightMeter and test Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix pallet-scheduler for new usage Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update primitives/weights/src/weight_meter.rs Co-authored-by: David <dvdplm@gmail.com> * More tests for can_accrue Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Clippy Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Remove defensive_accrue and fixup consumed_ratio I dont think there is a good use-case for defensive_accrue without saturation. Only in tests maybe, will remove for now until we have a use-case. Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Test Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: David <dvdplm@gmail.com> Co-authored-by: Gavin Wood <gavin@parity.io> * Allow other pallets to check asset ids. (#12666) * Make it easier for other pallets to check asset ids. * Avoid boxing * cargo fmt * derive type info for some grandpa types (#12683) * Safe TreeRoute constructor (#12691) * Safe TreeRoute constructor * Remove test duplicate * Better tree route error info * New `root_testing` pallet (#12451) * Move fill_block to RootOffences * docs * new pallet * new line * fix * Update frame/root-testing/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update frame/root-testing/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update bin/node/runtime/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update frame/root-testing/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update frame/root-testing/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update frame/root-testing/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * fixes * problem solved * revert * fix dependency * hopefully making the CI happy * ... * dummy call * remove dummy * fix warning Co-authored-by: Bastian Köcher <git@kchr.de> * [ci] Add DAG for build-rustdoc and check-dependent-project (#12687) * [ci] Debug ci runner * try gha * allow mac jobs fail * add dags * install protoc * fix protobuf name * fix dags * remove allow fail for mac jobs * remove gha * adjust cargo-check-macos * Collective: Benchmark with greater `MaxProposals` (#12454) * Collective: Benchmark with greated * fix * remove bs * id_to_remark_data * fix * remove hardcoded * clean up * simplify * questionable renaming * better variable name * better solution * no need for large length * better solution * Update frame/collective/src/benchmarking.rs Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * fix * test * remove test Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * [ci] fix buildah for publishing docker (#12703) * Make public is_passing and ReferendumStatus (#12667) * init * clean * remove manual getter for ReferendumStatus in favor of changing pub crate to pub for ReferendumStatus DecidingStatus Deposit types * rm status getters because fields are pub now * Asset Pallet: Support repeated destroys to safely destroy large assets (#12310) * Support repeated destroys to safely destroy large assets * require freezing accounts before destroying * support only deleting asset as final stage when there's no assets left * pre: introduce the RemoveKeyLimit config parameter * debug_ensure empty account in the right if block * update to having separate max values for accounts and approvals * add tests and use RemoveKeyLimit constant * add useful comments to the extrinsics, and calculate returned weight * add benchmarking for start_destroy and finish destroy * push failing benchmark logic * add benchmark tests for new functions * update weights via local benchmarks * remove extra weight file * Update frame/assets/src/lib.rs Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com> * Update frame/assets/src/types.rs Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com> * Update frame/assets/src/lib.rs Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com> * effect some changes from codereview * use NotFrozen error * remove origin checks, as anyone can complete destruction after owner has begun the process; Add live check for other extrinsics * fix comments about Origin behaviour * add AssetStatus docs * modularize logic to allow calling logic in on_idle and on_initialize hooks * introduce simple migration for assets details * reintroduce logging in the migrations * move deposit_Event out of the mutate block * Update frame/assets/src/functions.rs Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com> * Update frame/assets/src/migration.rs Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com> * move AssetNotLive checkout out of the mutate blocks * rename RemoveKeysLimit to RemoveItemsLimit * update docs * fix event name in benchmark * fix cargo fmt. * fix lint in benchmarking * Empty commit to trigger CI * Update frame/assets/src/lib.rs Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update frame/assets/src/lib.rs Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update frame/assets/src/functions.rs Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update frame/assets/src/functions.rs Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update frame/assets/src/functions.rs Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update frame/assets/src/lib.rs Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update frame/assets/src/functions.rs Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * effect change suggested during code review * move limit to a single location * Update frame/assets/src/functions.rs Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com> * rename events * fix weight typo, using rocksdb instead of T::DbWeight. Pending generating weights * switch to using dead_account.len() * rename event in the benchmarks * empty to retrigger CI * trigger CI to check cumulus dependency * trigger CI for dependent cumulus * Update frame/assets/src/migration.rs Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * move is-frozen to the assetStatus enum (#12547) * add pre and post migration hooks * update do_transfer logic to add new assert for more correct error messages * trigger CI * switch checking AssetStatus from checking Destroying state to checking live state * fix error type in tests from Frozen to AssetNotLive * trigger CI * change ensure check for fn reducible_balance() * change the error type to Error:<T,I>::IncorrectStatus to be clearer * Trigger CI Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com> Co-authored-by: parity-processbot <> Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * `seal_reentrant_count` returns contract reentrant count (#12695) * Add logic, test, broken benchmark * account_entrance_count * Addressing comments * Address @agryaznov's comments * Add test for account_entrance_count, fix ci * Cargo fmt * Fix tests * Fix tests * Remove delegated call from test, address comments * Minor fixes and indentation in wat files * Update test for account_entrance_count * Update reentrant_count_call test * Delegate call test * Cargo +nightly fmt * Address comments * Update reentrant_count_works test * Apply weights diff * Add fixture descriptions * Update comments as suggested * Update reentrant_count_call test to use seal_address * add missing code * cargo fmt * account_entrance_count -> account_reentrance_count * fix tests * fmt * normalize signatures Co-authored-by: yarikbratashchuk <yarik.bratashchuk@gmail.com> * Assets Pallet: reintroduce fungibles::Destroy trait (#12708) * update docs formatting * reintroduce the destroy trait * copy changes from original PR * remove witness * Trigger CI * Trigger CI * release `sp-core 7.0.0` and `sp-runtime 7.0.0` (#12599) * chore(release): sp-core v7.0.0 * chore(release): sp-runtime v7.0.0 * fix bad merge * Release `sp-keyring` and `pallet-contracts-primitives` `7.0.0` (#12716) * Bump sp-keyring * Bump pallet-contracts-primitives * Cargo.lock * Fix `cargo check` for `pallet-contracts-proc-macro` (#12706) * fix `cargo check` for pallet-contracts-proc-macro * add test for cargo-check of pallet-contracts-proc-macro * remove cargo-check-contracts-proc-macro https://github.com/paritytech/substrate/pull/12706/files#r1022783937 * [ci] Improve pipeline stopper (#12717) * [ci] Improve pipeline stopper * break test-linux-stable 1/3 * break test-linux-stable 2/3 * break test-linux-stable 3/3 * break cargo-check-benches 1/2 * break cargo-check-benches 2/2 * fix benches * sc-chainspec: Switch to `assimilate_storage` (#12720) Before it was using `build_storage` and `assimilate_storage` was returning an error. However, there was no real reason for `assimilate_storage` to return an error. This pr implements `assimilate_storage` and uses the default `build_storage` of the trait. * [Cleanup] Remove obsolete event from fast-unstake (#12725) Trivial, just removing unused code. * [Fix] Deposit for fast-unstake has to be define as pallet::constant (#12729) Fixes https://github.com/paritytech/substrate/issues/12618 * Add event testing example to pallet template (#12722) Add an example of how to test for events into the example pallet. Right now, the information is pretty hard to find without looking into pallet tests or finding some particular posts on the stackoverflow. * Remove the `wasmtime` feature flag (#12684) * Remove the `wasmtime` feature flag * rustfmt * Fix the light client protocol protobuf schema (#12732) * Fix the light client protocol protobuf schema * Add another test * Remove unused protobuf struct * Ok you have to use the nightly rustfmt apparently * Update template to remove clippy warnings (#12670) * Update template to remove clippy warnings * ".git/.scripts/bench-bot.sh" pallet dev pallet_lottery * Update templates from child project This should remove clippy warnings on generated files * Update after review * Update frame-weight-template.hbs Commit suggestion * ".git/.scripts/bench-bot.sh" pallet dev pallet_lottery * Rerun linter on linked project Updates from child project * ".git/.scripts/bench-bot.sh" pallet dev pallet_lottery Co-authored-by: command-bot <> * Check all crates (#12709) * check all crates individually It's relevant to check workspace crates individually because otherwise their compilation problems due to feature misconfigurations won't be caught, as exemplified by https://github.com/paritytech/substrate/issues/12705 * adapt to lack of multiple macos runners https://github.com/paritytech/substrate/pull/12709#discussion_r1022868752 * fix cancel-pipeline-cargo-check-each-crate-macos * fix cargo-check-each-crate-macos again * time command execution * fix YAML anchors * add explanation for rounding division * ensure the minimum of one crate per group * collect artifacts for pipeline stopper * revert hardcoded crates_per_group * re-add crates_per_group=1 * client/beefy: persist voter state (#12712) * client/beefy: prepare worker for persisting state * client/beefy: persist voter state * client/beefy: initialize persistent state * client/beefy: try to vote from the very beginning Now that voter is initialized from persistent state, it makes sense that it can attempt voting right away. This also helps the genesis case when we consider block `One` as mandatory. * client/beefy: add tests for voter state db * client/beefy: persist voter state as soon as initialized * client/beefy: make sure min-block-delta is at least 1 * client/beefy: persist state after voting Persist state after handling self vote to avoid double voting in case of voter restarts. * client/beefy: persist state after handling mandatory block vote For mandatory blocks we want to make sure we're not losing votes in case of crashes or restarts, since voter will not make further progress without finalizing them. * frame/beefy: use GENESIS_AUTHORITY_SET_ID on pallet genesis * client/beefy: initialize voter at either genesis or last finalized To guarantee unbroken chain of mandatory blocks justifications, voter will always resume from either last BEEFY-justified block or `pallet-beefy` genesis, whichever is more recent. Initialization walks back the chain from latest GRANDPA finalized block looking for one of the above. Along the way, it also records and enqueues for processing any BEEFY mandatory blocks that have been already GRANDPA finalized but not BEEFY finalized. * client/beefy: decouple voter init from aux db state load * client/beefy: fix voter init tests * remove debug prints * gadget future must be type () * fix init from last justification Signed-off-by: Adrian Catangiu <adrian@parity.io> * [Fix] Get target count from TargetList instead of storage (#12748) Co-authored-by: parity-processbot <> * Move block/state/warpc sync requests/responses to `ChainSync` (#12739) * Move block/state/warpc sync requests/responses to `ChainSync` * Apply suggestions from code review Co-authored-by: Bastian Köcher <git@kchr.de> * Apply review suggestions * cargo-fmt + doc fix * Fix tests Co-authored-by: Bastian Köcher <git@kchr.de> * perf: generate_initial_session_keys: load runtime only if its relevant (#12651) * perf: generate_initial_session_keys: load runtime only if its relevant * apply review suggestion * Update primitives/session/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: Bastian Köcher <git@kchr.de> * Prevent epochs pruning while finalizing blocks on epoch 0 (#12758) * Prevent epochs pruning while on epoch 0 * return error instead of expect in `feasibility_check` (#12745) * Update lib.rs * make defensive * fmt * fix batching migration * fix * fix Co-authored-by: parity-processbot <> * BEEFY: optimize voter event loop for fewer 'active' wakeups (#12760) * client/beefy: remove high-freq network events from main loop Network events are many and very frequent, remove the net-event-stream from the main voter loop and drastically reduce BEEFY voter task 'wakeups'. Instead have the `GossipValidator` track known peers as it already has callbacks for that coming from `GossipEngine`. Signed-off-by: acatangiu <adrian@parity.io> * Sort crates before splitting them into groups (+ some improvements) (#12755) * sort crates before splitting them into groups this is useful so that crates always get routed to a specific group for a given version of the source code, which means that jobs for each batch can be reliably retried individually * more verbose output * misc improvements * put uniq after sort uniq filters by adjacent lines * shellcheck * rm useless backlashes * handle edge case of no crates detected * contracts: Replace `sp-sandbox` and `wasmi-validation` by newest wasmi (#12501) * Replace sp-sandbox and wasmi-validation by just wasmi * ".git/.scripts/bench-bot.sh" pallet dev pallet_contracts * Re-check original code on re-instrumentation * Fix clippy * ".git/.scripts/bench-bot.sh" pallet dev pallet_contracts * Apply suggestions from code review Co-authored-by: Robin Freyler <robin.freyler@gmail.com> * Replace wasmi by ::wasmi * Bump wasmi to 0.20 * Add explanation for `unreachable` * Change proof * Fixup master merge * ".git/.scripts/bench-bot.sh" pallet dev pallet_contracts * Fixup naming inconsistencies introduced by reentrancy PR * Fix `scan_imports` docs * Apply suggestions from code review Co-authored-by: Sasha Gryaznov <hi@agryaznov.com> * Fixup suggestions * Remove unnecessary &mut * Fix test * ".git/.scripts/bench-bot.sh" pallet dev pallet_contracts * Fix benchmark merge fail * ".git/.scripts/bench-bot.sh" pallet dev pallet_contracts * Fix docs as suggested by code review * Improve docs for `CodeRejected` * Apply suggestions from code review Co-authored-by: Sasha Gryaznov <hi@agryaznov.com> * Fix logic bug when setting `deterministic_only` * Don't panic when module fails to compile * Apply suggestions from code review Co-authored-by: Robin Freyler <robin.freyler@gmail.com> Co-authored-by: command-bot <> Co-authored-by: Robin Freyler <robin.freyler@gmail.com> Co-authored-by: Sasha Gryaznov <hi@agryaznov.com> * update DefaultNoBound derive macro (#12723) fix derive for empty enums Update derive & ui tests clean up Apply suggestions from code review Co-authored-by: Bastian Köcher <git@kchr.de> rename variable formatting & clippy formatting Co-authored-by: parity-processbot <> * Fix rustdoc (#12777) * Fix table formatting Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix sp-runtime-interface table Using HTML now since multi-line tables are not a thing and fmt destroys them. Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * More rustdoc fixes Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix tags Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * More fixes... Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use Bastis patch Co-authored-by: Bastian Köcher <git@kchr.de> Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add more backticks Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * change ci image Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: alvicsam <alvicsam@gmail.com> * Allow Alliance Fellows to Give Up Voting Rights (#12730) * allow fellows to abdicate voting rights * rename founders to founding fellows, give equal power * Drop FoundingFellow role and veto call (#12762) * drop FoundingFellow role * drop veto call * Storage migration to remove founder role (#12766) * storage migration to remove founder role * skip migration if no members * truncate the final fellows set if overflows * change log - action order * MemberAbdicated -> FellowAbdicated Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com> * Add total nb to trie migration rpc (#12770) * Add total nb to trie migration rpc * fix and format * Use struct instead of tuple * fixes Co-authored-by: parity-processbot <> * add EnsureWithSuccess (#12775) * add EnsureWithSuccess * Apply suggestions from code review Co-authored-by: Bastian Köcher <git@kchr.de> * add docs Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: parity-processbot <> * Explicitly unset RUSTC_WRAPPER=sccache environment variable (#12771) * CI: Explicitly unset RUSTC_WRAPPER=sccache environment variable * Try with `rusty-cachier` disabled * Re-enable `rusty-cachier` and try with the staging image * Bring back `production` image * Sort crates before splitting them into groups (+ some improvements) (#12755) * sort crates before splitting them into groups this is useful so that crates always get routed to a specific group for a given version of the source code, which means that jobs for each batch can be reliably retried individually * more verbose output * misc improvements * put uniq after sort uniq filters by adjacent lines * shellcheck * rm useless backlashes * handle edge case of no crates detected * Revert "Sort crates before splitting them into groups (+ some improvements) (#12755)" This reverts commit fde839183a12a2bd51efc7143ebcddeed81ea6fa. Co-authored-by: João Paulo Silva de Souza <77391175+joao-paulo-parity@users.noreply.github.com> * contracts: Don't put unstable functions in special module (#12781) * Don't put unstable functions in special module * Apply suggestions from code review Co-authored-by: Sasha Gryaznov <hi@agryaznov.com> * cargo fmt Co-authored-by: Sasha Gryaznov <hi@agryaznov.com> * ed25519_verify: Support using dalek for historical blocks (#12661) * ed25519_verify: Support using dalek for historical blocks The switch from `ed25519-dalek` to `ed25519-zebra` was actually a breaking change. `ed25519-zebra` is more permissive. To support historical blocks when syncing a chain this pull request introduces an externalities extension `UseDalekExt`. This extension is just used as a signaling mechanism to `ed25519_verify` to use `ed25519-dalek` when it is present. Together with `ExtensionBeforeBlock` it can be used to setup a node in way to sync historical blocks that require `ed25519-dalek`, because they included a transaction that verified differently as when using `ed25519-zebra`. This feature can be enabled in the following way. In the chain service file, directly after the client is created, the following code should be added: ``` use sc_client_api::ExecutorProvider; client.execution_extensions().set_extensions_factory( sc_client_api::execution_extensions::ExtensionBeforeBlock::<Block, sp_io::UseDalekExt>::new(BLOCK_NUMBER_UNTIL_DALEK_SHOULD_BE_USED) ); ``` * Fix doc * More fixes * Update client/api/src/execution_extensions.rs Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> * Fix merge and warning * Fix docs Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> * client/beefy: fix on-demand justifications sync for old blocks (#12767) * client/beefy: fix on-demand justif sync for old blocks When receiving BEEFY justifications for old blocks the state might be pruned for them, in which case justification verification fails because BEEFY validator set cannot be retrieved from runtime state. Fix this by having the voter give the validator set to the `OnDemandJustificationsEngine` as request information. On receiving a BEEFY justification for requested block, the provided validator set will be used to validate the justification. Signed-off-by: acatangiu <adrian@parity.io> * Apply suggestions from code review Co-authored-by: Bastian Köcher <git@kchr.de> * impl review suggestions * client/beefy: fail initialization if state unavailable * beefy: remove spammy log Signed-off-by: acatangiu <adrian@parity.io> Co-authored-by: parity-processbot <> Co-authored-by: Bastian Köcher <git@kchr.de> * Remove Default, HasCompact, and TypeInfo trait bounds on AssetId (#12740) * Remove Default, HasCompact, and TypeInfo trait bounds on AssetId * don't use default in benchmarking * add helper trait * add helper to assets tx payment test * docs fixes * i'm confused * aha, cargo * move affected dispatchable calls into new indices * Helper -> BenchmarkHelper * benchmark use of helper * actually, don't break every call interface * use into on AssetIdParameter * Remove From from AssetIdParameter and use it in BenchmarkHelper * include from Co-authored-by: parity-processbot <> * pallet-mmr: move offchain logic to client-side gadget (#12753) * Move MMR utils methods from pallet to primitives Signed-off-by: Serban Iorga <serban@parity.io> * Add method to MmrApi * Move forks expanding logic from babe to primitives * Implement MMR gadget * Remove prunning logic from the MMR pallet * Code review changes: 1st iteration * Replace MaybeCanonEngine with CanonEngineBuilder * fix mmr_leaves_count() for kitchen sink demo * Update client/merkle-mountain-range/src/canon_engine.rs Co-authored-by: Adrian Catangiu <adrian@parity.io> * Code review changes: 2nd iteration * fix INDEXING_PREFIX * impl review comments * add documentation and minor rename Signed-off-by: Serban Iorga <serban@parity.io> Co-authored-by: Adrian Catangiu <adrian@parity.io> * Require rust-features check (#12796) * Typo Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Move rust feature check to docker and require not to fail Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add .docker-env Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Move test-rust-features check back to kubernetes Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * MMR: move RPC code from frame/ to client/ (#12805) * mmr: move MMR RPC from frame/ to client/ Signed-off-by: Adrian Catangiu <adrian@parity.io> * client/mmr: adjust logging levels to avoid spam * cargo fmt * remove unused imports Signed-off-by: Adrian Catangiu <adrian@parity.io> * chore: remove unused traits for wasm interface (#12792) * sc-transaction-handler: Fix potential crashes on exit (#12807) This fixes some potential crashes in the stream handling in `sc-transaction-handler`. * Don't announce blocks in `sync_to_tip_when_we_sync_together_with_multiple_peers` (#12783) * Fix syncing test * cargo fmt * Fix test * contracts: Replace cargo feature `unstable-interface` with config (#12787) * Replace cargo feature with config * Update frame/contracts/proc-macro/src/lib.rs Co-authored-by: Sasha Gryaznov <hi@agryaznov.com> Co-authored-by: Sasha Gryaznov <hi@agryaznov.com> * Bounties use SpendOrigin (#12808) * Bounties use SpendOrigin * Fix tests Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * fmt Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * tests: increase spend limits Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * fix benchmarks Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix child-bounties tests Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * ".git/.scripts/bench-bot.sh" pallet dev pallet_bounties Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: command-bot <> * Reduce provisioner work (#12749) * Move create_inherent_data call to use side * Make provide_inherent_data async * Fix tests * Apply suggestions from code review Co-authored-by: Bastian Köcher <git@kchr.de> * Log errors * Fix test * Fix test * fix * Deduplicate test code * fix * flag * Update client/consensus/slots/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Revert "Deduplicate test code" This reverts commit ba46adbe089329c78cd69ccdb08e27ed67bd77cf. * Fix test * remove commented out code * minor to start CI run * start CI * Update client/consensus/slots/src/lib.rs Co-authored-by: Marcin S. <marcin@bytedude.com> * Apply PR suggestions * Apply PR suggestions * Update client/consensus/slots/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * minor * kickoff CI * PR suggestions * Compute remaining duration instead of using slot_info.duration * Don't rely on sub implementat…

* Use wasm-opt on runtime * Optimize for size * Simplify fn compact_wasm_file * Run a lighter pass for non production builds * Disable optimizations and keep name section * Update wasm-opt * Remove dward sections * Update wasm-opt * Update wasm-opt

github-actions bot added the A3-in_progress Pull request is in progress. No review needed at this stage. label Sep 15, 2022

athei marked this pull request as ready for review September 15, 2022 16:55

github-actions bot added A0-please_review Pull request needs code review. and removed A3-in_progress Pull request is in progress. No review needed at this stage. labels Sep 15, 2022

athei changed the title ~~Replace wasm-gc with wasm-opt~~ WIP: Replace wasm-gc with wasm-opt Sep 15, 2022

paritytech deleted a comment from command-bot bot Sep 17, 2022

paritytech-ci requested a review from kianenigma as a code owner September 18, 2022 10:05

athei added B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit labels Sep 20, 2022

koute mentioned this pull request Sep 21, 2022

bench-bot: provide raw .json benchmark results as artifacts paritytech/command-bot#107

Closed

ggwpez approved these changes Sep 21, 2022

View reviewed changes

ggwpez reviewed Sep 21, 2022

View reviewed changes

athei force-pushed the at/wasm-opt branch from cb57a86 to 997c046 Compare September 21, 2022 20:36

paritytech deleted a comment from command-bot bot Sep 21, 2022

Update wasm-opt

a94f2b4

kripken mentioned this pull request Oct 17, 2022

Binary format: Don't emit empty Memory sections WebAssembly/binaryen#5145

Merged

brson mentioned this pull request Oct 17, 2022

Backport empty memories fix and make 110 point release brson/wasm-opt-rs#105

Closed

athei added 2 commits October 28, 2022 15:17

Merge branch 'master' into at/wasm-opt

4eafc07

Update wasm-opt

388f4d2

brson mentioned this pull request Oct 28, 2022

Downgrade cc dependency brson/wasm-opt-rs#113

Closed

koute approved these changes Oct 28, 2022

View reviewed changes

This was referenced Oct 28, 2022

Companion for update wasm-opt paritytech/polkadot#6209

Merged

Companion for update wasm-opt paritytech/cumulus#1806

Merged

paritytech-processbot bot merged commit 0e33b4e into master Oct 30, 2022

paritytech-processbot bot deleted the at/wasm-opt branch October 30, 2022 10:09

brson mentioned this pull request Nov 25, 2022

Add wasm-opt-for-rust maintenance grant w3f/Grants-Program#1305

Merged

10 tasks

jasl mentioned this pull request Dec 5, 2022

Upgrade to latest polkadot-v0.9.33 Phala-Network/phala-blockchain#1060

Merged

AurevoirXavier mentioned this pull request Jan 10, 2023

To polkadot-v0.9.33 darwinia-network/darwinia-2.0#194

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: Replace `wasm-gc` with `wasm-opt` #12280

WIP: Replace `wasm-gc` with `wasm-opt` #12280

athei commented Sep 15, 2022 •

edited

Loading

nazar-pc commented Sep 17, 2022

ggwpez left a comment

ggwpez Sep 21, 2022

athei commented Sep 21, 2022

koute commented Sep 22, 2022

athei commented Sep 23, 2022

koute commented Sep 23, 2022

athei commented Sep 23, 2022

koute commented Sep 23, 2022 •

edited

Loading

kripken commented Oct 17, 2022

brson commented Oct 17, 2022 •

edited

Loading

brson commented Oct 27, 2022

koute commented Oct 28, 2022

athei commented Oct 28, 2022

koute commented Oct 28, 2022

athei commented Oct 28, 2022

brson commented Oct 28, 2022 •

edited

Loading

koute left a comment

koute commented Oct 28, 2022 •

edited

Loading

athei commented Oct 29, 2022

ggwpez commented Oct 29, 2022

athei commented Oct 30, 2022

athei commented Oct 30, 2022

koute commented Oct 30, 2022

WIP: Replace wasm-gc with wasm-opt #12280

WIP: Replace wasm-gc with wasm-opt #12280

Conversation

athei commented Sep 15, 2022 • edited Loading

nazar-pc commented Sep 17, 2022

ggwpez left a comment

Choose a reason for hiding this comment

ggwpez Sep 21, 2022

Choose a reason for hiding this comment

athei commented Sep 21, 2022

koute commented Sep 22, 2022

Open questions

athei commented Sep 23, 2022

koute commented Sep 23, 2022

athei commented Sep 23, 2022

koute commented Sep 23, 2022 • edited Loading

kripken commented Oct 17, 2022

brson commented Oct 17, 2022 • edited Loading

brson commented Oct 27, 2022

koute commented Oct 28, 2022

athei commented Oct 28, 2022

koute commented Oct 28, 2022

athei commented Oct 28, 2022

brson commented Oct 28, 2022 • edited Loading

koute left a comment

Choose a reason for hiding this comment

koute commented Oct 28, 2022 • edited Loading

athei commented Oct 29, 2022

ggwpez commented Oct 29, 2022

athei commented Oct 30, 2022

athei commented Oct 30, 2022

koute commented Oct 30, 2022

WIP: Replace `wasm-gc` with `wasm-opt` #12280

WIP: Replace `wasm-gc` with `wasm-opt` #12280

athei commented Sep 15, 2022 •

edited

Loading

koute commented Sep 23, 2022 •

edited

Loading

brson commented Oct 17, 2022 •

edited

Loading

brson commented Oct 28, 2022 •

edited

Loading

koute commented Oct 28, 2022 •

edited

Loading