Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handling of disabled validators in backing subsystem #1259

Merged
merged 44 commits into from
Oct 20, 2023
Merged
Show file tree
Hide file tree
Changes from 36 commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
800d8e5
Add `disabled_validators` in staging runtime api
tdimitrov Aug 10, 2023
936216d
Cumulus changes
tdimitrov Sep 4, 2023
7165ef9
Handling of disabled validators in backing subsystem
tdimitrov Aug 31, 2023
24b5df5
Tests
tdimitrov Aug 31, 2023
eaca5ed
Merge branch 'master' into tsv-disabling-node-side
tdimitrov Sep 4, 2023
e4dee17
Fix compilation errors
tdimitrov Sep 4, 2023
23023c4
Merge branch 'tsv-disabling-node-side' into tsv-disabling-backing
tdimitrov Sep 5, 2023
8db76f5
spelling
tdimitrov Sep 5, 2023
e2846b2
clippy
tdimitrov Sep 5, 2023
8604b69
fmt
tdimitrov Sep 5, 2023
78afa76
clippy
tdimitrov Sep 5, 2023
9e39e58
Fix tests
tdimitrov Sep 5, 2023
0d0129c
Merge branch 'master' into tsv-disabling-node-side
tdimitrov Sep 11, 2023
23fbb41
Merge branch 'tsv-disabling-node-side' into tsv-disabling-backing
tdimitrov Sep 11, 2023
91d1a8d
Demote log level of a trace
tdimitrov Sep 11, 2023
3d7efab
Code review feedback
tdimitrov Sep 13, 2023
e68fa6a
tests
tdimitrov Sep 13, 2023
5e7655f
Merge branch 'master' into tsv-disabling-node-side
tdimitrov Sep 16, 2023
c6bca9c
Merge branch 'tsv-disabling-node-side' into tsv-disabling-backing
tdimitrov Sep 16, 2023
6fb4b7f
merge master and resolve conflicts (v8)
ordian Sep 27, 2023
70dc6b6
implement disabled_validators correctly
ordian Sep 27, 2023
bca3c83
add a CAVEAT comment
ordian Sep 27, 2023
f717d0b
cargo fmt
ordian Sep 27, 2023
2a5af89
Clarify docs
ordian Sep 27, 2023
2c53894
Merge branch 'master' into tsv-disabling-node-side
tdimitrov Sep 28, 2023
26866fa
Merge branch 'tsv-disabling-node-side' into tsv-disabling-backing
ordian Sep 28, 2023
a3bcb2f
Merge branch 'master' into tsv-disabling-node-side
ordian Oct 3, 2023
863fc7d
Merge branch 'tsv-disabling-node-side' into tsv-disabling-backing
ordian Oct 3, 2023
c44093d
Merge branch 'master' into tsv-disabling-backing
ordian Oct 16, 2023
1e561e9
Merge branch 'tsv-disabling' into tsv-disabling-backing
tdimitrov Oct 16, 2023
cf671f4
Update polkadot/node/core/backing/src/lib.rs
tdimitrov Oct 17, 2023
8807981
Don't kikoff validation work if we are not a validator
tdimitrov Oct 17, 2023
55f8e01
Merge branch 'tsv-disabling' into tsv-disabling-backing
tdimitrov Oct 17, 2023
b5609e1
Merge branch 'tsv-disabling' into tsv-disabling-backing
tdimitrov Oct 18, 2023
617e477
Extract `has_required_runtime` from provisioner to subsystem-util
tdimitrov Oct 19, 2023
d7f4315
Handle old runtimes
tdimitrov Oct 19, 2023
718f04b
Fix tests
tdimitrov Oct 19, 2023
275ce02
Use the specialized runtime request function for disabled validators
tdimitrov Oct 19, 2023
f2b17f1
Fix log level
tdimitrov Oct 19, 2023
7872f8f
Merge branch 'tsv-disabling' into tsv-disabling-backing
tdimitrov Oct 20, 2023
3c89059
Extract `get_disabled_validators_with_fallback` in a separate module …
tdimitrov Oct 20, 2023
15b5069
Small style fix
tdimitrov Oct 20, 2023
96c1358
Use `get_disabled_validators_with_fallback` in `Validator` from subsy…
tdimitrov Oct 20, 2023
1a6648a
Fix comments
tdimitrov Oct 20, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 90 additions & 14 deletions polkadot/node/core/backing/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,7 @@ use statement_table::{
},
Config as TableConfig, Context as TableContextTrait, Table,
};
use util::has_required_runtime;

mod error;

Expand Down Expand Up @@ -383,6 +384,21 @@ struct TableContext {
validator: Option<Validator>,
groups: HashMap<ParaId, Vec<ValidatorIndex>>,
validators: Vec<ValidatorId>,
disabled_validators: Vec<ValidatorIndex>,
}

impl TableContext {
// Returns `true` if the provided `ValidatorIndex` is in the disabled validators list
pub fn validator_is_disabled(&self, validator_idx: &ValidatorIndex) -> bool {
tdimitrov marked this conversation as resolved.
Show resolved Hide resolved
self.disabled_validators
.iter()
.any(|disabled_val_idx| *disabled_val_idx == *validator_idx)
}

// Returns `true` if the local validator is in the disabled validators list
pub fn local_validator_is_disabled(&self) -> Option<bool> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to return Option ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self.validator is an Option. Afair the code can be executed in cases where the local node is not a validator?

I decided to return an Option here too and handle it at the caller.

self.validator.as_ref().map(|v| v.disabled())
}
}

impl TableContextTrait for TableContext {
Expand Down Expand Up @@ -1010,21 +1026,50 @@ async fn construct_per_relay_parent_state<Context>(
let minimum_backing_votes =
try_runtime_api!(request_min_backing_votes(parent, session_index, ctx.sender()).await);

// TODO: https://github.com/paritytech/polkadot-sdk/issues/1940
// Once runtime ver `DISABLED_VALIDATORS_RUNTIME_REQUIREMENT` is released remove this `if`
// statement, add `request_from_runtime` call to the `try_join!` call above and use
// `try_runtime_api!` to get `disabled_validators`
let disabled_validators = if has_required_runtime(
ctx.sender(),
parent,
RuntimeApiRequest::DISABLED_VALIDATORS_RUNTIME_REQUIREMENT,
)
.await
{
let disabled_validators = request_from_runtime(parent, ctx.sender(), |tx| {
RuntimeApiRequest::DisabledValidators(tx)
})
.await
.await
.map_err(Error::JoinMultiple)?;

let disabled_validators = try_runtime_api!(disabled_validators);
disabled_validators
} else {
gum::debug!(target: LOG_TARGET, "Runtime doesn't support `DisabledValidators` - continuing with an empty disabled validators set");
vec![]
};

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code needs to stay until we release v8: #1940

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we extract that into subsystem-util or somewhere where statement-distribution and dispute-coordinator can reuse it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I was thinking about the same and I started working on it but against master.
My idea is to have a wrapper around request_from_runtime which does a version check and optionally returns a fallback version. I'll open a PR against master because I think this will be useful in general. We can backport it here easily after that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went into a macro rabbit hole. I'll just extract this implementation here and finish the other version in a more appropriate time.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, sounds good. However, I didn't mean to use a macro for that, just a regular function should be OK.

let signing_context = SigningContext { parent_hash: parent, session_index };
let validator =
match Validator::construct(&validators, signing_context.clone(), keystore.clone()) {
Ok(v) => Some(v),
Err(util::Error::NotAValidator) => None,
Err(e) => {
gum::warn!(
target: LOG_TARGET,
err = ?e,
"Cannot participate in candidate backing",
);
let validator = match Validator::construct(
&validators,
&disabled_validators,
signing_context.clone(),
keystore.clone(),
) {
Ok(v) => Some(v),
Err(util::Error::NotAValidator) => None,
Err(e) => {
gum::warn!(
target: LOG_TARGET,
err = ?e,
"Cannot participate in candidate backing",
);

return Ok(None)
},
};
return Ok(None)
},
};

let mut groups = HashMap::new();
let n_cores = cores.len();
Expand Down Expand Up @@ -1054,7 +1099,7 @@ async fn construct_per_relay_parent_state<Context>(
}
}

let table_context = TableContext { groups, validators, validator };
let table_context = TableContext { validator, groups, validators, disabled_validators };
let table_config = TableConfig {
allow_multiple_seconded: match mode {
ProspectiveParachainsMode::Enabled { .. } => true,
Expand Down Expand Up @@ -1728,6 +1773,19 @@ async fn kick_off_validation_work<Context>(
background_validation_tx: &mpsc::Sender<(Hash, ValidatedCandidateCommand)>,
attesting: AttestingData,
) -> Result<(), Error> {
// Do nothing if the local validator is disabled or not a validator at all
match rp_state.table_context.local_validator_is_disabled() {
Some(true) => {
gum::debug!(target: LOG_TARGET, "We are disabled - don't kick off validation");
tdimitrov marked this conversation as resolved.
Show resolved Hide resolved
return Ok(())
},
Some(false) => {}, // we are not disabled - move on
None => {
gum::debug!(target: LOG_TARGET, "We are not a validator - don't kick off validation");
return Ok(())
},
}

let candidate_hash = attesting.candidate.hash();
if rp_state.issued_statements.contains(&candidate_hash) {
return Ok(())
Expand Down Expand Up @@ -1785,6 +1843,16 @@ async fn maybe_validate_and_import<Context>(
},
};

// Don't import statement if the sender is disabled
if rp_state.table_context.validator_is_disabled(&statement.validator_index()) {
gum::debug!(
target: LOG_TARGET,
sender_validator_idx = ?statement.validator_index(),
"Not importing statement because the sender is disabled"
);
return Ok(())
}

let res = import_statement(ctx, rp_state, &mut state.per_candidate, &statement).await;

// if we get an Error::RejectedByProspectiveParachains,
Expand Down Expand Up @@ -1946,6 +2014,13 @@ async fn handle_second_message<Context>(
Some(r) => r,
};

// Just return if the local validator is disabled. If we are here the local node should be a
// validator but defensively use `unwrap_or(false)` to continue processing in this case.
if rp_state.table_context.local_validator_is_disabled().unwrap_or(false) {
gum::warn!(target: LOG_TARGET, "Local validator is disabled. Don't validate and second");
return Ok(())
}

// Sanity check that candidate is from our assignment.
if Some(candidate.descriptor().para_id) != rp_state.assignment {
gum::debug!(
Expand Down Expand Up @@ -1992,6 +2067,7 @@ async fn handle_statement_message<Context>(
) -> Result<(), Error> {
let _timer = metrics.time_process_statement();

// Validator disabling is handled in `maybe_validate_and_import`
match maybe_validate_and_import(ctx, state, relay_parent, statement).await {
Err(Error::ValidationFailed(_)) => Ok(()),
Err(e) => Err(e),
Expand Down
Loading