-
Notifications
You must be signed in to change notification settings - Fork 8
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Wire between Orchestrator - Provider (#101)
* wip, refactor provider trait * fmt * wip * wip, add from_config to NetworkSpec * config reorgs * add logic for create NetworkSpec * feat: refactored FileSystem trait and errors * chore: removed unused local_file, stdout will be hardcoded in provider at the moment * feat: moved MockFileSystem to InMemoryFileSystem, files too, and added mirror implementation of tokio::fs with tests * feat: added new method append to FileSystem trait * feat: implemented new append method on InMemoryFileSystem * feat: added anyhow dependencies to support crate * feat: refactored FileSystem trait to expose only a single wrapped error * feat: refacto InMemoryFileSystem following FileSystem trait refacto * feat: added conversion between io::Error and FileSystemError * feat: added implementation of LocalFileSystem using tokio::fs * feat: added nix crate dependency to workspace and provider crate * feat: added uuid with v4 as dev-dependencies for testing in support crate * feat: added unit tests for LocalFileSystem * feat: updated Provider, ProviderNamespace and ProviderNode traits and related DTO * feat: added anyhow as dependency to provider crate * feat: updated ProviderError * feat: work on NativeProvider with individual Node, Namespace and Provider struct threadsafe using RwLock and Arc, added implementation logic of resume/restart/pause, destroy and helpers for logs * feat: rename some provider constants * wip * feat: added uuid with v4 features as normal dependency * feat: added new set_mode method on FileSystem trait to modify permissions bits, added implementations and tests * feat: added builder for options types used in provider traits methods, moved error next to provider traits * feat: added modified implementation of run_script/run_command/copy_from_node in NativeProvider, removed unused comments * feat: moved filesystem, capabilities and tmp_dir out of NativeProviderInner * wip on network spec creation * move chain_spec to generators * change defaults for validator/invulnerable settings * Add test and clean-ups * nits and clean * fmt * fmt * fix docs/clippy * fixes from clippy * add TODO and allow dead_code for now * fmt * more nits * fmt * feat: make constructors and fields public for testing on InMemoryFileSystem * feat: removed unused types atm and added TransferedFile * feat: updated types for Provider methods output, updated non needed async methods * feat: added generate_files implementation on namespace using temporary nodes, moved some non mutable fields out of inners * feat: removed comment * feat: refactored provider types and added builders * feat(orchestrator) add generators * feat(orchestrator) add logic to compute the network spec * small changes in provider trait and native impl * wip, example to drive exec * make paras working, first design draft of network public api * cleanups * add spawner * add logic to add new nodes to running network and methods on nodes * modify example * fmt * allow to add collators to a running network * modify example * move Network related structs * reorg code * fmt * fixes and clean-up * removed commented code * clippy * fmt * clippy * fmt * clippy in example * fix validator/invulnerable true as default * fmt * fix base_dir for mac/linux compat * fix, allow generate files with fullpaths (encapsualate as part of the ns) * Add todos * fix p2p port for full_node in collator * fix p2p port for full_node in collator * clippy * fmt * Update crates/orchestrator/src/network_spec/node.rs Co-authored-by: Loris Moulin <45130584+l0r1s@users.noreply.github.com> * Update crates/orchestrator/src/network_spec/relaychain.rs Co-authored-by: Loris Moulin <45130584+l0r1s@users.noreply.github.com> * Update crates/orchestrator/src/generators/chain_spec.rs Co-authored-by: Nikos Kontakis <wirednkod@gmail.com> * Update crates/examples/examples/small_network_with_default.rs Co-authored-by: Loris Moulin <45130584+l0r1s@users.noreply.github.com> * Update crates/orchestrator/src/generators/bootnode_addr.rs Co-authored-by: Loris Moulin <45130584+l0r1s@users.noreply.github.com> * Update crates/orchestrator/src/network_spec/node.rs Co-authored-by: Loris Moulin <45130584+l0r1s@users.noreply.github.com> * Update crates/orchestrator/src/network_spec/node.rs Co-authored-by: Loris Moulin <45130584+l0r1s@users.noreply.github.com> * Update crates/orchestrator/src/network_spec/relaychain.rs Co-authored-by: Loris Moulin <45130584+l0r1s@users.noreply.github.com> * changes from feedback * reorg deps * clippy/fmt * reorg generators and add some unittest * more unittest and cleans * fmt/clippy * fmt/clippy * add TODOs from feedback --------- Co-authored-by: l0r1s <contact@lorismoulin.com> Co-authored-by: Loris Moulin <45130584+l0r1s@users.noreply.github.com> Co-authored-by: Nikos Kontakis <wirednkod@gmail.com>
- Loading branch information
4 people
committed
Sep 29, 2023
1 parent
656f56c
commit 8f8b03f
Showing
35 changed files
with
3,514 additions
and
50 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,14 +1,65 @@ | ||
| use std::time::Duration; | ||
|
|
||
| use configuration::NetworkConfigBuilder; | ||
| use orchestrator::{AddNodeOpts, Orchestrator}; | ||
| use provider::NativeProvider; | ||
| use support::fs::local::LocalFileSystem; | ||
|
|
||
| fn main() { | ||
| #[tokio::main] | ||
| async fn main() -> Result<(), Box<dyn std::error::Error>> { | ||
| let config = NetworkConfigBuilder::new() | ||
| .with_relaychain(|r| { | ||
| r.with_chain("rococo-local") | ||
| .with_default_command("polkadot") | ||
| .with_node(|node| node.with_name("alice")) | ||
| .with_node(|node| node.with_name("name")) | ||
| .with_node(|node| node.with_name("bob")) | ||
| }) | ||
| .with_parachain(|p| { | ||
| p.with_id(100) | ||
| .cumulus_based(true) | ||
| .with_collator(|n| n.with_name("collator").with_command("polkadot-parachain")) | ||
| }) | ||
| .build(); | ||
| .build() | ||
| .unwrap(); | ||
|
|
||
| let fs = LocalFileSystem; | ||
| let provider = NativeProvider::new(fs.clone()); | ||
| let orchestrator = Orchestrator::new(fs, provider); | ||
| let mut network = orchestrator.spawn(config).await?; | ||
| println!("🚀🚀🚀🚀 network deployed"); | ||
| // add a new node | ||
| let opts = AddNodeOpts { | ||
| rpc_port: Some(9444), | ||
| is_validator: true, | ||
| ..Default::default() | ||
| }; | ||
|
|
||
| // TODO: add check to ensure if unique | ||
| network.add_node("new1", opts, None).await?; | ||
|
|
||
| tokio::time::sleep(Duration::from_secs(5)).await; | ||
|
|
||
| // Example of some opertions that you can do | ||
| // with `nodes` (e.g pause, resume, restart) | ||
| // pause the node | ||
| // network.pause_node("new1").await?; | ||
| // println!("node new1 paused!"); | ||
|
|
||
| // tokio::time::sleep(Duration::from_secs(5)).await; | ||
|
|
||
| // network.resume_node("new1").await?; | ||
| // println!("node new1 resumed!"); | ||
|
|
||
| let col_opts = AddNodeOpts { | ||
| command: Some("polkadot-parachain".try_into()?), | ||
| ..Default::default() | ||
| }; | ||
| network.add_node("new-col-1", col_opts, Some(100)).await?; | ||
| println!("new collator deployed!"); | ||
|
|
||
| // For now let just loop.... | ||
| #[allow(clippy::empty_loop)] | ||
| loop {} | ||
|
|
||
| println!("{:?}", config.unwrap()); | ||
| // Ok(()) | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| //! Zombienet Orchestrator error definitions. | ||
|
|
||
| use provider::ProviderError; | ||
| use support::fs::FileSystemError; | ||
|
|
||
| use crate::generators; | ||
|
|
||
| #[derive(Debug, thiserror::Error)] | ||
| pub enum OrchestratorError { | ||
| // TODO: improve invalid config reporting | ||
| #[error("Invalid network configuration: {0}")] | ||
| InvalidConfig(String), | ||
| #[error("Invalid configuration for node: {0}, field: {1}")] | ||
| InvalidNodeConfig(String, String), | ||
| #[error("Invariant not fulfilled {0}")] | ||
| InvariantError(&'static str), | ||
| #[error("Global network spawn timeout: {0} secs")] | ||
| GlobalTimeOut(u32), | ||
| #[error("Generator error")] | ||
| GeneratorError(#[from] generators::errors::GeneratorError), | ||
| #[error("Provider error")] | ||
| ProviderError(#[from] ProviderError), | ||
| #[error("FileSystem error")] | ||
| FileSystemError(#[from] FileSystemError), | ||
| #[error(transparent)] | ||
| SpawnerError(#[from] anyhow::Error), | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| pub mod chain_spec; | ||
| pub mod errors; | ||
| pub mod para_artifact; | ||
|
|
||
| mod bootnode_addr; | ||
| mod command; | ||
| mod identity; | ||
| mod key; | ||
| mod keystore; | ||
| mod port; | ||
|
|
||
| pub use bootnode_addr::generate as generate_node_bootnode_addr; | ||
| pub use command::{ | ||
| generate_for_cumulus_node as generate_node_command_cumulus, | ||
| generate_for_node as generate_node_command, GenCmdOptions, | ||
| }; | ||
| pub use identity::generate as generate_node_identity; | ||
| pub use key::generate as generate_node_keys; | ||
| pub use keystore::generate as generate_node_keystore; | ||
| pub use port::generate as generate_node_port; |
Oops, something went wrong.