Skip to content

Commit

Permalink
Merge pull request #2596 from hyperledger/develop
Browse files Browse the repository at this point in the history
[chore] release 1.1 merge
  • Loading branch information
sownak authored Jul 4, 2024
2 parents 2cb4d29 + d5efd51 commit c62e8fe
Show file tree
Hide file tree
Showing 995 changed files with 31,593 additions and 48,164 deletions.
162 changes: 162 additions & 0 deletions .github/workflows/aws_corda_deploy.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
##############################################################################################
# Copyright Accenture. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
##############################################################################################

##############################################################################################
# Workflow: Deploy Hyperledger Bevel's CORDA DLT Platform to an EKS Cluster.

# Prerequisites:
# 1. An accessible EKS Cluster
# 2. A Vault instance accessible from GitHub Runner
# 3. A completed network.yaml file stored in GitHub Secrets

# Workflow Overview:
# 1. This GitHub Actions workflow automates the seamless deployment of "BEVEL's CORDA" platform to an EKS cluster.
# 2. Utilizing secure environment variables, the workflow manages sensitive information related to AWS, Docker, Cluster, Vault, and Git.
# 3. The workflow dynamically customizes a network configuration file by substituting placeholders with values derived from environment variables.
# 4. It uses tool Ansible to deploy the platform.
##############################################################################################

# Name of the workflow
name: Deploy or Reset Corda Network to an EKS Cluster

# Triggers for the workflow
on:
# Manually trigger the workflow through the GitHub Actions UI
workflow_dispatch:
inputs:
action:
description: 'Choose action: Deploy or Reset'
required: true
default: 'deploy'
type: choice
options:
- 'deploy'
- 'reset'
paths-ignore:
- 'docs/**'
- '**/charts/**'
- '**/releases/**'

# Jobs to be executed
jobs:
deployment:
runs-on: ubuntu-latest
permissions:
contents: write
environment: Bevel-AWS-Deployment
env:
AWS_ACCESS_KEY_ID: "${{ secrets.AWS_ACCESS_KEY_ID }}" # AWS Access Key ID
AWS_SECRET_ACCESS_KEY: "${{ secrets.AWS_SECRET_ACCESS_KEY }}" # AWS Secret Access Key
AWS_REGION: "${{ secrets.AWS_REGION }}" # EKS cluster zone
CLUSTER_CONTEXT: "${{ secrets.CLUSTER_CONTEXT }}" # Context name for the EKS cluster
KUBECONFIG: "${{ secrets.ENCODED_KUBECONFIG }}" # Provide Kubernetes configuration file in encoded base64 format
DOCKER_URL: "${{ secrets.DOCKER_URL }}" # URL of the Docker registry
DOCKER_USERNAME: "${{ secrets.DOCKER_USERNAME }}" # Docker registry username
DOCKER_PASSWORD: "${{ secrets.DOCKER_PASSWORD }}" # Docker registry password
EXTERNAL_URL_SUFFIX: "${{ secrets.EXTERNAL_URL_SUFFIX }}" # Suffix for external URLs
GIT_USER_NAME: "${{ secrets.GIT_USER_NAME }}" # Git username for Git operations
GIT_EMAIL_ADDR: "${{ secrets.GIT_EMAIL_ADDR }}" # Git email address for Git operations
GIT_TOKEN: "${{ secrets.GIT_TOKEN }}" # Git token with required permissions for authentication
GIT_BRANCH: "${{ vars.GIT_BRANCH }}" # Git branch to be used in the deployment
GIT_PRIVATE_SSH_KEY: "${{ secrets.GIT_PRIVATE_SSH_KEY }}" # Private SSH key for Git authentication in encoded base64 format
VAULT_ADDR: "${{ secrets.VAULT_ADDR }}" # Vault Server DNS name
VAULT_TOKEN: "${{ secrets.VAULT_TOKEN }}" # Token for authentication with Vault

# Steps to be executed within the job
steps:
# Checkout the repository code
- name: Checkout Repository
uses: actions/checkout@v2.4.0

# Java installation
- name: Install java
uses: actions/setup-java@v2
with:
distribution: 'adopt'
java-version: '8'

# Configure AWS credentials
- name: AWS Setup
uses: aws-actions/configure-aws-credentials@v2
with:
aws-access-key-id: "${{ env.AWS_ACCESS_KEY_ID }}"
aws-secret-access-key: "${{ env.AWS_SECRET_ACCESS_KEY }}"
aws-region: "${{ env.AWS_REGION }}"

# Set up BEVEL's Corda network configuration file
- name: BEVEL's Corda Network Configuration file Setup
run: |
# Prepare network configuration file for deployment
mkdir -p build/
cp "./platforms/r3-corda/configuration/samples/workflow/network-proxy-corda.yaml" "build/network-corda.yaml"
NETWORK_CONF_FILE="build/network-corda.yaml"
# Decode and store private SSH key
echo "${{ env.GIT_PRIVATE_SSH_KEY }}" | base64 --decode > /home/runner/private_ssh_key
# Define placeholder values for the network configuration file
declare -A placeholders=(
["NETWORK_VERSION"]="4.9"
["FLUX_SUFFIX"]="corda"
["PORT_RANGE_FROM"]=15010
["PORT_RANGE_TO"]=15090
["DOCKER_URL"]="${{ env.DOCKER_URL }}"
["DOCKER_USERNAME"]="${{ env.DOCKER_USERNAME }}"
["DOCKER_PASSWORD"]="${{ env.DOCKER_PASSWORD }}"
["USER_DIRECTORY"]="$(pwd)"
["EXTERNAL_URL_SUFFIX"]="${{ env.EXTERNAL_URL_SUFFIX }}"
["AWS_ACCESS_KEY"]="${{ env.AWS_ACCESS_KEY_ID }}"
["AWS_SECRET_KEY"]="${{ env.AWS_SECRET_ACCESS_KEY }}"
["AWS_REGION"]="${{ env.AWS_REGION}}"
["CLUSTER_CONTEXT"]="${{ env.CLUSTER_CONTEXT }}"
["CLUSTER_CONFIG"]="/home/runner/.kube/build_config/kubeconfig"
["VAULT_ADDR"]="${{ env.VAULT_ADDR }}"
["VAULT_ROOT_TOKEN"]="${{ env.VAULT_TOKEN }}"
["GIT_USERNAME"]="${{ env.GIT_USER_NAME }}"
["GIT_TOKEN"]="${{ env.GIT_TOKEN }}"
["GIT_EMAIL_ADDR"]="${{ env.GIT_EMAIL_ADDR }}"
["GIT_BRANCH"]="${{ env.GIT_BRANCH }}"
["PRIVATE_KEY_PATH"]="/home/runner/private_ssh_key"
)
# Replace placeholders in the network configuration file
for placeholder in "${!placeholders[@]}"; do
sed -i "s#${placeholder}#${placeholders[$placeholder]}#g" "$NETWORK_CONF_FILE"
done
# Deploy BEVEL's Corda Platform
- name: Deploy BEVEL's Corda Platform
run: |
# Setup Kubernetes configuration
mkdir -p /home/runner/.kube/build_config
echo "${{ env.KUBECONFIG }}" | base64 --decode > /home/runner/.kube/build_config/kubeconfig
export KUBECONFIG="/home/runner/.kube/build_config/kubeconfig"
# Configure Git user settings
git config --global user.email "${{ env.GIT_EMAIL_ADDR }}"
git config --global user.name "${{ env.GIT_USER_NAME }}"
# Install required tools and Ansible collections
mkdir -p ~/bin
export PATH=$PATH:~/bin
pip3 install openshift=='0.13.1'
pip install ansible jmespath jinja2-time
ansible-galaxy collection install -r platforms/shared/configuration/requirements.yaml
# Set reset variable
if [ "${{ github.event.inputs.action }}" == "reset" ]; then
reset=true
else
reset=false
fi
# Deploy the BEVEL's corda DLT platform
ansible-playbook platforms/shared/configuration/site.yaml \
-i platforms/shared/inventory/ansible_provisioners \
-e @build/network-corda.yaml \
-e 'ansible_python_interpreter=/usr/bin/python3' -e "reset=$reset"
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -40,4 +40,5 @@
*_custom.tpl
**/charts/*.tgz
**/files/*.json
**/files/*.crt
requirements.lock
4 changes: 4 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,10 @@ RUN rm /etc/apt/apt.conf.d/docker-clean
RUN mkdir /etc/ansible/
RUN /bin/echo -e "[ansible_provisioners:children]\nlocal\n[local]\nlocalhost ansible_connection=local" > /etc/ansible/hosts

RUN curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.27.0/bin/linux/amd64/kubectl
RUN chmod +x ./kubectl
RUN mv ./kubectl /usr/local/bin

# Install krew for bevel-operator-fabric
RUN (set -x; cd "$(mktemp -d)" && \
OS="$(uname | tr '[:upper:]' '[:lower:]')" && \
Expand Down
10 changes: 5 additions & 5 deletions Dockerfile.jdk8
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,11 @@
#
# SPDX-License-Identifier: Apache-2.0
##############################################################################################

# USAGE:
# docker build . -t bevel-build
# docker run -v $(pwd):/home/bevel/ bevel-build

FROM ubuntu:20.04

# Create working directory
WORKDIR /home/
ENV OPENSHIFT_VERSION='0.13.1'
Expand Down Expand Up @@ -37,13 +35,17 @@ RUN apt-get update && apt-get install -y \
apt-get clean && \
ln -s /usr/bin/python3 /usr/bin/python && \
rm -rf /var/lib/apt/lists/*
RUN npm install -g ajv-cli
RUN npm install -g ajv-cli
RUN apt-get update && apt-get install -y python3-venv

RUN rm /etc/apt/apt.conf.d/docker-clean
RUN mkdir /etc/ansible/
RUN /bin/echo -e "[ansible_provisioners:children]\nlocal\n[local]\nlocalhost ansible_connection=local" > /etc/ansible/hosts

RUN curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.27.0/bin/linux/amd64/kubectl
RUN chmod +x ./kubectl
RUN mv ./kubectl /usr/local/bin

# Copy the provisional script to build container
COPY ./run.sh /home
COPY ./reset.sh /home
Expand All @@ -58,6 +60,4 @@ ENV PATH=/root/bin:/root/.local/bin/:$PATH

#path to mount the repo
VOLUME /home/bevel/


CMD ["/home/run.sh"]
87 changes: 55 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,74 +16,97 @@
- [Hyperledger Indy](#hyperledger-indy)
- [Quorum](#quorum)
- [Hyperledger Besu](#hyperledger-besu)
- [Substrate](#substrate)
- [Contact](#contact)
- [Contributing](#contributing)
- [Initial Committers](#initial-committers)
- [Sponsor](#sponsor)

## Short Description
An automation framework for rapidly and consistently deploying production-ready Distributed Ledger Technology (DLT) platforms.
An automation framework and helm charts for rapidly and consistently deploying production-ready Distributed Ledger Technology (DLT) platforms.

## Scope of Project
Hyperledger Bevel delivers an automation framework for rapidly and consistently deploying production-ready DLT platforms to cloud infrastructure.
Hyperledger Bevel is an automation framework for rapidly and consistently deploying production-ready DLT platforms to cloud infrastructure.

![What is Hyperledger Bevel?](./docs/images/hyperledger-bevel-overview.png "What is Hyperledger Bevel?")

Hyperledger Bevel is an accelerator/tool that helps developers rapidly set up and deploy secure, scalable and production-ready DLT network(s) that also allows new organizations to be easily on-boarded on the network. Bevel facilitates a safe and secure way of deploying and operating different DLT platforms.

It includes:
- Helm charts to deploy different DLT nodes and to generate the related crypto/identities.
- Helm charts for various operational features like adding new nodes, and deploying smart contracts.
- Helm charts to deploy Hyperledger Cacti connectors for Fabric, Quorum and Besu networks.
- Ansible playbooks and modular role definitions to automate the deployment of Helm charts.
- Ansible playbooks and roles to automate deployment of Hyperledger fabric using bevel-operator-fabric(Kubernetes operator for managing Hyperledger Fabric networks).
- Integrated CD using GitOps so that once the network is set up, all changes can be done via git PRs/merges.
- Configuration for Ambassador Edge Stack, HAProxy (for Hyperledger Fabric) and Isto Ingress (for Substrate) to act as Ingress Controller.
- Helm charts to **deploy** different DLT nodes and to generate the related crypto/identities.
- Helm charts for various **operational features** like adding new nodes, and deploying smart contracts.
- Helm charts to deploy Hyperledger **Cacti connectors** for Fabric, Quorum and Besu networks.
- **Ansible playbooks** and modular role definitions to automate the deployment of Helm charts.
- Ansible playbooks and roles to automate deployment of Hyperledger fabric using **bevel-operator-fabric** (Kubernetes operator for managing Hyperledger Fabric networks).
- Integrated CD using **GitOps** so that once the network is set up, all changes can be done via git PRs/merges.
- Configuration for Ambassador Edge Stack, HAProxy (for Hyperledger Fabric) and Istio Ingress (for Substrate) to act as Ingress Controller.

Hyperledger Bevel currently supports R3 Corda OS and Enterprise, Hyperledger Fabric, Hyperledger Indy, Hyperledger Besu, Quorum and Substrate. Other DLT platforms can easily be added.

### Getting Started
## Getting Started

To get started with the framework quickly, follow our [Getting Started guidelines](https://hyperledger-bevel.readthedocs.io/en/latest/gettingstarted.html).

Detailed operator and developer documentation is available on [our ReadTheDocs site](https://hyperledger-bevel.readthedocs.io/en/latest/index.html).

The documentation can also be built locally be following instructions in the `docs` folder.

### Hyperledger Fabric
For Hyperledger Fabric, we use the official Docker containers provided by that project. A number of different Ansible scripts will allow you to either create a new network (across clouds) or join an existing network.
## Hyperledger Fabric
For Hyperledger Fabric, there are two ways to deploy the network.

![Hyperledger Bevel - Fabric](./docs/images/hyperledger-bevel-fabric.png "Hyperledger Bevel for Hyperledger Fabric")
- Using `helm install`: Follow the [Fabric Charts readme](./platforms/hyperledger-fabric/charts/README.md).
- Using Ansible: A number of different Ansible scripts will allow you to either create a new network (across clouds) or join an existing network.

### Corda Enterprise
For Corda Enterprise, we build Docker containers from the Corda source with licensed jars. A number of different Ansible scripts will allow you to either create a new network (across clouds) or join an existing network.
![Hyperledger Bevel - Fabric](./docs/images/hyperledger-bevel-fabric.png "Hyperledger Bevel for Hyperledger Fabric")

![Hyperledger Bevel - Corda Enterprise](./docs/images/hyperledger-bevel-corda-ent.png "Hyperledger Bevel for Corda Enterprise")
## Corda Enterprise
For Corda Enterprise, there are two ways to deploy the network.

### Corda Opensource
For Corda Opensource, we build Docker containers from the Corda source. A number of different Ansible scripts will allow you to either create a new network (across clouds) or join an existing network.
- Using `helm install`: Follow the [Corda Enterprise Charts readme](./platforms/r3-corda-ent/charts/README.md).
- Using Ansible: A number of different Ansible scripts will allow you to either create a new network (across clouds) or join an existing network.

![Hyperledger Bevel - Corda](./docs/images/hyperledger-bevel-corda.png "Hyperledger Bevel for Corda")
![Hyperledger Bevel - Corda Enterprise](./docs/images/hyperledger-bevel-corda-ent.png "Hyperledger Bevel for Corda Enterprise")

### Hyperledger Indy
For Hyperledger Indy, we build Docker containers from our source code. A number of different Ansible scripts will allow you to create a new network (across clouds).
## Corda Opensource
For Corda Opensource, there are two ways to deploy the network.

- Using `helm install`: Follow the [Corda Charts readme](./platforms/r3-corda/charts/README.md).
- Using Ansible: A number of different Ansible scripts will allow you to either create a new network (across clouds) or join an existing network.

![Hyperledger Bevel - Corda](./docs/images/hyperledger-bevel-corda.png "Hyperledger Bevel for Corda")

## Hyperledger Indy
For Hyperledger Indy, there are two ways to deploy the network.

- Using `helm install`: Follow the [Indy Charts readme](./platforms/hyperledger-indy/charts/README.md).
- Using Ansible: A number of different Ansible scripts will allow you to create a new network (across clouds).

![Hyperledger Bevel - Indy](./docs/images/hyperledger-bevel-indy.png "Hyperledger Bevel for Hyperledger Indy")

### Quorum
For Quorum, we use the official Docker containers provided by Quorum. A number of different Ansible scripts will allow you to either create a new network (across clouds) with choice of Consensus (between IBFT and RAFT) and a transaction Manager.
## Quorum
For Quorum, there are two ways to deploy the network.

- Using `helm install`: Follow the [Quorum Charts readme](./platforms/quorum/charts/README.md).
- Using Ansible: A number of different Ansible scripts will allow you to either create a new network (across clouds) with choice of Consensus and a transaction Manager.

![Hyperledger Bevel - Quorum](./docs/images/hyperledger-bevel-quorum.png "Hyperledger Bevel for Quorum")

## Hyperledger Besu
For Hyperledger Besu, there are two ways to deploy the network.

- Using `helm install`: Follow the [Besu Charts readme](./platforms/hyperledger-besu/charts/README.md).
- Using Ansible: A number of different Ansible scripts will allow you to create a new network (across clouds).

![Hyperledger Bevel - Quorum](./docs/images/hyperledger-bevel-quorum.png "Hyperledger Bevel for Quorum")
![Hyperledger Bevel - Besu](./docs/images/hyperledger-bevel-besu.png "Hyperledger Bevel for Hyperledger Besu")

### Hyperledger Besu
For Hyperledger Besu, we use the official Docker containers provided by that project. A number of different Ansible scripts will allow you to create a new network (across clouds).
## Substrate
For Substrate, there are two ways to deploy the network.

![Hyperledger Bevel - Besu](./docs/images/hyperledger-bevel-besu.png "Hyperledger Bevel for Hyperledger Besu")
- Using `helm install`: Follow the [Substrate Charts readme](./platforms/substrate/charts/README.md).
- Using Ansible: A number of different Ansible scripts will allow you to create a new network (across clouds).

### Substrate
For Substrate, we use the official Docker containers provided by that project. A number of different Ansible scripts will allow you to create a new network (across clouds).
![Hyperledger Bevel - Substrate](./docs/images/hyperledger-bevel-substrate.png "Hyperledger Bevel for Substrate")

![Hyperledger Bevel - Substrate](./docs/images/hyperledger-bevel-substrate.png "Hyperledger Bevel for Substrate")
## Contact
We welcome your questions & feedback on our [Discord channel](https://discord.com/channels/905194001349627914/941739691336679454). [Please join our Discord first](https://discord.gg/hyperledger).

Expand All @@ -92,8 +115,8 @@ We welcome contributions to Hyperledger Bevel in many forms, and there’s alway

Please review [contributing](./CONTRIBUTING.md) guidelines to get started.

# Build
If you are not using the provided Jenkins automation scripts, you can run the provisioning scripts within a docker runtime independent from your target Kubernetes cluster.
## Build
If you are not using the provided Jenkins automation scripts, you can run the provisioning scripts within a docker runtime independent of your target Kubernetes cluster.
```
# Build provisioning image
docker build . -t ghcr.io/hyperledger/bevel-build
Expand Down
Loading

0 comments on commit c62e8fe

Please sign in to comment.