Skip to content

Commit

Permalink
Merge branch 'develop' into task/BCF-3136/generic-public-onchain-sign…
Browse files Browse the repository at this point in the history
…ing-stragegy
  • Loading branch information
vyzaldysanchez committed Apr 23, 2024
2 parents c097e2f + 394844e commit 64790be
Show file tree
Hide file tree
Showing 250 changed files with 1,558 additions and 1,013 deletions.
5 changes: 5 additions & 0 deletions .changeset/chilled-bikes-unite.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"chainlink": minor
---

#internal Add script to create test database user and update docs
5 changes: 5 additions & 0 deletions .changeset/kind-deers-leave.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"chainlink": patch
---

bump mockery in makefile #updated
5 changes: 5 additions & 0 deletions .changeset/sour-needles-collect.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"chainlink": patch
---

cleanup set #bugfix
1 change: 1 addition & 0 deletions .github/workflows/ci-core.yml
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,7 @@ jobs:
if: ${{ needs.filter.outputs.changes == 'true' }}
uses: ./.github/actions/setup-go
- name: Run short tests
if: ${{ needs.filter.outputs.changes == 'true' }}
run: go test -short ./...
- name: Setup Solana
if: ${{ needs.filter.outputs.changes == 'true' }}
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,9 @@ tools/clroot/db.sqlite3-wal
.DS_Store
.envrc
.env*
.dbenv
!charts/chainlink-cluster/.env.example
!crib/.env.example
!.github/actions/setup-postgres/.env
.direnv
.idea
Expand Down
6 changes: 5 additions & 1 deletion GNUmakefile
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,10 @@ testscripts: chainlink-test ## Install and run testscript against testdata/scrip
testscripts-update: ## Update testdata/scripts/* files via testscript.
make testscripts TS_FLAGS="-u"

.PHONY: setup-testdb
setup-testdb: ## Setup the test database.
./core/scripts/setup_testdb.sh

.PHONY: testdb
testdb: ## Prepares the test database.
go run . local db preparetest
Expand All @@ -125,7 +129,7 @@ gomods: ## Install gomods

.PHONY: mockery
mockery: $(mockery) ## Install mockery.
go install github.com/vektra/mockery/v2@v2.38.0
go install github.com/vektra/mockery/v2@v2.42.2

.PHONY: codecgen
codecgen: $(codecgen) ## Install codecgen
Expand Down
20 changes: 14 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,20 +167,28 @@ go generate ./...

5. Prepare your development environment:

```bash
export CL_DATABASE_URL=postgresql://127.0.0.1:5432/chainlink_test?sslmode=disable
```
The tests require a postgres database. In turn, the environment variable
`CL_DATABASE_URL` must be set to value that can connect to `_test` database, and the user must be able to create and drop
the given `_test` database.

Note: Other environment variables should not be set for all tests to pass

6. Drop/Create test database and run migrations:
There helper script for initial setup to create an appropriate test user. It requires postgres to be running on localhost at port 5432. You will be prompted for
the `postgres` user password

```bash
make setup-testdb
```

This script will save the `CL_DATABASE_URL` in `.dbenv`

Changes to database require migrations to be run. Similarly, `pull`'ing the repo may require migrations to run.
After the one-time setup above:
```
source .dbenv
make testdb
```

If you do end up modifying the migrations for the database, you will need to rerun

7. Run tests:

```bash
Expand Down
156 changes: 3 additions & 153 deletions charts/chainlink-cluster/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,112 +2,9 @@

Example CL nodes cluster for system level tests

Install `kubefwd` (no nixpkg for it yet, planned)

```
brew install txn2/tap/kubefwd
```

If you want to build images you need [docker](https://docs.docker.com/engine/install/) service running

Enter the shell (from the root project dir)

```
nix develop
```

# Develop

## New cluster

We are using [devspace](https://www.devspace.sh/docs/getting-started/installation?x0=3)

Configure the cluster, see `deployments.app.helm.values` and [values.yaml](./values.yaml) comments for more details

Set up your K8s access

```
export DEVSPACE_IMAGE="..."
./setup.sh ${my-personal-namespace-name-crib}
```

Create a .env file based on the .env.sample file

```sh
cp .env.sample .env
# Fill in the required values in .env
```

Build and deploy the current state of your repository

```
devspace deploy
```

Default `ttl` is `72h`, use `ttl` command to update if you need more time

Valid values are `1h`, `2m`, `3s`, etc. Go time format is invalid `1h2m3s`

```
devspace run ttl ${namespace} 120h
```

If you want to deploy an image tag that is already available in ECR, use:

```
devspace deploy --override-image-tag "<image-tag>"
```

If you want to deploy an image tag from a public ECR repo, use:

```
export DEVSPACE_IMAGE=public.ecr.aws/chainlink/chainlink
devspace deploy --override-image-tag 2.9.0
```

Forward ports to check UI or run tests

```
devspace run connect ${my-personal-namespace-name-crib}
```

List ingress hostnames

```
devspace run ingress-hosts
```

Destroy the cluster

```
devspace purge
```

## Running load tests

Check this [doc](../../integration-tests/load/ocr/README.md)

If you used `devspace dev ...` always use `devspace reset pods` to switch the pods back

# Helm

If you would like to use `helm` directly, please uncomment data in `values.yaml`

## Install from local files

```
helm install -f values.yaml cl-cluster .
```

Forward all apps (in another terminal)

```
sudo kubefwd svc -n cl-cluster
```

Then you can connect and run your tests

## Install from release
Note: The setup below doesn't work at the moment.

Add the repository

Expand All @@ -130,6 +27,7 @@ helm install -f values.yaml cl-cluster .
```

## Create a new release
Note: The setup below doesn't work at the moment.

Bump version in `Chart.yml` add your changes and add `helm_release` label to any PR to trigger a release

Expand All @@ -143,52 +41,4 @@ helm test cl-cluster

```
helm uninstall cl-cluster
```

# Grafana dashboard

We are using [Grabana](https://github.com/K-Phoen/grabana) lib to create dashboards programmatically

You can also select dashboard platform in `INFRA_PLATFORM` either `kubernetes` or `docker`

You can select the dashboard panels with `PANELS_INCLUDED` which is a list of panel names separated by comma
If you don't specify it will include core panels by default

```
export LOKI_TENANT_ID=promtail
export LOKI_URL=...
export GRAFANA_URL=...
export GRAFANA_TOKEN=...
export PROMETHEUS_DATA_SOURCE_NAME=Thanos
export LOKI_DATA_SOURCE_NAME=Loki
export INFRA_PLATFORM=kubernetes
export GRAFANA_FOLDER=DashboardCoreDebug
export DASHBOARD_NAME=CL-Cluster
devspace run dashboard_deploy
```

Open Grafana folder `DashboardCoreDebug` and find dashboard `ChainlinkClusterDebug`

# Testing

Deploy your dashboard and run soak/load [tests](../../integration-tests/load/), check [README](../../integration-tests/README.md) for further explanations

```
devspace run dashboard_deploy
devspace run workload
devspace run dashboard_test
```

# Local Testing

Go to [dashboard-lib](../../dashboard) and link the modules locally

```
cd dashboard
pnpm link --global
cd charts/chainlink-cluster/dashboard/tests
pnpm link --global dashboard-tests
```

Then run the tests with commands mentioned above
```
2 changes: 1 addition & 1 deletion common/client/mock_head_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion common/client/mock_node_client_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion common/client/mock_node_selector_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion common/client/mock_node_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion common/client/mock_rpc_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion common/client/mock_send_only_client_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion common/client/mock_send_only_node_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion common/headtracker/mocks/head_trackable.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion common/headtracker/mocks/head_tracker.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion common/txmgr/mocks/tx_manager.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion common/txmgr/types/mocks/forwarder_manager.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion common/txmgr/types/mocks/key_store.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion common/txmgr/types/mocks/reaper_chain_config.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion common/txmgr/types/mocks/tx_attempt_builder.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion common/txmgr/types/mocks/tx_store.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion common/txmgr/types/mocks/tx_strategy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion common/types/mocks/head.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions contracts/.changeset/hungry-days-occur.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@chainlink/contracts": patch
---

set cleanup #bugfix
Loading

0 comments on commit 64790be

Please sign in to comment.