Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backport of ci: add test-integrations into release/1.14.x #17040

Merged

Conversation

hc-github-team-consul-core
Copy link
Contributor

Backport

This PR is auto-generated from #16915 to be assessed for backporting due to the inclusion of the label backport/1.14.

WARNING automatic cherry-pick of commits failed. Commits will require human attention.

merge conflict error: POST https://api.github.com/repos/hashicorp/consul/merges: 409 Merge conflict []

The below text is copied from the body of the original PR.


Description

This configuration currently works in ENT but fails in OSS. GitHub is helping us look at it. The docker based tests in compatability-tests and upgrade-test fail with intermittent communication failures when docker containers need to talk to each other as is the case for 3 servers joining in a cluster.

As part of troubleshooting that issue, most iteration has been done on this which comments out all of the envoy, nomad, and vault stuff so that it only runs the two jobs with docker based tests.

Remaining items Checklist

  • get docker based compatibility tests and upgrade tests to pass in OSS. They currently pass in ENT.
  • revisit testsplitting for envoy
  • audit commands for GHA vs CircleCI in OSS and ENT
  • ensure envoy, nomad, vault versions match CircleCI for OSS and ENT

Overview of commits

im2nguyen and others added 30 commits February 27, 2023 09:27
* Upgrade ember-cli-string-helpers

* add extra lock change
* Update ingress-gateways.mdx

Added an example of running the HELM install for the ingress gateways using values.yaml

* Apply suggestions from code review

* Update ingress-gateways.mdx

Adds closing back ticks on example command. The suggesting UI strips them out.

---------

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Registering gRPC balancers is thread-unsafe because they are stored in a
global map variable that is accessed without holding a lock. Therefore,
it's expected that balancers are registered _once_ at the beginning of
your program (e.g. in a package `init` function) and certainly not after
you've started dialing connections, etc.

> NOTE: this function must only be called during initialization time
> (i.e. in an init() function), and is not thread-safe.

While this is fine for us in production, it's challenging for tests that
spin up multiple agents in-memory. We currently register a balancer per-
agent which holds agent-specific state that cannot safely be shared.

This commit introduces our own registry that _is_ thread-safe, and
implements the Builder interface such that we can call gRPC's `Register`
method once, on start-up. It uses the same pattern as our resolver
registry where we use the dial target's host (aka "authority"), which is
unique per-agent, to determine which builder to use.
Fixes a regression in #16044

The consul acl token read -self cli command should not require an -accessor-id because typically the persona invoking this would not already know the accessor id of their own token.
…ane in the Envoy compat matrix (#16462)

* Update envoy.mdx
…6408)

* test(gateways): add API Gateway HTTPRoute ParentRef change test

* test(gateways): add checkRouteError helper

* test(gateways): remove EOF check

in CI this seems to sometimes be 'connection reset by peer' instead

* Update test/integration/consul-container/test/gateways/http_route_test.go
* add http url path rewrite

* add Mike's test back in

* update kind to use api.APIGateway
… VERSION file (#16467)

Fixes a regression from #15631 in the output of `consul version` from:

    Consul v1.16.0-dev
    +ent
    Revision 56b86ac+CHANGES

to

    Consul v1.16.0-dev+ent
    Revision 56b86ac+CHANGES
* converted main services page to services overview page

* set up services usage dirs

* added Define Services usage page

* converted health checks everything page to Define Health Checks usage page

* added Register Services and Nodes usage page

* converted Query with DNS to Discover Services and Nodes Overview page

* added Configure DNS Behavior usage page

* added Enable Static DNS Lookups usage page

* added the Enable Dynamic Queries DNS Queries usage page

* added the Configuration dir and overview page - may not need the overview, tho

* fixed the nav from previous commit

* added the Services Configuration Reference page

* added Health Checks Configuration Reference page

* updated service defaults configuraiton entry to new configuration ref format

* fixed some bad links found by checker

* more bad links found by checker

* another bad link found by checker

* converted main services page to services overview page

* set up services usage dirs

* added Define Services usage page

* converted health checks everything page to Define Health Checks usage page

* added Register Services and Nodes usage page

* converted Query with DNS to Discover Services and Nodes Overview page

* added Configure DNS Behavior usage page

* added Enable Static DNS Lookups usage page

* added the Enable Dynamic Queries DNS Queries usage page

* added the Configuration dir and overview page - may not need the overview, tho

* fixed the nav from previous commit

* added the Services Configuration Reference page

* added Health Checks Configuration Reference page

* updated service defaults configuraiton entry to new configuration ref format

* fixed some bad links found by checker

* more bad links found by checker

* another bad link found by checker

* fixed cross-links between new topics

* updated links to the new services pages

* fixed bad links in scale file

* tweaks to titles and phrasing

* fixed typo in checks.mdx

* started updating the conf ref to latest template

* update SD conf ref to match latest CT standard

* Apply suggestions from code review

Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com>

* remove previous version of the checks page

* fixed cross-links

* Apply suggestions from code review

Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com>

---------

Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com>
Does the required dance with the local HTTP endpoint to get the required
data for the jwt based auth setup in Azure. Keeps support for 'legacy'
mode where all login data is passed on via the auth methods parameters.
Refactored check for hardcoded /login fields.
* Changed titles for services pages to sentence style cap

* missed a meta title
* add new release notes
---------

Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
* fix: return error msg if acl policy not found

* changelog

* add test
…s and policies (#16288)

* Deprecate merge-policies and add options add-policy-name/add-policy-id to improve CLI token update command

* deprecate merge-roles fields

* Fix potential flakey tests and update ux to remove 'completely' + typo fixes
* Update v1_15_x.mdx

---------

Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
* Suppress AlreadyRegisteredError to fix test retries

* Remove duplicate sink
Adds support for a jwt token in a file. Simply reads the file and sends
the read in jwt along to the vault login.

It also supports a legacy mode with the jwt string being passed
directly. In which case the path is made optional.
Adds support for Kubernetes jwt/token file based auth. Only needs to
read the file and save the contents as the jwt/token.
NET-2396: refactor test to reduce duplication
NET-2841: PART 1 - refactor NewPeeringCluster to support custom config
@github-actions github-actions bot added pr/dependencies PR specifically updates dependencies of project theme/acls ACL and token generation theme/agent-cache Agent Cache theme/api Relating to the HTTP API interface theme/certificates Related to creating, distributing, and rotating certificates in Consul theme/cli Flags and documentation for the CLI interface theme/config Relating to Consul Agent configuration, including reloading theme/connect Anything related to Consul Connect, Service Mesh, Side Car Proxies theme/consul-terraform-sync Relating to Consul Terraform Sync and Network Infrastructure Automation theme/contributing Additions and enhancements to community contributing materials theme/envoy/xds Related to Envoy support theme/health-checks Health Check functionality theme/internals Serf, Raft, SWIM, Lifeguard, Anti-Entropy, locking topics theme/telemetry Anything related to telemetry or observability theme/tls Using TLS (Transport Layer Security) or mTLS (mutual TLS) to secure communication theme/ui Anything related to the UI type/ci Relating to continuous integration (CI) tooling for testing or releases type/docs Documentation needs to be created/updated/clarified labels Apr 19, 2023
@jmurret jmurret marked this pull request as ready for review April 19, 2023 02:56
@jmurret jmurret requested a review from a team April 19, 2023 02:56
@jmurret jmurret requested review from a team as code owners April 19, 2023 02:56
@jmurret jmurret requested review from dekimsey and jeanneryan and removed request for a team April 19, 2023 02:56
jmurret and others added 2 commits April 18, 2023 20:57
* add test-integrations workflow

* add test-integrations success job

* update vault integration testing versions (#16949)

* change parallelism to 4 forgotestsum.  use env.CONSUL_VERSION so we can see the version.

* use env for repeated values

* match test to circleci

* fix envvar

* fix envvar 2

* fix envvar 3

* fix envvar 4

* fix envvar 5

* make upgrade and compatibility tests match circleci

* run go env to check environment

* debug docker

Signed-off-by: Dan Bond <danbond@protonmail.com>

* debug docker

Signed-off-by: Dan Bond <danbond@protonmail.com>

* revert debug docker

Signed-off-by: Dan Bond <danbond@protonmail.com>

* going back to command that worked 5 days ago for compatibility tests

* Update Envoy versions to reflect changes in #16889

* cd to test dir

* try running ubuntu latest

* update PR with latest changes that work in enterprise

* yaml still sucks

* test GH fix (localhost resolution)

* change for testing

* test splitting and ipv6 lookup for compatibility and upgrade tests

* fix indention

* consul as image name

* remove the on push

* add gotestsum back in

* removing the use of the gotestsum download action

* yaml sucks today just like yesterday

* fixing nomad tests

* worked out the kinks on enterprise

---------

Signed-off-by: Dan Bond <danbond@protonmail.com>
Co-authored-by: John Eikenberry <jae@zhar.net>
Co-authored-by: Dan Bond <danbond@protonmail.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Co-authored-by: Sarah <sthompson@hashicorp.com>
@jmurret jmurret enabled auto-merge (squash) April 19, 2023 02:59
@jmurret jmurret merged commit f1ac45e into release/1.14.x Apr 19, 2023
@jmurret jmurret deleted the backport/jm/test-integrations/strangely-obliging-wombat branch April 19, 2023 03:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr/dependencies PR specifically updates dependencies of project theme/acls ACL and token generation theme/agent-cache Agent Cache theme/api Relating to the HTTP API interface theme/certificates Related to creating, distributing, and rotating certificates in Consul theme/cli Flags and documentation for the CLI interface theme/config Relating to Consul Agent configuration, including reloading theme/connect Anything related to Consul Connect, Service Mesh, Side Car Proxies theme/consul-terraform-sync Relating to Consul Terraform Sync and Network Infrastructure Automation theme/contributing Additions and enhancements to community contributing materials theme/envoy/xds Related to Envoy support theme/health-checks Health Check functionality theme/internals Serf, Raft, SWIM, Lifeguard, Anti-Entropy, locking topics theme/telemetry Anything related to telemetry or observability theme/tls Using TLS (Transport Layer Security) or mTLS (mutual TLS) to secure communication theme/ui Anything related to the UI type/ci Relating to continuous integration (CI) tooling for testing or releases type/docs Documentation needs to be created/updated/clarified
Projects
None yet
Development

Successfully merging this pull request may close these issues.