-
Notifications
You must be signed in to change notification settings - Fork 9.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[testing framework] Add documentation #33454
Conversation
f72c6a9
to
b455c33
Compare
Reference: #398 Reference: hashicorp/terraform#33454 Adds support for the `terraform test` command, which currently supports JSON machine-readable output and one flag for configuring the tests directory away from the command default of `tests`. The command will return a non-zero status if any of the tests fail, which returns an error back to callers of the `Test` function. If consumers need access to the pass/fail test results, the terraform-json Go module will need to be enhanced to support the test summary JSON, e.g. ``` {"@Level":"info","@message":"Failure! 0 passed, 1 failed.","@module":"terraform.ui","@timestamp":"2023-07-25T10:03:42.980799-04:00","test_summary":{"status":"fail","passed":0,"failed":1,"errored":0,"skipped":0},"type":"test_summary"} ``` Output of new end-to-end testing: ``` $ TFEXEC_E2ETEST_VERSIONS=1.5.3,1.6.0-alpha20230719 go test -count=1 -run='TestTest' -v ./tfexec/internal/e2etest ... --- PASS: TestTest (9.50s) --- SKIP: TestTest/test_command_passing-1.5.3 (4.06s) --- PASS: TestTest/test_command_passing-1.6.0-alpha20230719 (5.44s) ... --- PASS: TestTestError (0.48s) --- SKIP: TestTestError/test_command_failing-1.5.3 (0.27s) --- PASS: TestTestError/test_command_failing-1.6.0-alpha20230719 (0.21s) ```
@liamcervante it would be great to update the machine-readable UI documentation as part of this as well 👍 |
66c5d9b
to
bdda1cd
Compare
bdda1cd
to
ebd2ba9
Compare
00b0247
to
3763b40
Compare
3763b40
to
6650aa6
Compare
6650aa6
to
243a7ed
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't quite get through, but I'll finish this work first thing tomorrow! It is looking super good so far! Here is the first part of my review, and please feel free to push back on anything that doesn't feel good! ✨
Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this looks great overall! I have a bunch of small style changes, but I think the content is wonderful ✨
Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>
b1bc69d
to
77712db
Compare
Reference: #398 Reference: hashicorp/terraform#33454 Adds support for the `terraform test` command, which currently supports JSON machine-readable output and one flag for configuring the tests directory away from the command default of `tests`. The command will return a non-zero status if any of the tests fail, which returns an error back to callers of the `Test` function. If consumers need access to the pass/fail test results, the terraform-json Go module will need to be enhanced to support the test summary JSON, e.g. ``` {"@Level":"info","@message":"Failure! 0 passed, 1 failed.","@module":"terraform.ui","@timestamp":"2023-07-25T10:03:42.980799-04:00","test_summary":{"status":"fail","passed":0,"failed":1,"errored":0,"skipped":0},"type":"test_summary"} ``` Output of new end-to-end testing: ``` $ TFEXEC_E2ETEST_VERSIONS=1.5.3,1.6.0-alpha20230719 go test -count=1 -run='TestTest' -v ./tfexec/internal/e2etest ... --- PASS: TestTest (9.50s) --- SKIP: TestTest/test_command_passing-1.5.3 (4.06s) --- PASS: TestTest/test_command_passing-1.6.0-alpha20230719 (5.44s) ... --- PASS: TestTestError (0.48s) --- SKIP: TestTestError/test_command_failing-1.5.3 (0.27s) --- PASS: TestTestError/test_command_failing-1.6.0-alpha20230719 (0.21s) ```
Reminder for the merging maintainer: if this is a user-visible change, please update the changelog on the appropriate release branch. |
I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions. |
Add documentation for the Terraform Test command, and update the upgrade guide in preparation for Terraform v1.6.