-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Record and Playback] Only record the tests that have changed #5156
Comments
Some thoughts on how to implement this - Discussion in mochajs/mocha#1998 might be helpful. |
This might also be sufficient to close this other issue: #4879 |
My proposal is to run the playback mode for the tests and if a test fails, then record it. |
What I want to do (though this will change as I dive into the code):
|
Ignore the previous points, here's my draft PR: #7213 My current plan is to finish the unit tests and turn it ready for review. |
This PR intends to add soft-record, a way to only record what hasn't changed. To be able to check whether the tests have changed, we check whether a MD5 hash of the test function's source code has changed from the previous time a recording stored this MD5 to the current test execution. Any new recording will store this MD5 into the recorded file. To only record the tests that have changed, the user must specify the environment variable TEST_MODE, with value "soft-record". Fixes #5156
To reduce the amount of changes per PR, as well as to keep older untraceable tests clean of the folder tree, we should:
Note: To find the changing tests we can either:
a. Use git (either a Node client or child_process) to get the changed files through a diff.
b. Run the playback of each test to see if it doesn't pass, and then add them to the changed-files-pool.
- If so, we need to suppress any console message that might be related to the playback tests before attempting to record them.
- We must also make sure that these playbacks don't add substantial delay to the recordings.
The text was updated successfully, but these errors were encountered: