Skip to content

Commit

Permalink
fix: Change checkout cache strategy (#35259)
Browse files Browse the repository at this point in the history
Summary:
This PR updates he cache strategy for the `checkout_with_cache command`.

The previous strategy was using three keys in descending priority order to restore from the cache:
* `<< parameters.checkout_base_cache_key >>-{{ arch }}-{{ .Branch }}-{{ .Revision }}`
* `<< parameters.checkout_base_cache_key >>-{{ arch }}-{{ .Branch }}`
*`<< parameters.checkout_base_cache_key >>-{{ arch }}`

When it saves, it always saves using the first key.

The restore works as it follows:
1. It tries to restore the cache using the first key
2. If it fails, it checks whether there is a cache hit for a key that matches the second key as a pattern
3. If it fails, it checks whether there is a cache hit for a key that matches the third pattern
4. Otherwise, it is a cache miss.

This does not work well. Imagine that you submit some code in commit `xxxx` for branch `abc`.
The CI run the first time, it misses all the three keys and checks out the code normally.
Then, it stores the checked out code in the `checkout_key-abc-xxxx` key.

Then, you submit a commit `yyyy` in the same branch.
The CI starts, it tries with the key `checkout_key-abc-yyyy` but it misses
It tries with the key `checkout_key-abc` and it finds the cache for `checkout_key-abc-xxxx` and it restores it, forgetting about your changes.

While doing the release, we created a tag in a commit X. Then we manually had to remove it, but the CI had a cached version of .git with the tag for
the `0.71-stable` branch. And the release failed because the tag was already existing.

### Why this should work

With this solution, we are going to cache using the commit. If there is no cache for a specific commit, it will be a miss. It won't try to be smart and
retrieve the code from previous caches.

This should prevent stale caches and if we manually remove a tag, and then we do a new commit, it should work.

This is a good trade-off that allows to pay the checkout cost only for the first batch of jobs of the pipeline.

**NOTE:** This still won't work if we don't do a new commit.

## Changelog

[General] [Fixed] - Change Cache strategy to avoid cache bumps in Release

Pull Request resolved: #35259

Test Plan: 1. CircleCI must be green

Reviewed By: jacdebug

Differential Revision: D41120895

Pulled By: cipolleschi

fbshipit-source-id: 2b45da01803197dbe4a25a313a9dfc53a976d096
  • Loading branch information
Riccardo Cipolleschi authored and facebook-github-bot committed Nov 8, 2022
1 parent 9cb0261 commit ddba780
Showing 1 changed file with 1 addition and 4 deletions.
5 changes: 1 addition & 4 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -132,10 +132,7 @@ commands:
type: string
steps:
- restore_cache:
keys:
- << parameters.checkout_base_cache_key >>-{{ arch }}-{{ .Branch }}-{{ .Revision }}
- << parameters.checkout_base_cache_key >>-{{ arch }}-{{ .Branch }}-
- << parameters.checkout_base_cache_key >>-{{ arch }}-
key: << parameters.checkout_base_cache_key >>-{{ arch }}-{{ .Branch }}-{{ .Revision }}
- checkout
- save_cache:
key: << parameters.checkout_base_cache_key >>-{{ arch }}-{{ .Branch }}-{{ .Revision }}
Expand Down

0 comments on commit ddba780

Please sign in to comment.