TrustUpdater can be used to fetch specific trust roots:
* Currently supports fetching
  * ctfe keys and
  * the rekor key
* Caches target files in ~/.cache/sigstore-python/
* Stores metadata in ~/.local/share/sigstore-python/
* Expects to either
  * find the metadata _for the given URL_ in metadata store
  * (for prod and stage only) find the boostrap root.json in
    sigstore/_store

The "API" that TrustUpdater provides is not meant to be final: it is the
minimal one that should fulfill current needs. Nothing uses the TrustUpdater
yet, but it's testable:

>>> from sigstore._tuf import TrustUpdater, DEFAULT_TUF_URL
>>> updater = TrustUpdater(DEFAULT_TUF_URL)
>>> rekor_key_bytes = updater.get_rekor_key()

Co-authored-by: Joshua Lock <jlock@vmware.com>
Co-authored-by: wxjdsr <wxjdsr@126.com>
Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>

CTKeyring:
 * Take bytes as constructore input: this makes it easier to
   feed things from either CLI arguments or the TUF trust updater.
 * Remove tests that no longer make sense. The prod/staging contant
   should still be tested but TUF is now used in the same flows:
   Unsure how to best test this.

Use TUF to find the CTFE and rekor key when using "production" or
"staging".

Note that "staging" is currently untested: I am not sure even the URL
makes sense.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>

Use TUF to get CTFE/Rekor keys in the non-staging, non-production
flow. As before, the assumption is that user wants production
keys in this case.

Refactor TrustUpdater so that it does not do network traffic if
nothing is requested.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>

* Assume that the active fulcio certs in the repository form
  a certificate chain that cryptography can ingest
* Refactor RekorClient construction so that we avoid
  constructing multiple TrustTupdaters

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>

If this is not production or staging but rekor key is not given,
use production: this is what original (non-tuf) code was doing as well.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>

https://tuf-root-staging.storage.googleapis.com/ does work as staging
repository.

There is still a bug somewhere as staging verify currently fails.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>

This is needed to bootstrap the TUF metadata with
--staging.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>

Signed-off-by: William Woodruff <william@trailofbits.com>

Signed-off-by: William Woodruff <william@trailofbits.com>

Signed-off-by: William Woodruff <william@trailofbits.com>

Signed-off-by: William Woodruff <william@trailofbits.com>

Signed-off-by: William Woodruff <william@trailofbits.com>

Signed-off-by: William Woodruff <william@trailofbits.com>

* Bring back RekorClient.production() and RekorClient.staging(): these
  are simple but make the calling code slightly clearer maybe
* Add a TrustUpdater argument to those methods: if you use
  production/staging, you need a TrustUpdater
* The TrustUpdater can not be constructed inside RekorClient
  as other components may need it as well.

It's not perfectly elegant for the caller but it's not horrible either:
  updater = TrustUpdater.staging()
  client = RekorClient.staging(updater)
This design means TrustUpdater does not know anything about the sigstore
mechanisms: it just discovers and downloads files.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>

No functional change, just refactor the target discovery into a single
method.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>

The test doesn't make a lot of sense now that the keys are not being
read from the _store.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>

These are now made available via the _internal.tuf module.

There is still a case to be made for embedding keys and certs in the
wheel (to optimize the first run experience, and the experience for
those who might not persist their caches, e.g. CI systems). But:
 * Testing this without embedded keys first likely makes sense:
   We get more experience and feedback on the trust update system
 * There should be some automated system that updates the embedded keys.
   Otherwise obsolete keys will be embedded and no-one notices.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>

This allows running (otherwise offline) staging tests without network access.
* Add a fixture that mocks tuf.ngclient fetcher: it returns files from
  test assets
* Mark the relevant tests with mock_staging_tuf fixture
* Mark test_verifier_production() as "online": there is no way to test
  production tuf repository offline as it expires every two weeks

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>

Signer.production() and Signer.staging() now require a network
connection for TUF initialization: they can't be used in
parametrized test setup as that happens even if the test is marked
online.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>

This test asserts that we make the network requests that we expect:
* Uses mock staging TUF repository
* Uses empty HOME dir to ensure known starting point for caches
* tests both cold and hot caches

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>

Make sure the rekor key content is correct
* use empty home dir
* use mock TUF staging repository

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>

Signed-off-by: William Woodruff <william@trailofbits.com>

Signed-off-by: William Woodruff <william@trailofbits.com>

Signed-off-by: William Woodruff <william@trailofbits.com>

Signed-off-by: William Woodruff <william@trailofbits.com>

Signed-off-by: William Woodruff <william@trailofbits.com>

Signed-off-by: William Woodruff <william@trailofbits.com>

Signed-off-by: William Woodruff <william@trailofbits.com>

Annoying.

Signed-off-by: William Woodruff <william@trailofbits.com>

Signed-off-by: Alex Cameron <asc@tetsuo.sh>

Signed-off-by: Alex Cameron <asc@tetsuo.sh>

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>

This is a potential improvement, not a necessary one.

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>

Also tweak one annotation (remove unneeded quotes)

Signed-off-by: Jussi Kukkonen <jkukkonen@google.com>

Signed-off-by: William Woodruff <william@trailofbits.com>