Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run CI dependency checks on a Mac, to properly detect iOS-specific dependencies #3565

Merged
merged 1 commit into from
Sep 10, 2020

Conversation

rfk
Copy link
Contributor

@rfk rfk commented Sep 10, 2020

Trying to debug failing dependency checks in CI that we can't reproduce locally.

@codecov-commenter
Copy link

codecov-commenter commented Sep 10, 2020

Codecov Report

Merging #3565 into main will increase coverage by 7.20%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3565      +/-   ##
==========================================
+ Coverage   69.47%   76.67%   +7.20%     
==========================================
  Files         230      230              
  Lines       30879    30879              
==========================================
+ Hits        21453    23677    +2224     
+ Misses       9426     7202    -2224     
Impacted Files Coverage Δ
components/fxa-client/src/device.rs 78.21% <0.00%> (+1.83%) ⬆️
components/support/guid/src/lib.rs 97.94% <0.00%> (+2.56%) ⬆️
components/viaduct/src/headers.rs 72.47% <0.00%> (+2.75%) ⬆️
components/support/rc_crypto/nss/src/ec.rs 94.49% <0.00%> (+3.21%) ⬆️
components/sync15/src/key_bundle.rs 85.58% <0.00%> (+3.60%) ⬆️
components/support/sync15-traits/src/telemetry.rs 92.39% <0.00%> (+5.07%) ⬆️
components/sync15/src/bso_record.rs 98.48% <0.00%> (+5.55%) ⬆️
components/support/rc_crypto/src/hawk_crypto.rs 89.70% <0.00%> (+5.88%) ⬆️
components/sync15/src/request.rs 95.25% <0.00%> (+6.67%) ⬆️
...ents/support/sync15-traits/src/server_timestamp.rs 87.71% <0.00%> (+7.01%) ⬆️
... and 28 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9ce6889...0770a36. Read the comment docs.

@rfk
Copy link
Contributor Author

rfk commented Sep 10, 2020

OK so, when calculating the dependency list in CI, this is discarding some but not all of the new dependencies introduced by glean:

RuntimeError: Dependency details have changed from those in ./DEPENDENCIES.md:
--- 
+++ 
@@ -12,7 +12,6 @@
 * [MIT License: bincode](#mit-license-bincode)
 * [MIT License: bytes](#mit-license-bytes)
 * [MIT License: caseless](#mit-license-caseless)
-* [MIT License: dashmap](#mit-license-dashmap)
 * [MIT License: h2](#mit-license-h2)
 * [MIT License: http-body](#mit-license-http-body)
 * [MIT License: hyper](#mit-license-hyper)
@@ -24,7 +23,6 @@
 * [MIT License: mio](#mit-license-mio)
 * [MIT License: openssl-sys](#mit-license-openssl-sys)
 * [MIT License: ordered-float](#mit-license-ordered-float)
-* [MIT License: oslog](#mit-license-oslog)
 * [MIT License: schannel](#mit-license-schannel)
 * [MIT License: slab](#mit-license-slab)
 * [MIT License: synstructure](#mit-license-synstructure)
@@ -434,7 +432,6 @@
 
 The following text applies to code linked from these dependencies:
 [adler](https://github.com/jonas-schievink/adler.git),
-[ahash](https://github.com/tkaitchuck/ahash),
 [android_log-sys](https://github.com/nercury/android_log-sys-rs),
 [android_logger](https://github.com/Nercury/android_logger-rs),
 [anyhow](https://github.com/dtolnay/anyhow),
@@ -444,8 +441,6 @@
 [cc](https://github.com/alexcrichton/cc-rs),
 [cfg-if](https://github.com/alexcrichton/cfg-if),
 [chrono](https://github.com/chronotope/chrono),
-[const-random-macro](https://github.com/tkaitchuck/constrandom),
-[const-random](https://github.com/tkaitchuck/constrandom),
 [core-foundation-sys](https://github.com/servo/core-foundation-rs),
 [core-foundation](https://github.com/servo/core-foundation-rs),
 [crc32fast](https://github.com/srijs/rust-crc32fast),
@@ -508,7 +503,6 @@
 [pin-utils](https://github.com/rust-lang-nursery/pin-utils),
 [pkg-config](https://github.com/rust-lang/pkg-config-rs),
 [ppv-lite86](https://github.com/cryptocorrosion/cryptocorrosion),
-[proc-macro-hack](https://github.com/dtolnay/proc-macro-hack),
 [proc-macro2](https://github.com/alexcrichton/proc-macro2),
 [prost-derive](https://github.com/danburkert/prost),
 [prost](https://github.com/danburkert/prost),

Following the chain of dependencies reported in Cargo.lock, these all seem to be unique dependencies of oslog. Glean itself takes a platform-specific dependency on oslog like this:

[target.'cfg(target_os = "ios")'.dependencies]
oslog = { version = "0.0.3", default-features = false, features = ["logger"] }

So maybe this is not being activated in the build plan on CI, but is being activated in build plans when running locally.

@rfk
Copy link
Contributor Author

rfk commented Sep 10, 2020

Aha! As of #1720 the dependency checks run on a linux machine rather than a Mac.

Unfortunately, this script can't properly detect iOS-specific dependencies when run on anything other than a Mac, for dumb Mac platform lockin reasons. I'm going to revert the above change.

@rfk rfk changed the title Print more context when CI fails due to dependency changes. Run CI dependency checks on a Mac, to properly detect iOS-specific dependencies Sep 10, 2020
…pendencies.

This also prints out more context when CI fails due to dependency changes, to
help with debugging similar problems in future..
@rfk rfk marked this pull request as ready for review September 10, 2020 05:19
@rfk rfk requested a review from eoger September 10, 2020 05:19
@rfk
Copy link
Contributor Author

rfk commented Sep 10, 2020

@eoger I'm going to merge this to unbreak CI, but we can have a broader conversation about the CI costs involved here if necessary. (For the record, I hate this this platform-specific stuff is necessary here 🤮)

@rfk rfk merged commit 5046deb into main Sep 10, 2020
@rfk rfk deleted the fix-dependencies-ci branch September 10, 2020 05:24
@badboy
Copy link
Member

badboy commented Sep 10, 2020

For dependency checks we shouldn't need to compile, so running with --target aarch64-apple-ios should be possible.

@rfk
Copy link
Contributor Author

rfk commented Sep 10, 2020

The specific command that we end up trying to use here would be something like:

cargo +nightly -Z unstable-options build --build-plan --quiet --locked --package megazord_ios --target aarch64-apple-ios

Which fails for me with:

error: Error loading target specification: failed to get iphoneos SDK path: No such file or directory

But it's been a while since we touched this stuff, there may be a different command to use to get the info we need here.

@badboy
Copy link
Member

badboy commented Sep 10, 2020

Ah, I didn't actually check which command gets run.
It's not clear if build plans will stay so I guess this is a good point in time to find out if there's a more stable way to get the info we need.

@rfk
Copy link
Contributor Author

rfk commented Sep 10, 2020

Oh, interesting, thanks!

I had an earlier version of this script that used cargo tree rather than the buildplan, but IIRC it had the same problem with needing to run on a Mac.

@rfk
Copy link
Contributor Author

rfk commented Sep 10, 2020

I filed #3572 to surface the unsupportedness of buildplans as an independent issue for us to deal with.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants