Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cross-compile builder to Windows for PRs on Travis #49866

Merged
merged 1 commit into from
Apr 14, 2018

Conversation

Mark-Simulacrum
Copy link
Member

@Mark-Simulacrum Mark-Simulacrum commented Apr 11, 2018

I chose a completely arbitrary windows target here (I have no idea what's best, we could do multiple -- they are relatively fast).

r? @alexcrichton

@rust-highfive
Copy link
Collaborator

warning Warning warning

  • These commits modify submodules.

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Apr 11, 2018
@Mark-Simulacrum
Copy link
Member Author

Oh, and I should cc @aidanhs and probably @kennytm

@TimNN
Copy link
Contributor

TimNN commented Apr 11, 2018

Your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
[00:00:07] error: Server does not allow request for unadvertised object e0bb71ae93695d01cbeaf5dd7a43f0a7b40fb32f
[00:00:07] Fetched in submodule path 'src/libcompiler_builtins', but it did not contain e0bb71ae93695d01cbeaf5dd7a43f0a7b40fb32f. Direct fetching of that commit failed.
---
[00:00:08] error: Server does not allow request for unadvertised object e0bb71ae93695d01cbeaf5dd7a43f0a7b40fb32f
[00:00:08] Fetched in submodule path 'src/libcompiler_builtins', but it did not contain e0bb71ae93695d01cbeaf5dd7a43f0a7b40fb32f. Direct fetching of that commit failed.
---
[00:00:09] error: Server does not allow request for unadvertised object e0bb71ae93695d01cbeaf5dd7a43f0a7b40fb32f
[00:00:09] Fetched in submodule path 'src/libcompiler_builtins', but it did not contain e0bb71ae93695d01cbeaf5dd7a43f0a7b40fb32f. Direct fetching of that commit failed.
---
[00:00:10] error: Server does not allow request for unadvertised object e0bb71ae93695d01cbeaf5dd7a43f0a7b40fb32f
[00:00:10] Fetched in submodule path 'src/libcompiler_builtins', but it did not contain e0bb71ae93695d01cbeaf5dd7a43f0a7b40fb32f. Direct fetching of that commit failed.
---
[00:00:11] error: Server does not allow request for unadvertised object e0bb71ae93695d01cbeaf5dd7a43f0a7b40fb32f
[00:00:11] Fetched in submodule path 'src/libcompiler_builtins', but it did not contain e0bb71ae93695d01cbeaf5dd7a43f0a7b40fb32f. Direct fetching of that commit failed.
[00:00:11] The command has failed after 5 attempts.
---
$ ls -lat $HOME/Library/Logs/DiagnosticReports/
ls: cannot access /home/travis/Library/Logs/DiagnosticReports/: No such file or directory
travis_time:end:0fce8196:start=1523416322650234846,finish=1523416322656769057,duration=6534211
travis_fold:end:after_failure.2
travis_fold:start:after_failure.3
travis_time:start:2b17b0e2
$ find $HOME/Library/Logs/DiagnosticReports -type f -name '*.crash' -not -name '*.stage2-*.crash' -not -name 'com.apple.CoreSimulator.CoreSimulatorService-*.crash' -exec printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" {} \; -exec head -750 {} \; -exec echo travis_fold":"end:crashlog \; || true
find: `/home/travis/Library/Logs/DiagnosticReports': No such file or directory
travis_time:end:2b17b0e2:start=1523416322663415776,finish=1523416322669924092,duration=6508316
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:059b9345
$ dmesg | grep -i kill
[   10.607272] init: failsafe main process (1093) killed by TERM signal

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

Copy link
Member

@alexcrichton alexcrichton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

Looking at Travis it looks like this took ~6m, although I don't think it's checking librustc yet. I wonder if to solve the miniz-sys issue we could create a shim shell script C compiler which basically does nothing? Or otherwise provide overrides for the relevant build scripts?

Additionally, would it be possible to skip the submodule updates in this PR? I think most of them shouldn't be necessary other than maybe libcompiler_builtins, right?

@@ -17,6 +17,10 @@ use std::process::Command;
use build_helper::{run, native_lib_boilerplate};

fn main() {
if *"1" == *env::var_os("CHECK").unwrap_or_default() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be called RUSTBUILD_CHECK and that way I think we could skip the == *"1" and instead just check for its presence

@Mark-Simulacrum
Copy link
Member Author

We need submodule updates for at least src/tools/cargo and src/tools/rls I think (and maybe others) since Cargo needs those to resolve the Cargo.lock/Cargo.toml configuration IIRC.

@alexcrichton
Copy link
Member

Ah good point yeah, it may be that the submodules are too entangled to remove them from being updated

@Mark-Simulacrum Mark-Simulacrum force-pushed the pr-travis-windows branch 2 times, most recently from 45e747c to fe63344 Compare April 11, 2018 16:07
@Mark-Simulacrum
Copy link
Member Author

Note: Making all build scripts no-ops doesn't work as winapi (at least) needs it's build script to run in order to compile (or so I assume). That means we'll probably need a whitelist -- how do you feel about that, @alexcrichton? Is a whitelist fine? I'm not really happy with that solution/answer...

@alexcrichton
Copy link
Member

Hm ok we probably don't want to bend over backwards too much as this may run the risk of causing more bugs than it discovers, so maybe we could use a docker image with a MinGW toolchain and go from there?

The main thing we want to skip is LLVM mostly

@TimNN
Copy link
Contributor

TimNN commented Apr 11, 2018

Your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
$ ls -lat $HOME/Library/Logs/DiagnosticReports/
ls: cannot access /home/travis/Library/Logs/DiagnosticReports/: No such file or directory
travis_time:end:00c61524:start=1523462589305727837,finish=1523462589311236929,duration=5509092
travis_fold:end:after_failure.2
travis_fold:start:after_failure.3
travis_time:start:013423bb
$ find $HOME/Library/Logs/DiagnosticReports -type f -name '*.crash' -not -name '*.stage2-*.crash' -not -name 'com.apple.CoreSimulator.CoreSimulatorService-*.crash' -exec printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" {} \; -exec head -750 {} \; -exec echo travis_fold":"end:crashlog \; || true
find: `/home/travis/Library/Logs/DiagnosticReports': No such file or directory
travis_time:end:013423bb:start=1523462589316487295,finish=1523462589321653722,duration=5166427
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:062f7100
$ dmesg | grep -i kill
[   11.259908] init: failsafe main process (1094) killed by TERM signal

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@TimNN
Copy link
Contributor

TimNN commented Apr 11, 2018

Your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
$ ls -lat $HOME/Library/Logs/DiagnosticReports/
ls: cannot access /home/travis/Library/Logs/DiagnosticReports/: No such file or directory
travis_time:end:111f63b0:start=1523463323129364932,finish=1523463323139017390,duration=9652458
travis_fold:end:after_failure.2
travis_fold:start:after_failure.3
travis_time:start:33269af8
$ find $HOME/Library/Logs/DiagnosticReports -type f -name '*.crash' -not -name '*.stage2-*.crash' -not -name 'com.apple.CoreSimulator.CoreSimulatorService-*.crash' -exec printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" {} \; -exec head -750 {} \; -exec echo travis_fold":"end:crashlog \; || true
find: `/home/travis/Library/Logs/DiagnosticReports': No such file or directory
travis_time:end:33269af8:start=1523463323147430575,finish=1523463323156014950,duration=8584375
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:3370a7f0
$ dmesg | grep -i kill
[   11.520591] init: failsafe main process (1094) killed by TERM signal

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@Mark-Simulacrum
Copy link
Member Author

@alexcrichton Okay, so I've gone with the docker approach now and it compiles std, test, and rustc. I think this is pretty much ready to land from my perspective pending any concerns from you; there are effectively no rustbuild changes to speak of: just making rustc get built on check for --target (not --host) builds.

Copy link
Member

@alexcrichton alexcrichton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! Just a few minor nits

.travis.yml Outdated
@@ -176,6 +176,8 @@ matrix:
if: branch = auto
- env: IMAGE=x86_64-gnu-distcheck
if: branch = auto
- env: IMAGE=mingw-check
if: type = pull_request
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd actually run this on the auto branch as well just to be extra sure it's always passing

COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh

ENV SCRIPT python2.7 ../x.py check --target=i686-pc-windows-gnu -vv
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The -vv here can probably be removed

@@ -64,7 +64,6 @@ pub struct Rustc {

impl Step for Rustc {
type Output = ();
const ONLY_HOSTS: bool = true;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of removing this, would --host=i686-pc-windows-gnu work below?

@Mark-Simulacrum
Copy link
Member Author

Nits fixed.

@alexcrichton
Copy link
Member

@bors: r+

@bors
Copy link
Contributor

bors commented Apr 12, 2018

📌 Commit de34533 has been approved by alexcrichton

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 12, 2018
kennytm added a commit to kennytm/rust that referenced this pull request Apr 13, 2018
…r=alexcrichton

Cross-compile builder to Windows for PRs on Travis

I chose a completely arbitrary windows target here (I have no idea what's best, we could do multiple -- they are relatively fast).

r? @alexcrichton
kennytm added a commit to kennytm/rust that referenced this pull request Apr 14, 2018
…r=alexcrichton

Cross-compile builder to Windows for PRs on Travis

I chose a completely arbitrary windows target here (I have no idea what's best, we could do multiple -- they are relatively fast).
bors added a commit that referenced this pull request Apr 14, 2018
Rollup of 14 pull requests

Successful merges: #49908, #49876, #49916, #49951, #49465, #49922, #49866, #49915, #49886, #49913, #49852, #49958, #49871, #49864

Failed merges:
@bors bors merged commit de34533 into rust-lang:master Apr 14, 2018
@Mark-Simulacrum Mark-Simulacrum deleted the pr-travis-windows branch June 8, 2019 13:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants