Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NVPTX target specification #57937

Merged
merged 8 commits into from
Feb 2, 2019
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,8 @@ matrix:
if: branch = auto
- env: IMAGE=mingw-check
if: type = pull_request OR branch = auto
- env: IMAGE=nvptx-cuda
if: branch = auto

- stage: publish toolstate
if: branch = master AND type = push
Expand Down
1 change: 1 addition & 0 deletions src/bootstrap/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -831,6 +831,7 @@ impl Build {
!target.contains("msvc") &&
!target.contains("emscripten") &&
!target.contains("wasm32") &&
!target.contains("nvptx") &&
!target.contains("fuchsia") {
Some(self.cc(target))
} else {
Expand Down
4 changes: 2 additions & 2 deletions src/bootstrap/sanity.rs
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ pub fn check(build: &mut Build) {
panic!("the iOS target is only supported on macOS");
}

if target.contains("-none-") {
if target.contains("-none-") || target.contains("nvptx") {
if build.no_std(*target).is_none() {
let target = build.config.target_config.entry(target.clone())
.or_default();
Expand All @@ -165,7 +165,7 @@ pub fn check(build: &mut Build) {
}

if build.no_std(*target) == Some(false) {
panic!("All the *-none-* targets are no-std targets")
panic!("All the *-none-* and nvptx* targets are no-std targets")
}
}

Expand Down
1 change: 1 addition & 0 deletions src/ci/docker/dist-various-2/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ ENV TARGETS=$TARGETS,x86_64-sun-solaris
ENV TARGETS=$TARGETS,x86_64-unknown-linux-gnux32
ENV TARGETS=$TARGETS,x86_64-unknown-cloudabi
ENV TARGETS=$TARGETS,x86_64-fortanix-unknown-sgx
ENV TARGETS=$TARGETS,nvptx64-nvidia-cuda

ENV X86_FORTANIX_SGX_LIBS="/x86_64-fortanix-unknown-sgx/lib/"

Expand Down
18 changes: 18 additions & 0 deletions src/ci/docker/nvptx-cuda/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
FROM ubuntu:18.04

RUN apt-get update
RUN apt-get install -y --no-install-recommends \
g++ make file curl ca-certificates python git \
cmake sudo gdb

# FIXME: setup `ptx-linker` CI for automatic binary releases.
RUN curl -sL https://github.com/denzp/rust-ptx-linker/releases/download/v0.9.0-alpha/rust-ptx-linker.linux64.tar.gz | \
tar -xzvC /usr/bin

COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh

ENV TARGETS=nvptx64-nvidia-cuda

ENV SCRIPT python2.7 /checkout/x.py test --target $TARGETS \
src/test/run-make
6 changes: 6 additions & 0 deletions src/librustc/ty/context.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1675,6 +1675,12 @@ impl<'a, 'gcx, 'tcx> TyCtxt<'a, 'gcx, 'tcx> {
}
false
}

/// Determine whether identifiers in the assembly have strict naming rules.
/// Currently, only NVPTX* targets need it.
pub fn has_strict_asm_symbol_naming(&self) -> bool {
self.gcx.sess.target.target.arch.contains("nvptx")
}
}

impl<'a, 'tcx> TyCtxt<'a, 'tcx, 'tcx> {
Expand Down
1 change: 1 addition & 0 deletions src/librustc_codegen_ssa/back/link.rs
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,7 @@ pub fn linker_and_flavor(sess: &Session) -> (PathBuf, LinkerFlavor) {
LinkerFlavor::Ld => "ld",
LinkerFlavor::Msvc => "link.exe",
LinkerFlavor::Lld(_) => "lld",
LinkerFlavor::PtxLinker => "rust-ptx-linker",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm… so we are depending on external project by default here. Something that users are very unlikely to have installed by default.

I wonder what the error looks like when rust-ptx-linker is not in $PATH. Perhaps it would make sense to ship rust-ptx-linker as part of rustlib like we do with e.g. lld component for some targets...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's true. On the other hand, to get started with CUDA development in Rust, users will follow either docs or tutorials, where it should be mentioned how to setup the environment.

The error message is, currently:

error: linker `rust-ptx-linker` not found
  |
  = note: No such file or directory (os error 2)

error: aborting due to previous error

Personally, I would love to have the linker to be shipped via rustup. But perhaps, it will be preferable to implement this as a separate PR?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But perhaps, it will be preferable to implement this as a separate PR?

Sure.

}), flavor)),
(Some(linker), None) => {
let stem = if linker.extension().and_then(|ext| ext.to_str()) == Some("exe") {
Expand Down
125 changes: 125 additions & 0 deletions src/librustc_codegen_ssa/back/linker.rs
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,10 @@ impl LinkerInfo {
LinkerFlavor::Lld(LldFlavor::Wasm) => {
Box::new(WasmLd::new(cmd, sess, self)) as Box<dyn Linker>
}

LinkerFlavor::PtxLinker => {
Box::new(PtxLinker { cmd, sess }) as Box<dyn Linker>
}
}
}
}
Expand Down Expand Up @@ -1080,3 +1084,124 @@ fn exported_symbols(tcx: TyCtxt, crate_type: CrateType) -> Vec<String> {

symbols
}

/// Much simplified and explicit CLI for the NVPTX linker. The linker operates
/// with bitcode and uses LLVM backend to generate a PTX assembly.
pub struct PtxLinker<'a> {
cmd: Command,
sess: &'a Session,
}

impl<'a> Linker for PtxLinker<'a> {
fn link_rlib(&mut self, path: &Path) {
self.cmd.arg("--rlib").arg(path);
}

fn link_whole_rlib(&mut self, path: &Path) {
self.cmd.arg("--rlib").arg(path);
}

fn include_path(&mut self, path: &Path) {
self.cmd.arg("-L").arg(path);
}

fn debuginfo(&mut self) {
self.cmd.arg("--debug");
}

fn add_object(&mut self, path: &Path) {
self.cmd.arg("--bitcode").arg(path);
}

fn args(&mut self, args: &[String]) {
self.cmd.args(args);
}

fn optimize(&mut self) {
self.cmd.arg(match self.sess.opts.optimize {
Copy link
Member

@nagisa nagisa Jan 27, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels suspect. I’m not sure the optimisation level for the leaf crate should influence the optimisation level for the whole crate graph. It feels like this should at least in some way depend on the LTO flag?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I followed args pattern from other linkers here.
At the moment the linker runs both LTO and optimisation passes over the final (complete) module when -O{1,2,3} is specified.

Does Rust runs optimisations before emitting bitcode object files? If yes, it can be avoided on linker's side then.

Copy link
Member

@nagisa nagisa Jan 28, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Rust runs optimisations before emitting bitcode object files?

Yes it does.

I followed args pattern from other linkers here.

For gcc and/or clang optimization flags during linkage mean very little if anything at all AFAIK. It definitely does not invoke some sort of global program re-optimisation, which is what happens with ptx-linker, it seems.

Such global-program optimisation is what lto would usually do, which is why I suggested that perhaps that flag is what should be accounted for :)

That could be left as a future endeavour as well, but invoking “LTO” just from -Copt-level feels wrong.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanation and suggestion, I overlooked Session::lto(&self) before!

I've also changed ptx-linker to not perform final global optimisation - indeed it didn't affect much.

OptLevel::No => "-O0",
OptLevel::Less => "-O1",
OptLevel::Default => "-O2",
OptLevel::Aggressive => "-O3",
OptLevel::Size => "-Os",
OptLevel::SizeMin => "-Os"
});
}

fn output_filename(&mut self, path: &Path) {
self.cmd.arg("-o").arg(path);
}

fn finalize(&mut self) -> Command {
::std::mem::replace(&mut self.cmd, Command::new(""))
}

fn link_dylib(&mut self, _lib: &str) {
panic!("external dylibs not supported")
}

fn link_rust_dylib(&mut self, _lib: &str, _path: &Path) {
panic!("external dylibs not supported")
}

fn link_staticlib(&mut self, _lib: &str) {
panic!("staticlibs not supported")
}

fn link_whole_staticlib(&mut self, _lib: &str, _search_path: &[PathBuf]) {
panic!("staticlibs not supported")
}

fn framework_path(&mut self, _path: &Path) {
panic!("frameworks not supported")
}

fn link_framework(&mut self, _framework: &str) {
panic!("frameworks not supported")
}

fn position_independent_executable(&mut self) {
}

fn full_relro(&mut self) {
}

fn partial_relro(&mut self) {
}

fn no_relro(&mut self) {
}

fn build_static_executable(&mut self) {
}

fn gc_sections(&mut self, _keep_metadata: bool) {
}

fn pgo_gen(&mut self) {
}

fn no_default_libraries(&mut self) {
}

fn build_dylib(&mut self, _out_filename: &Path) {
}

fn export_symbols(&mut self, _tmpdir: &Path, _crate_type: CrateType) {
}

fn subsystem(&mut self, _subsystem: &str) {
}

fn no_position_independent_executable(&mut self) {
}

fn group_start(&mut self) {
}

fn group_end(&mut self) {
}

fn cross_lang_lto(&mut self) {
}
}
1 change: 1 addition & 0 deletions src/librustc_codegen_utils/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
#![feature(nll)]
#![allow(unused_attributes)]
#![feature(rustc_diagnostic_macros)]
#![feature(in_band_lifetimes)]

#![recursion_limit="256"]

Expand Down
Loading