Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for building S.P.CoreLib in a separate job #34166

Merged
merged 31 commits into from
Apr 2, 2020
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
74cb414
Add support for building S.P.CoreLib in a separate job
mangod9 Mar 26, 2020
def5d70
PR Comments.
mangod9 Mar 27, 2020
fc99faa
fix deletion typo
mangod9 Mar 27, 2020
de8f012
Adding step to upload output directory.
mangod9 Mar 27, 2020
34a19e8
update libraries job to depend on corelib instead of product
mangod9 Mar 27, 2020
eb1ee34
reverting libraries dependency change
mangod9 Mar 27, 2020
478b77f
Add OSX and musl_arm64 corelib builds + use full product for mono
mangod9 Mar 28, 2020
a6a2a3b
Another attempt to fix mono iOS case
mangod9 Mar 28, 2020
fbfb35e
propagating runtimeFlavor to libraries base-job.yml
mangod9 Mar 28, 2020
46efc4d
updating DependsOn for libraries job to use corelib
mangod9 Mar 28, 2020
4fb3931
Merge remote-tracking branch 'upstream/master'
mangod9 Mar 28, 2020
57b5d81
Make LIbraries Test Build also depend on Product
mangod9 Mar 28, 2020
6436fb6
download product dependency when testScope is not null
mangod9 Mar 28, 2020
0683d82
reverting a change which is not required
mangod9 Mar 28, 2020
f21a514
Adding runtime.depproj to libraries restore, and pretest.proj to test…
mangod9 Mar 30, 2020
19e6701
Fixing run_test_job to download product artifact
mangod9 Mar 30, 2020
6b73ed5
run pretest.proj on test runs as well.
mangod9 Mar 30, 2020
bb09589
Add libraries -restore to test run
mangod9 Mar 30, 2020
2ec1f5a
Updating pretest to use TargetArchitecture
mangod9 Mar 30, 2020
77dfbcf
Merge remote-tracking branch 'upstream/master'
mangod9 Mar 30, 2020
e6d4882
Moving runtime restore from \restore\dirs.proj
mangod9 Mar 31, 2020
afe6717
fix -allConfigurations build
mangod9 Mar 31, 2020
a082f9c
Adding corelib job to coreclr=and=Libraries.
mangod9 Mar 31, 2020
fec325d
Add corelib to other pipelines.
mangod9 Mar 31, 2020
dfef8c3
libraries test should depend on full product
mangod9 Mar 31, 2020
2e4d00a
remove upload unsigned artifacts step from official build
mangod9 Mar 31, 2020
42dfd0d
Merge remote-tracking branch 'upstream/master'
mangod9 Apr 1, 2020
56bf4c0
PR Comments
mangod9 Apr 1, 2020
18e5190
typo fix :(
mangod9 Apr 1, 2020
e76db3a
Fix run-test-job pretest build
mangod9 Apr 2, 2020
e3c15b0
removed some unused corelib configs.
mangod9 Apr 2, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
121 changes: 121 additions & 0 deletions eng/pipelines/coreclr/templates/build-corelib-job.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
parameters:
archType: ''
buildConfig: ''
condition: true
container: ''
crossrootfsDir: ''
mangod9 marked this conversation as resolved.
Show resolved Hide resolved
isOfficialBuild: false
osGroup: ''
osSubgroup: ''
platform: ''
pool: ''
signBinaries: false
stagedBuild: false
testGroup: ''
timeoutInMinutes: ''
variables: {}

### Product build
jobs:
- template: xplat-pipeline-job.yml
parameters:
buildConfig: ${{ parameters.buildConfig }}
_BuildConfig: ${{ parameters.buildConfig }}
archType: ${{ parameters.archType }}
osGroup: ${{ parameters.osGroup }}
osSubgroup: ${{ parameters.osSubgroup }}
testGroup: ${{ parameters.testGroup }}
helixType: 'build/product/'
enableMicrobuild: true
stagedBuild: ${{ parameters.stagedBuild }}
pool: ${{ parameters.pool }}
condition: ${{ parameters.condition }}

name: ${{ format('coreclr_corelib_build_{0}{1}_{2}_{3}', parameters.osGroup, parameters.osSubgroup, parameters.archType, parameters.buildConfig) }}
displayName: ${{ format('CoreLib Product Build {0}{1} {2} {3}', parameters.osGroup, parameters.osSubgroup, parameters.archType, parameters.buildConfig) }}
mangod9 marked this conversation as resolved.
Show resolved Hide resolved
mangod9 marked this conversation as resolved.
Show resolved Hide resolved

# Run all steps in the container.
# Note that the containers are defined in platform-matrix.yml
container: ${{ parameters.container }}

timeoutInMinutes: ${{ parameters.timeoutInMinutes }}

crossrootfsDir: ${{ parameters.crossrootfsDir }}

gatherAssetManifests: true
variables:
- name: osGroup
value: ${{ parameters.osGroup }}
- name: osSubgroup
value: ${{ parameters.osSubgroup }}
- name: stripSymbolsArg
value: ''
# Strip symbols only on the release build
- ${{ if eq(parameters.buildConfig, 'Release') }}:
- name: stripSymbolsArg
value: '-stripsymbols'
mangod9 marked this conversation as resolved.
Show resolved Hide resolved
- name: compilerArg
value: ''
- name: publishLogsArtifactPrefix
value: 'BuildLogs_CoreLib'
- name: officialBuildIdArg
value: ''
- ${{ if and(eq(variables['System.TeamProject'], 'internal'), ne(variables['Build.Reason'], 'PullRequest')) }}:
- name: officialBuildIdArg
value: '/p:OfficialBuildId=$(Build.BuildNumber)'

- ${{ parameters.variables }}

steps:

# Install native dependencies
mangod9 marked this conversation as resolved.
Show resolved Hide resolved
# Linux builds use docker images with dependencies preinstalled,
# and FreeBSD builds use a build agent with dependencies
# preinstalled, so we only need this step for OSX and Windows.
- ${{ if eq(parameters.osGroup, 'OSX') }}:
- script: sh $(Build.SourcesDirectory)/eng/install-native-dependencies.sh $(osGroup)
displayName: Install native dependencies
- ${{ if eq(parameters.osGroup, 'Windows_NT') }}:
# Necessary to install python
- script: $(Build.SourcesDirectory)\eng\common\init-tools-native.cmd -InstallDirectory $(Build.SourcesDirectory)\native-tools -Force
displayName: Install native dependencies

# # Install internal tools on official builds
# # Since our internal tools are behind an authenticated feed,
# # we need to use the DotNetCli AzDO task to restore from the feed using a service connection.
# # We can't do this from within the build, so we need to do this as a separate step.
- ${{ if and(eq(variables['System.TeamProject'], 'internal'), ne(variables['Build.Reason'], 'PullRequest')) }}:
- template: /eng/pipelines/common/restore-internal-tools.yml
parameters:
installDotnet: true

# Build Private CoreLib
- script: $(Build.SourcesDirectory)$(dir)build$(scriptExt) -subsetCategory coreclr -subset corelib $(crossArg) -arch $(archType) -c $(buildConfig) $(officialBuildIdArg) -ci
mangod9 marked this conversation as resolved.
Show resolved Hide resolved
displayName: Build System.Private.CoreLib only
mangod9 marked this conversation as resolved.
Show resolved Hide resolved


# Publish corelib output directory for consumption by libraries.
- template: /eng/pipelines/common/upload-artifact-step.yml
parameters:
rootFolder: $(buildProductRootFolderPath)
includeRootFolder: false
archiveType: $(archiveType)
tarCompression: $(tarCompression)
archiveExtension: $(archiveExtension)
artifactName: $(corelibProductArtifactName)
displayName: 'corelib build'
mangod9 marked this conversation as resolved.
Show resolved Hide resolved

# Save packages using the prepare-signed-artifacts format.
mangod9 marked this conversation as resolved.
Show resolved Hide resolved
- ${{ if eq(parameters.isOfficialBuild, true) }}:
- template: /eng/pipelines/common/upload-unsigned-artifacts-step.yml
parameters:
name: ${{ parameters.platform }}

# Publish Logs
- task: PublishPipelineArtifact@1
displayName: Publish Logs
inputs:
targetPath: $(Build.SourcesDirectory)/artifacts/log
artifactName: '$(publishLogsArtifactPrefix)_$(osGroup)$(osSubgroup)_$(archType)_$(buildConfig)'
continueOnError: true
condition: always()
3 changes: 3 additions & 0 deletions eng/pipelines/coreclr/templates/xplat-pipeline-job.yml
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,9 @@ jobs:
- name: buildProductRootFolderPath
value: '$(Build.SourcesDirectory)/artifacts/bin/coreclr/$(osGroup).$(archType).$(buildConfigUpper)'

- name: corelibProductArtifactName
mangod9 marked this conversation as resolved.
Show resolved Hide resolved
value: 'CoreLib_$(osGroup)$(osSubgroup)_$(archType)_$(buildConfig)'

- name: managedTestArtifactName
value: 'CoreCLRManagedTestArtifacts_${{ parameters.managedTestBuildOsGroup }}${{ parameters.managedTestBuildOsSubgroup }}_$(archType)_$(buildConfig)'

Expand Down
2 changes: 1 addition & 1 deletion eng/pipelines/libraries/base-job.yml
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ jobs:

- ${{ if ne(parameters.liveRuntimeBuildConfig, '') }}:
- _runtimeDownloadPath: '$(Build.SourcesDirectory)/artifacts/transport/${{ parameters.runtimeFlavor }}'
- _runtimeArtifactName: '$(runtimeFlavorName)Product_${{ parameters.osGroup }}${{ parameters.osSubgroup }}_${{ parameters.archType }}_${{ parameters.liveRuntimeBuildConfig }}'
mangod9 marked this conversation as resolved.
Show resolved Hide resolved
- _runtimeArtifactName: 'CoreLib_${{ parameters.osGroup }}${{ parameters.osSubgroup }}_${{ parameters.archType }}_${{ parameters.liveRuntimeBuildConfig }}'
mangod9 marked this conversation as resolved.
Show resolved Hide resolved
- _runtimeArtifactsPathArg: ' /p:RuntimeArtifactsPath=$(_runtimeDownloadPath)'
- _testRunNamePrefixSuffix: $(runtimeFlavorName)_${{ parameters.liveRuntimeBuildConfig }}

Expand Down
38 changes: 38 additions & 0 deletions eng/pipelines/runtime.yml
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,44 @@ jobs:
- eng/pipelines/mono/*
- eng/pipelines/libraries/*

#
mangod9 marked this conversation as resolved.
Show resolved Hide resolved
# Build CoreLib checked
#
- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/coreclr/templates/build-corelib-job.yml
buildConfig: checked
mangod9 marked this conversation as resolved.
Show resolved Hide resolved
platforms:
- Linux_x64
- Linux_arm
- Linux_arm64
- Linux_musl_x64
- Windows_NT_x86
- Windows_NT_x64
- Windows_NT_arm
- Windows_NT_arm64
jobParameters:
testGroup: innerloop

#
# Build CoreLib release
#
- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/coreclr/templates/build-corelib-job.yml
buildConfig: release
platforms:
- Linux_x64
- Linux_arm
- Linux_arm64
- Linux_musl_x64
- Windows_NT_x86
- Windows_NT_x64
- Windows_NT_arm
- Windows_NT_arm64
jobParameters:
testGroup: innerloop

#
# Build CoreCLR checked
# Only when CoreCLR is changed
Expand Down