Skip to content

Commit

Permalink
Sync eng/common directory with azure-sdk-tools for PR 1525 (#17717)
Browse files Browse the repository at this point in the history 
* Use SubscriptionId throughout TestResources

Fixes #1454

* Resolve PR feedback

* Default DeleteAfterHours to 48 for SDK team

Also makes a few other adjustments for subscriptions, like restoring the previous one if available and another was specified.

* Resolve PR feedback

* Change deployment mode to Complete

Also fixes an issue where if the user opted not to deploy to the same resource group, the script would continue execution anyway.

* Use consistent aka links to satisfy link checker

Only need it for the new Update-TestResources.ps1 script, but I wanted them to look consistent.

Co-authored-by: Heath Stewart <heaths@microsoft.com>
  • Loading branch information
@azure-sdk @heaths
azure-sdk and heaths authored Apr 1, 2021
1 parent fac3304 commit 2acd938
Show file tree
Hide file tree
Showing 8 changed files with 659 additions and 109 deletions.
133 changes: 89 additions & 44 deletions eng/common/TestResources/New-TestResources.ps1
View file
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ param (
# Azure SDK Developer Playground subscription
[Parameter()]
[ValidatePattern('^[0-9a-f]{8}(-[0-9a-f]{4}){3}-[0-9a-f]{12}$')]
[string] $SubscriptionId = 'faa080af-c1d8-40ad-9cce-e1a450ca5b57',
[string] $SubscriptionId,

[Parameter(ParameterSetName = 'Provisioner', Mandatory = $true)]
[ValidatePattern('^[0-9a-f]{8}(-[0-9a-f]{4}){3}-[0-9a-f]{12}$')]
Expand All @@ -49,8 +49,8 @@ param (
[string] $ProvisionerApplicationSecret,

[Parameter()]
[ValidateRange(0, [int]::MaxValue)]
[int] $DeleteAfterHours,
[ValidateRange(1, [int]::MaxValue)]
[int] $DeleteAfterHours = 48,

[Parameter()]
[string] $Location = '',
Expand Down Expand Up @@ -166,7 +166,7 @@ try {
Log "Generated base name '$BaseName' for CI build"
} elseif (!$BaseName) {
$BaseName = "$UserName$ServiceDirectory"
Log "BaseName was not set. Using default base name: '$BaseName'"
Log "BaseName was not set. Using default base name '$BaseName'"
}

# Make sure pre- and post-scripts are passed formerly required arguments.
Expand Down Expand Up @@ -200,35 +200,74 @@ try {
# Make sure the user is logged in to create a service principal.
$context = Get-AzContext;
if (!$context) {
$subscriptionName = $SubscriptionId
Log 'User not logged in. Logging in now...'
$context = (Connect-AzAccount).Context
}

# Use cache of well-known team subs without having to be authenticated.
$wellKnownSubscriptions = @{
'faa080af-c1d8-40ad-9cce-e1a450ca5b57' = 'Azure SDK Developer Playground'
'a18897a6-7e44-457d-9260-f2854c0aca42' = 'Azure SDK Engineering System'
'2cd617ea-1866-46b1-90e3-fffb087ebf9b' = 'Azure SDK Test Resources'
$currentSubcriptionId = $context.Subscription.Id

# If no subscription was specified, try to select the Azure SDK Developer Playground subscription.
# Ignore errors to leave the automatically selected subscription.
if ($SubscriptionId) {
if ($currentSubcriptionId -ne $SubscriptionId) {
Log "Selecting subscription '$SubscriptionId'"
$null = Select-AzSubscription -Subscription $SubscriptionId

$exitActions += {
Log "Selecting previous subscription '$currentSubcriptionId'"
$null = Select-AzSubscription -Subscription $currentSubcriptionId
}

# Update the context.
$context = Get-AzContext
}
} else {
if ($currentSubcriptionId -ne 'faa080af-c1d8-40ad-9cce-e1a450ca5b57') {
Log "Attempting to select subscription 'Azure SDK Developer Playground (faa080af-c1d8-40ad-9cce-e1a450ca5b57)'"
$null = Select-AzSubscription -Subscription 'faa080af-c1d8-40ad-9cce-e1a450ca5b57' -ErrorAction Ignore

if ($wellKnownSubscriptions.ContainsKey($SubscriptionId)) {
$subscriptionName = '{0} ({1})' -f $wellKnownSubscriptions[$SubscriptionId], $SubscriptionId
# Update the context.
$context = Get-AzContext
}

Log "You are not logged in; connecting to $subscriptionName"
$context = (Connect-AzAccount -Subscription $SubscriptionId).Context
$SubscriptionId = $context.Subscription.Id
$PSBoundParameters['SubscriptionId'] = $SubscriptionId
}

# Use cache of well-known team subs without having to be authenticated.
$wellKnownSubscriptions = @{
'faa080af-c1d8-40ad-9cce-e1a450ca5b57' = 'Azure SDK Developer Playground'
'a18897a6-7e44-457d-9260-f2854c0aca42' = 'Azure SDK Engineering System'
'2cd617ea-1866-46b1-90e3-fffb087ebf9b' = 'Azure SDK Test Resources'
}

# Print which subscription is currently selected.
$subscriptionName = $context.Subscription.Id
if ($wellKnownSubscriptions.ContainsKey($subscriptionName)) {
$subscriptionName = '{0} ({1})' -f $wellKnownSubscriptions[$subscriptionName], $subscriptionName
}

Log "Using subscription '$subscriptionName'"

# Make sure the TenantId is also updated from the current context.
# PSBoundParameters is not updated to avoid confusing parameter sets.
if (!$TenantId) {
$TenantId = $context.Subscription.TenantId
}

# If no test application ID is specified during an interactive session, create a new service principal.
if (!$TestApplicationId) {

# Cache the created service principal in this session for frequent reuse.
$servicePrincipal = if ($AzureTestPrincipal -and (Get-AzADServicePrincipal -ApplicationId $AzureTestPrincipal.ApplicationId)) {
$servicePrincipal = if ($AzureTestPrincipal -and (Get-AzADServicePrincipal -ApplicationId $AzureTestPrincipal.ApplicationId) -and $AzureTestSubscription -eq $SubscriptionId) {
Log "TestApplicationId was not specified; loading cached service principal '$($AzureTestPrincipal.ApplicationId)'"
$AzureTestPrincipal
} else {
Log 'TestApplicationId was not specified; creating a new service principal'
$global:AzureTestPrincipal = New-AzADServicePrincipal -Role Owner
Log "TestApplicationId was not specified; creating a new service principal in subscription '$SubscriptionId'"
$global:AzureTestPrincipal = New-AzADServicePrincipal -Role Owner -Scope "/subscriptions/$SubscriptionId"
$global:AzureTestSubscription = $SubscriptionId

Log "Created service principal '$AzureTestPrincipal'"
Log "Created service principal '$($AzureTestPrincipal.ApplicationId)'"
$AzureTestPrincipal
}

Expand All @@ -251,13 +290,15 @@ try {
if ($ProvisionerApplicationId) {
$null = Disable-AzContextAutosave -Scope Process

Log "Logging into service principal '$ProvisionerApplicationId'"
Log "Logging into service principal '$ProvisionerApplicationId'."
Write-Warning 'Logging into service principal may fail until the principal is fully propagated.'

$provisionerSecret = ConvertTo-SecureString -String $ProvisionerApplicationSecret -AsPlainText -Force
$provisionerCredential = [System.Management.Automation.PSCredential]::new($ProvisionerApplicationId, $provisionerSecret)

# Use the given subscription ID if provided.
$subscriptionArgs = if ($SubscriptionId) {
@{SubscriptionId = $SubscriptionId}
@{Subscription = $SubscriptionId}
} else {
@{}
}
Expand Down Expand Up @@ -292,7 +333,7 @@ try {

# If the ServiceDirectory is an absolute path use the last directory name
# (e.g. D:\foo\bar\ -> bar)
$serviceName = if (Split-Path -IsAbsolute $ServiceDirectory) {
$serviceName = if (Split-Path -IsAbsolute $ServiceDirectory) {
Split-Path -Leaf $ServiceDirectory
} else {
$ServiceDirectory
Expand All @@ -307,16 +348,15 @@ try {
"rg-$BaseName"
}

# Tag the resource group to be deleted after a certain number of hours if specified.
$tags = @{
Creator = $UserName
ServiceDirectory = $ServiceDirectory
}

if ($PSBoundParameters.ContainsKey('DeleteAfterHours')) {
$deleteAfter = [DateTime]::UtcNow.AddHours($DeleteAfterHours)
$tags.Add('DeleteAfter', $deleteAfter.ToString('o'))
}
# Tag the resource group to be deleted after a certain number of hours.
Write-Warning "Any clean-up scripts running against subscription '$SubscriptionId' may delete resource group '$ResourceGroupName' after $DeleteAfterHours hours."
$deleteAfter = [DateTime]::UtcNow.AddHours($DeleteAfterHours).ToString('o')
$tags['DeleteAfter'] = $deleteAfter

if ($CI) {
# Add tags for the current CI job.
Expand Down Expand Up @@ -348,11 +388,15 @@ try {
# New-AzResourceGroup would've written an error and stopped the pipeline by default anyway.
Write-Verbose "Successfully created resource group '$($resourceGroup.ResourceGroupName)'"
}
elseif (!$resourceGroup -and !$PSCmdlet.ShouldProcess($resourceGroupName)) {
# If the -WhatIf flag was passed, there will be no resource group created. Fake it.
$resourceGroup = [PSCustomObject]@{
ResourceGroupName = $resourceGroupName
Location = $Location
elseif (!$resourceGroup) {
if (!$PSCmdlet.ShouldProcess($resourceGroupName)) {
# If the -WhatIf flag was passed, there will be no resource group created. Fake it.
$resourceGroup = [PSCustomObject]@{
ResourceGroupName = $resourceGroupName
Location = $Location
}
} else {
Write-Error "Resource group '$ResourceGroupName' already exists." -Category ResourceExists -RecommendedAction "Delete resource group '$ResourceGroupName', or overwrite it when redeploying."
}
}

Expand Down Expand Up @@ -413,16 +457,16 @@ try {
$lastDebugPreference = $DebugPreference
try {
if ($CI) {
$DebugPreference = "Continue"
$DebugPreference = 'Continue'
}
New-AzResourceGroupDeployment -Name $BaseName -ResourceGroupName $resourceGroup.ResourceGroupName -TemplateFile $templateFile -TemplateParameterObject $templateFileParameters
New-AzResourceGroupDeployment -Name $BaseName -ResourceGroupName $resourceGroup.ResourceGroupName -TemplateFile $templateFile -TemplateParameterObject $templateFileParameters -Mode Complete -Force:$Force
} catch {
Write-Output @"
Write-Output @'
#####################################################
# For help debugging live test provisioning issues, #
# see http://aka.ms/azsdk/engsys/live-test-help, #
#####################################################
"@
'@
throw
} finally {
$DebugPreference = $lastDebugPreference
Expand Down Expand Up @@ -466,7 +510,7 @@ try {

if ($OutFile) {
if (!$IsWindows) {
Write-Host "File option is supported only on Windows"
Write-Host 'File option is supported only on Windows'
}

$outputFile = "$templateFile.env"
Expand Down Expand Up @@ -595,7 +639,11 @@ is passed to the ARM template as 'tenantId'.
Optional subscription ID to use for new resources when logging in as a
provisioner. You can also use Set-AzContext if not provisioning.
The default is the Azure SDK Developer Playground subscription ID.
If you do not specify a SubscriptionId and are not logged in, one will be
automatically selected for you by the Connect-AzAccount cmdlet.
Once you are logged in (or were previously), the selected SubscriptionId
will be used for subsequent operations that are specific to a subscription.
.PARAMETER ProvisionerApplicationId
The AAD Application ID used to provision test resources when a provisioner is
Expand All @@ -614,17 +662,14 @@ If none is specified New-TestResources.ps1 uses the TestApplicationSecret.
This value is not passed to the ARM template.
.PARAMETER DeleteAfterHours
Optional. Positive integer number of hours from the current time to set the
Positive integer number of hours from the current time to set the
'DeleteAfter' tag on the created resource group. The computed value is a
timestamp of the form "2020-03-04T09:07:04.3083910Z".
If this value is not specified no 'DeleteAfter' tag will be assigned to the
created resource group.
An optional cleanup process can delete resource groups whose "DeleteAfter"
timestamp is less than the current time.
This isused for CI automation.
This is used for CI automation.
.PARAMETER Location
Optional location where resources should be created. If left empty, the default
Expand Down Expand Up @@ -660,8 +705,8 @@ Save test environment settings into a test-resources.json.env file next to test-
The environment file would be scoped to the current repository directory.
.EXAMPLE
Connect-AzAccount -Subscription "REPLACE_WITH_SUBSCRIPTION_ID"
New-TestResources.ps1 -ServiceDirectory 'keyvault'
Connect-AzAccount -Subscription 'REPLACE_WITH_SUBSCRIPTION_ID'
New-TestResources.ps1 keyvault
Run this in a desktop environment to create new AAD apps and Service Principals
that can be used to provision resources and run live tests.
Expand Down
28 changes: 14 additions & 14 deletions eng/common/TestResources/New-TestResources.ps1.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@ Deploys live test resources defined for a service directory to Azure.
```
New-TestResources.ps1 [-BaseName <String>] [-ResourceGroupName <String>] [-ServiceDirectory] <String>
[-TestApplicationId <String>] [-TestApplicationSecret <String>] [-TestApplicationOid <String>]
[-DeleteAfterHours <Int32>] [-Location <String>] [-Environment <String>] [-ArmTemplateParameters <Hashtable>]
[-AdditionalParameters <Hashtable>] [-EnvironmentVariables <Hashtable>] [-CI] [-Force] [-OutFile] [-WhatIf]
[-Confirm] [<CommonParameters>]
[-SubscriptionId <String>] [-DeleteAfterHours <Int32>] [-Location <String>] [-Environment <String>]
[-ArmTemplateParameters <Hashtable>] [-AdditionalParameters <Hashtable>] [-EnvironmentVariables <Hashtable>]
[-CI] [-Force] [-OutFile] [-WhatIf] [-Confirm] [<CommonParameters>]
```

### Provisioner
Expand Down Expand Up @@ -55,8 +55,8 @@ specified in $ProvisionerApplicationId and $ProvisionerApplicationSecret.

### EXAMPLE 1
```
Connect-AzAccount -Subscription "REPLACE_WITH_SUBSCRIPTION_ID"
New-TestResources.ps1 -ServiceDirectory 'keyvault'
Connect-AzAccount -Subscription 'REPLACE_WITH_SUBSCRIPTION_ID'
New-TestResources.ps1 keyvault
```

Run this in a desktop environment to create new AAD apps and Service Principals
Expand Down Expand Up @@ -231,16 +231,20 @@ Optional subscription ID to use for new resources when logging in as a
provisioner.
You can also use Set-AzContext if not provisioning.
The default is the Azure SDK Developer Playground subscription ID.
If you do not specify a SubscriptionId and are not logged in, one will be
automatically selected for you by the Connect-AzAccount cmdlet.
Once you are logged in (or were previously), the selected SubscriptionId
will be used for subsequent operations that are specific to a subscription.
```yaml
Type: String
Parameter Sets: Provisioner
Parameter Sets: (All)
Aliases:

Required: False
Position: Named
Default value: faa080af-c1d8-40ad-9cce-e1a450ca5b57
Default value: None
Accept pipeline input: False
Accept wildcard characters: False
```
Expand Down Expand Up @@ -286,19 +290,15 @@ Accept wildcard characters: False
```
### -DeleteAfterHours
Optional.
Positive integer number of hours from the current time to set the
'DeleteAfter' tag on the created resource group.
The computed value is a
timestamp of the form "2020-03-04T09:07:04.3083910Z".
If this value is not specified no 'DeleteAfter' tag will be assigned to the
created resource group.
An optional cleanup process can delete resource groups whose "DeleteAfter"
timestamp is less than the current time.
This isused for CI automation.
This is used for CI automation.
```yaml
Type: Int32
Expand All @@ -307,7 +307,7 @@ Aliases:

Required: False
Position: Named
Default value: 0
Default value: 48
Accept pipeline input: False
Accept wildcard characters: False
```
Expand Down
Loading

0 comments on commit 2acd938

Please sign in to comment.