Add support for multiple secrets to Pulumi shell plugin

[ringods]

Overview

Add support for multiple secrets in the Pulumi shell plugin.

Type of change

• Created a new plugin
• Improved an existing plugin
• Fixed a bug in an existing plugin
• Improved contributor utilities or experience

Related Issue(s)

• Relates: Allow specifying multiple backends for a shell plugin #192

How To Test

• Activate multiple plugins.
• Create a Pulumi project and create resources from a provider matching one of the shell plugins, e.g. Github.
• Run pulumi up
• Verify that:
  • the Pulumi access token was injected
  • the Github token was injected

Changelog

[AndyTitu]

If this Personal Access Token is for authenticating to Pulumi Cloud, instead of deleting it from the usages, you could consider:

1. creating a separate CredentialUsage in this usages slice, with the Description: Credentials for authenticating to Pulumi Cloud, and using the Name: credname.PersonalAccessToken, pattern you used before, and NeedsAuth: needsauth.ForCommand("login")

2. Creating a second plugin for better isolation between definitions called PulumiCloud or something similar, and that can define its own pulumi executable similar to the one from the other plugin, and all the rules of defining the credential usage from 1 still apply

[ringods]

@AndyTitu

needsauth.ForCommand("login") would inject the credential from 1Pwd into the shell command, but this shell command writes the credential to a file on disk, which defeats the purpose of using the shell plugin IMO.

I could use some help creating the proper schema to support the following scenarios:

1. always injecting secrets for regular Pulumi CLI commands which require credentials: new, preview, up, destroy, ...
2. inject PULUMI_BACKEND_URL when a custom state backend was configured. This can be a cloud object storage URL (s3, azure blob storage, ...) or the self-hosted variant of our Pulumi Cloud (business critical customers)
3. inject PULUMI_ACCESS_TOKEN when Pulumi Cloud (or self-hosted, see previous) is used.
4. inject any other credentials in the setup, depending the op plugin configuration for the current folder

Could you put me on the right track?

[AndyTitu]

I'm first going to go through a few things you mentioned, to make sure I'm getting everything right. We want to:

1. provision 3rd party CLI credentials whenever new, preview, up, destroy, etc are used.
2. provision either of the PULUMI_ACCESS_TOKEN or PULUMI_BACKEND_URL based on what state backend is being used. Sometimes both can be provisioned, other times only one of them can be provisioned.
3. not do anything when login is used

If all of the above are true then I'm proposing creating 2 credential usages:

Uses: []schema.CredentialUsage{
			{
				Description: "State backend credentials",
				Name:        credname.PersonalAccessToken,
				NeedsAuth: needsauth.IfAny(
					needsauth.ForCommand("new"),
					needsauth.ForCommand("preview"),
					needsauth.ForCommand("up"),
				),
				Optional: true,
			},
			{
				Description: "Credentials to provision for 3rd party CLIs",
				SelectFrom: &schema.CredentialSelection{
					ID:                    "project",
					IncludeAllCredentials: true,
					AllowMultiple:         true,
				},
				NeedsAuth: needsauth.IfAny(
					needsauth.ForCommand("new"),
					needsauth.ForCommand("preview"),
					needsauth.ForCommand("up"),
				),
				Optional: true,
			},
		},

Let me know if you have any questions about the specific chosen values on the specified fields of the credential usages above.

Another thing that I'm proposing is to make both fields in PersonalAccessToken optional so that this requirement can be satisfied:

Sometimes both can be provisioned, other times only one of them can be provisioned.

in the context of PULUMI_BACKEND_URL and PULUMI_ACCESS_TOKEN

[ringods]

@AndyTitu about this recommendation:

Another thing that I'm proposing is to make both fields in PersonalAccessToken optional so that this requirement can be satisfied:

Sometimes both can be provisioned, other times only one of them can be provisioned.

in the context of PULUMI_BACKEND_URL and PULUMI_ACCESS_TOKEN

My latest code has the token and backend URL split up in separate CredentialUsage schema definitions. See the latest code in this PR. Is this the setup you intended?

Also, with multiple CredentialUsage schema definitions, can you advice how to write the Go unit tests? They are failing now, I tried to update these but I don't get them to work.

[AndyTitu]

Yes, that is the intended setup. But I've seen the code advanced a bit from that state. As mentioned on developer Slack, given the current limitations, it could be worth it to first focus on either making it run correctly with the schema while ignoring the UX limitations when importing or focus on what Pulumi users are mostly using (between a PAT without a backend endpoint url, or vice-versa, or both at the same time) and optimise for that workflow.

After that it's done, we can look into any failing tests.

What do you think @ringods ?

[ringods]

Closing PR due to pulumi/pulumi#13919