Skip to content
This repository has been archived by the owner on Feb 16, 2023. It is now read-only.

Map all secrets from directory to environment variables #286

Closed
jpcoenen opened this issue May 11, 2020 · 8 comments
Closed

Map all secrets from directory to environment variables #286

jpcoenen opened this issue May 11, 2020 · 8 comments
Labels
enhancement New feature or request

Comments

@jpcoenen
Copy link
Member

Currently, a secrethub.env file has to be created to easily map multiple secrets to environment variables when using secrethub run.

However, in many cases the names of secrets already describe the names of the environment variables they should be mounted to. By adding the --secrets-dir path/to/dir flag, you could easily map load all secrets to the environment.

The mapping could follow a default set of rules, e.g. replace dirs (/) with _ etc. This could drastically decrease the setup time needed, as no templates need to be written.

Example

This tree:

org/repo/my-app/
    aws/
        access_key_id
        secret_access_key
    db/
        username
        password

Would map to:

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
DB_USERNAME
DB_PASSWORD

by running:

secrethub run --secrets-dir=org/repo/my-app/ -- node app.js
@jpcoenen jpcoenen added the enhancement New feature or request label May 11, 2020
@mgrlabs
Copy link

mgrlabs commented May 12, 2020

Hi there,

How does this work if you were to reference org/repo/my-app/aws? Would you get?

ACCESS_KEY_ID
SECRET_ACCESS_KEY

Regards,
Mark.

@jpcoenen
Copy link
Member Author

Hi Mark,

My current proposal would indeed be that using --secrets-dir=org/repo/my-app/aws results in:

ACCESS_KEY_ID
SECRET_ACCESS_KEY

Is that what you'd expect?

@mgrlabs
Copy link

mgrlabs commented May 12, 2020

That's a pretty cool idea to map subordinate directories as he prefix for the environment variable. The example I would give is where I would store secrets required for Terraform to deploy into Azure in org/product/dev/arm containing:

client_id
client_secret
tentant_id
subscription_id

would map to:

ARM_CLIENT_ID
ARM_CLIENT_SECRET
ARM_TENANT_ID
ARM_SUBSCRIPTION_ID

This would work well in that case. Gives us flexibility. Looks good :)

Regards,
Mark.

@stavalfi
Copy link

stavalfi commented Jul 4, 2020

  • I'm assuming that this feature will let us avoid writing a secrethub.env file (or removing some content from it). if yes, keep reading. if not, I'm wrong and just ignore this response).

  • I'm assuming that the community is allowed to have an opinion about this issue so I will write my own. if not, ignore it.


when a project has a secrethub.env file, there are advantages of visibility and explicitness (maybe its the same) about which env-var are used. it's a single-source-of-truth for the secrets that are being used in the project.

when developing in a project when the project made a use of this feature, I can't know which env-var that is being used, is a secret or not.

a possible response can be:

  • a developer shouldn't be care if a env-var is a secret or not.
    • I'm not sure that I have much expirience to answer this but my intoition tells me that the developer should know if a env-var is a secret or not.
    • you guys can argue that the developer can list the secrets of a project in secrethub-cli. but what if he doesn't have a read permission in this repo? maybe he doesn't have have an account?
  • a project should have an abstraction over secrethub: a function that returns all secrets as variables instead of making use of env-var directly in the code. so the issue that I mentioned above, is not a problem.

What do you guys think?

@SimonBarendse
Copy link
Member

Hi @stavalfi , yes, we welcome any input from the community!

This feature is indeed meant to be used instead of a secrethub.env file.

Note however, that we will still be supporting secrethub.env files and all other methods already available to pass environment variables (--envar flag, secret references, environment files and environment variables passed by the OS).

So, if you prefer the secrethub.env files, feel free to keep using those!

As you also noted, this --secrets-dir flag doesn't require explicit definition of the environment variables in the secrethub.env file or any other place.
This can be seen both as an advantage (Your directory in SecretHub is already the source of truth of all secrets in your project, so using that to determine which environment variables need to be populated saves time of maintaining a secrethub.env file that just echos that state) or as a disadvantage (you no longer have a secrethub.env file in source control which exactly defines which secrets you'll be using, making your Git repo the source of truth and always having a commit for any changes to your environment). I believe it largely depends on whether you want your directory in SecretHub or your Git repo to be the source of truth about which secrets are being used.

I imagine the preference and trade-offs between both methods to vary between projects and processes, which is why we aim to support both and leave it up to you to pick the solution that best fits your use.


You mentioned using both secrets and non-secrets in the environment. We've seen many users store both of these in SecretHub. You can also store values in SecretHub that are not secret. We might add features later which make it possible to differentiate between these two types of values stored in SecretHub (e.g. no masking, other access levels etc.) but this is not something we've currently planned. However, please use the current set of features as you see fit, including storing other values than secrets.


You mentioned a user without read permissions not being able to see what secrets will be used when the --secrets-dir flag is used. I can imagine adding a separate permission later (named e.g. list), which allows you to read the metadata (e.g. names of the secrets), but doesn't allow you to read the secret data. However, this is not on the roadmap yet.


a project should have an abstraction over secrethub: a function that returns all secrets as variables instead of making use of env-var directly in the code. so the issue that I mentioned above, is not a problem.

I'm not sure I quite understand what you mean. Do you mean to call SecretHub directly from your code? Could you please elaborate on this a bit? Maybe an example can help?

@stavalfi
Copy link

stavalfi commented Jul 6, 2020

the roadmap sounds very cool! thanks for the great answer :)

@SimonBarendse
Copy link
Member

This has been merged in #299 and will be included in the next release

@SimonBarendse
Copy link
Member

This has been released in v0.41.0

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants