Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specify properties of the Secret as part of Database instance - Description, ExcludeCharacters #4144

Closed
2 tasks
rebolyte opened this issue Sep 18, 2019 · 13 comments · Fixed by #10583
Closed
2 tasks
Assignees
Labels
@aws-cdk/aws-rds Related to Amazon Relational Database effort/small Small work item – less than a day of effort feature-request A feature should be added or improved. in-progress This issue is being actively worked on. p1
Milestone

Comments

@rebolyte
Copy link

rebolyte commented Sep 18, 2019

When doing new rds.DatabaseInstance(this, 'AppDb', { ... }), a secret is automatically generated with the database's connection info. This is super useful, but not quite sufficient for my use case. I'd like to be able to specify the secret properties when creating the RDS instance.

Use Case

I need to specify some other characters to exclude from the generated password (specifically ;, since I'll be assembling a semicolon-delimited connection string like Host=abc.com;Port=5432;Username=admin;Password=p@ssw0rd;).

Proposed Solution

Add a secretProperties property to the RDS DatabaseInstanceProps that mixes in any specified options with the defaults when creating the DB secret, e.g.:

const appDbInstance = new DatabaseInstance(this, 'AppDb', {
	databaseName: 'AppDb',
	engine: DatabaseInstanceEngine.POSTGRES,
	engineVersion: '10.9',
	secretProperties: {
		excludeCharacters: '"@/\\;'
	}
});

Other

I would override this manually, but trying this only adds a property to the SecretTargetAttachment not the Secret itself:

const cfnAppSecret = appDbInstance.secret!.node.defaultChild as CfnSecretTargetAttachment;

cfnAppSecret.addOverride('Properties.GenerateSecretString.ExcludeCharacters', '"@/\\;');
  • 👋 I may be able to implement this feature request
  • ⚠️ This feature might incur a breaking change

This is a 🚀 Feature Request

@rebolyte rebolyte added feature-request A feature should be added or improved. needs-triage This issue or PR still needs to be triaged. labels Sep 18, 2019
@SomayaB SomayaB added the @aws-cdk/aws-rds Related to Amazon Relational Database label Sep 18, 2019
@jogold
Copy link
Contributor

jogold commented Sep 20, 2019

You can do this as a workaround:

const dbSecret = appDbInstance.node.tryFindChild('Secret') as rds.DatabaseSecret;
const cfnSecret = dbSecret.node.defaultChild as secretsmanager.CfnSecret;
cfnSecret.addPropertyOverride('GenerateSecretString.ExcludeCharacters', '"@/\\;');

which will target the AWS::SecretsManager::Secret.

@NGL321 NGL321 removed the needs-triage This issue or PR still needs to be triaged. label Oct 4, 2019
@NGL321
Copy link
Contributor

NGL321 commented Oct 4, 2019

@rebolyte,

Thanks for submitting this request! It seems like a reasonable feature, however it may take some time to address since it is a fairly specific use-case (if other community members disagree, please leave a comment here!).

If you want to see it implemented faster, the best way may be to submit a PR, and we will make sure it gets reviewed!

@skinny85 skinny85 assigned nija-at and unassigned skinny85 Feb 6, 2020
@nija-at nija-at added the effort/small Small work item – less than a day of effort label Feb 17, 2020
@nija-at nija-at changed the title Ability to specify ExcludeCharacters for RDS Secret Specify properties of the Secret when creating Database instance - Description, ExcludeCharacters Feb 17, 2020
@nija-at nija-at changed the title Specify properties of the Secret when creating Database instance - Description, ExcludeCharacters Specify properties of the Secret as part of Database instance - Description, ExcludeCharacters Feb 17, 2020
@thibaut-pro
Copy link

if other community members disagree, please leave a comment here!

We are impacted by this. The backtick ` character in a database password breaks our django configuration.

@nija-at
Copy link
Contributor

nija-at commented Mar 2, 2020

Note that you can workaround this limitation by creating your own Secret and using the secretValueFromJson() API to pass in the masterPassword.

@Spacerat
Copy link
Contributor

Spacerat commented Mar 4, 2020

We also ran into this, since DMS is not compatible with any of ;+%, and arrived at @nija-at 's workaround. I'd volunteer to write a PR for this if someone can confirm an allowable approach. I'd lean towards just allowing users to pass a custom secret via DatabaseInstanceProps.

@ahammond
Copy link
Contributor

I tried the following and am still seeing secrets which break DMS when I rotate:

    const masterSecret = this.cluster.addRotationSingleUser();
    new cdk.CfnOutput(this, 'TheMasterSecret', { value: masterSecret.toString() });

    // Workaround to avoid DMS breaking passwords
    // https://github.com/aws/aws-cdk/issues/4144#issuecomment-533500069
    const dbSecret = this.cluster.node.tryFindChild('Secret') as rds.DatabaseSecret;
    const cfnSecret = dbSecret.node.defaultChild as secretsmanager.CfnSecret;
    cfnSecret.addPropertyOverride('GenerateSecretString.ExcludeCharacters', '"@/\\;+%');

I destroyed and redeployed the entire stack. Then I clicked Rotate secret immediately a few times to check the secrets it was generating. I got a secret with % in it after a couple of tries.

What do I need to do to ensure DMS compatible secrets?

@ahammond
Copy link
Contributor

I see that aws-samples/aws-secrets-manager-rotation-lambdas#32 now supports configurable EXCLUDE_CHARACTERS via an env-var of the same name.

@ahammond
Copy link
Contributor

It looks like while the lambda is updated, the SAM isn't yet: https://console.aws.amazon.com/lambda/home?region=us-east-1#/create/app?applicationId=arn:aws:serverlessrepo:us-east-1:297356227824:applications/SecretsManagerRDSMySQLRotationSingleUser doesn't have excludedCharacters in the Application settings. I guess next step is finding that code and adding a PR.

@nija-at nija-at assigned skinny85 and unassigned nija-at Jul 14, 2020
@playphil
Copy link

playphil commented Sep 1, 2020

I noticed excludeCharacters now appears in the AWS console within SecretsManagerRDSMySQLRotationSingleUser — version 1.1.60 for region us-east-1 https://console.aws.amazon.com/lambda/home?region=us-east-1#/create/app?applicationId=arn:aws:serverlessrepo:us-east-1:297356227824:applications/SecretsManagerRDSMySQLRotationSingleUser

@skinny85
Copy link
Contributor

@ahammond now that #10110 has been merged, can we close this issue?

@skinny85 skinny85 added the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. label Sep 11, 2020
@ahammond
Copy link
Contributor

@skinny85 I think the next and last step involves adding some plumbing to the Aurora module. I'll take a stab at it tomorrow.

@skinny85 skinny85 added in-progress This issue is being actively worked on. and removed response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. labels Sep 11, 2020
@skinny85 skinny85 added the p1 label Sep 11, 2020
@skinny85 skinny85 added this to the [GA] RDS milestone Sep 22, 2020
skinny85 pushed a commit to skinny85/aws-cdk that referenced this issue Sep 29, 2020
…eSecret

Change the default excludeCharacters for Cluster,
Instance and DatabaseSecret to the character set " %+~`#$&*()|[]{}:;<>?!'/@\"\\",
as the previous set ('"@/\\')
had a tendency to generate problematic passwords that wouldn't work in the shell,
or with services like DMS.
Do the same for single- and multi-user rotations in Cluster and Instance as well.
Also allow passing a custom excludeCharacters for Credentials and SnapshotCredentials,
and also in addSingleUserRotation and addMultiUserRotation.

Fixes aws#4144

BREAKING CHANGE: the default excludeCharacters set for Instance, Cluster and DatabaseSecret is now " %+~`#$&*()|[]{}:;<>?!'/@\"\\"
* **rds**: the default excludeCharacters for addSingleUserRotation and addMultiUserRotation is now " %+~`#$&*()|[]{}:;<>?!'/@\"\\"
* **rds**: Instance.addSingleUserRotation now takes options as the first argument, instead of just Duration
* **rds**: Cluster.addSingleUserRotation now takes options as the first argument, instead of just Duration
skinny85 pushed a commit to skinny85/aws-cdk that referenced this issue Sep 29, 2020
…eSecret

Change the default excludeCharacters for Cluster,
Instance and DatabaseSecret to the character set " %+~`#$&*()|[]{}:;<>?!'/@\"\\",
as the previous set ('"@/\\')
had a tendency to generate problematic passwords that wouldn't work in the shell,
or with services like DMS.
Do the same for single- and multi-user rotations in Cluster and Instance as well.
Also allow passing a custom excludeCharacters for Credentials and SnapshotCredentials,
and also in addSingleUserRotation and addMultiUserRotation.

Fixes aws#4144

BREAKING CHANGE: the default excludeCharacters set for Instance, Cluster and DatabaseSecret is now " %+~`#$&*()|[]{}:;<>?!'/@\"\\"
* **rds**: the default excludeCharacters for addSingleUserRotation and addMultiUserRotation is now " %+~`#$&*()|[]{}:;<>?!'/@\"\\"
* **rds**: Instance.addSingleUserRotation now takes options as the first argument, instead of just Duration
* **rds**: Cluster.addSingleUserRotation now takes options as the first argument, instead of just Duration
skinny85 pushed a commit to skinny85/aws-cdk that referenced this issue Sep 29, 2020
…eSecret

Change the default excludeCharacters for Cluster,
Instance and DatabaseSecret to the character set " %+~`#$&*()|[]{}:;<>?!'/@\"\\",
as the previous set ('"@/\\')
had a tendency to generate problematic passwords that wouldn't work in the shell,
or with services like DMS.
Do the same for single- and multi-user rotations in Cluster and Instance as well.
Also allow passing a custom excludeCharacters for Credentials and SnapshotCredentials,
and also in addSingleUserRotation and addMultiUserRotation.

Fixes aws#4144

BREAKING CHANGE: the default excludeCharacters set for Instance, Cluster and DatabaseSecret is now " %+~`#$&*()|[]{}:;<>?!'/@\"\\"
* **rds**: the default excludeCharacters for addSingleUserRotation and addMultiUserRotation is now " %+~`#$&*()|[]{}:;<>?!'/@\"\\"
* **rds**: Instance.addSingleUserRotation now takes options as the first argument, instead of just Duration
* **rds**: Cluster.addSingleUserRotation now takes options as the first argument, instead of just Duration
skinny85 pushed a commit to skinny85/aws-cdk that referenced this issue Sep 29, 2020
…eSecret

Change the default excludeCharacters for Cluster,
Instance and DatabaseSecret to the character set " %+~`#$&*()|[]{}:;<>?!'/@\"\\",
as the previous set ('"@/\\')
had a tendency to generate problematic passwords that wouldn't work in the shell,
or with services like DMS.
Do the same for single- and multi-user rotations in Cluster and Instance as well.
Also allow passing a custom excludeCharacters for Credentials and SnapshotCredentials,
and also in addSingleUserRotation and addMultiUserRotation.

Fixes aws#4144

BREAKING CHANGE: the default excludeCharacters set for Instance, Cluster and DatabaseSecret is now " %+~`#$&*()|[]{}:;<>?!'/@\"\\"
* **rds**: the default excludeCharacters for addSingleUserRotation and addMultiUserRotation is now " %+~`#$&*()|[]{}:;<>?!'/@\"\\"
* **rds**: Instance.addSingleUserRotation now takes options as the first argument, instead of just Duration
* **rds**: Cluster.addSingleUserRotation now takes options as the first argument, instead of just Duration
skinny85 added a commit that referenced this issue Sep 29, 2020
…words for Cluster, Instance, DatabaseSecret

Change the default excludeCharacters for Cluster,
Instance and DatabaseSecret to the character set ``" %+~`#$&*()|[]{}:;<>?!'/@\"\\"``,
as the previous set (`'"@/\\'`)
had a tendency to generate problematic passwords that wouldn't work in the shell,
or with services like DMS.
Do the same for single- and multi-user rotations in Cluster and Instance as well.
Also allow passing a custom excludeCharacters for Credentials and SnapshotCredentials,
and also in addSingleUserRotation and addMultiUserRotation.

Fixes #4144

BREAKING CHANGE: the default generated password exclude characters set for Instance, Cluster and `DatabaseSecret` is now ``" %+~`#$&*()|[]{}:;<>?!'/@\"\\"``
* **rds**: the default generated password exclude characters for `addSingleUserRotation()` and `addMultiUserRotation()` in Cluster and Instance is now ``" %+~`#$&*()|[]{}:;<>?!'/@\"\\"``
* **rds**: `Instance.addSingleUserRotation()` now takes options object as the first argument, instead of just `Duration`
* **rds**: `Cluster.addSingleUserRotation()` now takes options object as the first argument, instead of just `Duration`
* **rds**: `SnapshotCredentials.fromGeneratedPassword()` now takes an option object as the second argument, instead of just `IKey`

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@mmeylan
Copy link

mmeylan commented Oct 27, 2020

@skinny85 anything to do to migrate existing DatabaseSecret ? It seems since upgrading to 'aws-cdk@1.65' I can't connect to my DB using the master username & password generated with DatabaseSecret

@skinny85
Copy link
Contributor

@mmeylan is this the same as #10800?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/aws-rds Related to Amazon Relational Database effort/small Small work item – less than a day of effort feature-request A feature should be added or improved. in-progress This issue is being actively worked on. p1
Projects
None yet