aws_s3_deployment.BucketDeployment: deploying a simple zip-file is not possible

[gratinierer]

Describe the bug

Try to push a simple zip-file to S3

        aws_s3_deployment.BucketDeployment(self,
                                           'CodebuildBucketDeployment',
                                           sources=[aws_s3_deployment.Source.asset(
                                               'aZipFile.zip')],
                                           extract=True,
                                           destination_bucket=deployment_bucket
                                           )

Expected Behavior

deployment_bucket contains one file named aZipFile.zip after deployment

Current Behavior

the content gets extracted during deployment, so you don't get a zip-file but the extracted content.
setting extract to False does also not have the desired effect, because the you get a zip file but with a random-uuid-name and not aZipFile.zip.

Reproduction Steps

see above

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.148.0 (build e5740c0)

Framework Version

No response

Node.js Version

20.15

OS

Linux

Language

Python

Language Version

No response

Other information

No response

[ashishdhingra]

@gratinierer Good morning. Thanks for opening the issue. If your scenario is to just upload simple .zip to S3 bucket, then aws-cdk-lib.aws_s3_deployment.BucketDeployment might not be the right approach. If you refer aws-cdk-lib.aws_s3_deployment documentation:

• This library allows populating an S3 bucket with the contents of .zip files from other S3 buckets or from local disk. Under the hood:

  • The custom resource invokes its associated Lambda function, which downloads the .zip archive, extracts it and issues aws s3 sync --delete against the destination bucket (in this case websiteBucket). If there is more than one source, the sources will be downloaded and merged pre-deployment at this step.

• By default, files are zipped, then extracted into the destination bucket. Per Keep Files Zipped, as you also noted, you can use the option extract: false to disable this behavior, in which case, files will remain in a zip file when deployed to S3. To reference the object keys, or filenames, which will be deployed to the bucket, you can use the objectKeys getter on the bucket deployment..

In your case, you could use object_keys on the BucketDeployment. You could use destination_key_prefix to add a key prefix, but that would not preserve the file name. Kindly note that there are allowed character limitations for S3 object keys. As a workaround, you could use a custom resource that makes AWS JavaScript SDK call to invoke CopyObject S3 API operation on the object key accessed via object_keys property getter.

Kindly note that CDK is not an AWS SDK (which allows to upload and invoke S3 operations). It allows you to define infrastructure as a code and allows to deploy the stack via CloudFormation.

Thanks,
Ashish

[gratinierer]

Hi @ashishdhingra,
Thanks for your detailed comment.

By default, files are zipped, then extracted into the destination bucket.

That‘s the point.
This does not happen in my case.
I think, cdk detects, that the file already is a zip and omits the zipping but triggers the unzipping.

i found a simple, but not beautiful workaround: i zip my zip file a second time, than do the s3 deployment. In that case i get my zip-file with the correct name to the bucket.

but i still think, that this woraround should no be needed, if the cdk would always follow the same flow, no matter if you pass a zip file or another payload.

[ashishdhingra]

Hi @ashishdhingra, Thanks for your detailed comment.

By default, files are zipped, then extracted into the destination bucket.

That‘s the point. This does not happen in my case. I think, cdk detects, that the file already is a zip and omits the zipping but triggers the unzipping.

i found a simple, but not beautiful workaround: i zip my zip file a second time, than do the s3 deployment. In that case i get my zip-file with the correct name to the bucket.

but i still think, that this woraround should no be needed, if the cdk would always follow the same flow, no matter if you pass a zip file or another payload.

@gratinierer The default behavior to unzip the .zip files is the designed behavior of S3 Bucket Deployment. Kindly note that it accepts sources as a list of assets that need to be deployed. This is used in scenarios where the requirement is to host static contents (may be static website) in an S3 bucket (refer Hosting a static website using Amazon S3). Your use case is trying to use this module to upload file to an S3 bucket which would normally be done using AWS SDK(s).

Upon checking documentation again, I noticed that the DeployTimeSubstitutedFile construct provides the same custom resource workaround as I proposed in #30971 (comment). The following code will upload the named zip file with the specified destination key in an S3 bucket:

new s3deploy.DeployTimeSubstitutedFile(this, 'MyZipFile', {
      source: path.join(__dirname, '../testbucket-someissue-deployment.zip'),
      destinationKey: 'testbucket-someissue-deployment.zip',
      destinationBucket: s3.Bucket.fromBucketName(this, 'testbucket', 'testbucket-someissue'),
      substitutions: {}
    });

I tested it locally and notice the file named testbucket-someissue-deployment.zip in the specified S3 bucket after CDK deployment.

Thanks,
Ashish

[github-actions]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

[gratinierer]

Hi @ashishdhingra ,
thanks for your hint.
Have you tested this with a binary file like zip?
It seems that there is some processing which changes the file.
In my case the uploaded file is corrupt after a push via DeployTimeSubstitutedFile.
can you check that on your side?

[ashishdhingra]

DeployTimeSubstitutedFile

@gratinierer Good morning. I think you are correct. Looks like DeployTimeSubstitutedFile is only suitable for YAML files to upload these and perform some substitutions, resulting in corruption when used with .zip files. We cannot use S3 assets since it would use CDK bootstrap bucket to upload the asset.

The possible workaround in this case is to use custom resource along with extract flag as false to rename the ZIP file per your requirement after BucketDeployment finishes.