pipelines: BundlingOptions Docker Login

[argarinpauljohn]

Describe the bug

We are using CodePipeline in which we correctly specified our Docker Hub secret. It is correctly retrieved by the codebuild project. In this pipeline, we have a stage in which we are building a stack that has an aws_s3_assets.s3_assets resource. In this resource, we use `BundlingOptions' like this:

bundling=BundlingOptions(
                image=DockerImage.from_registry("python:3.10"),
                command=[
                    "sh",
                    "-c",
                    """
                    pip install poetry;
                ],
                user="root",
            ),

However, pulling the image python:3.10 still results in a rate-limit issue. It seems like it's not using the specified Docker Hub secret we specified in the pipeline level. I verified using some command that we still haven't reached our account's rate limit.

Our pipeline looks like this:

        pipeline = CodePipeline(
            self,
            "DataPipeline",
            pipeline_name="DataPipeline",
            docker_credentials=[DockerCredential.docker_hub(cdk.aws_secretsmanager.Secret.from_secret_name_v2(scope, "DockerHubSecret", secret_name="DockerHubSecret"))],
            synth=ShellStep(
                "Synth",
                input=github_source,
                commands=[
                    "pip install poetry",
                    "cd cdk",
                    "poetry install --no-dev",
                ],
                primary_output_directory="./cdk/cdk.out",
            ),
            code_build_defaults=CodeBuildOptions(role_policy=self._get_read_only_access_statements()),
            docker_enabled_for_synth=True,
        )

to which we add the stage that creates the s3_assets.

Expected Behavior

Use the specified Docker Hub secrets then login to avoid the rate limit issue.

Current Behavior

Unable to find image 'python:3.10' locally
266 | docker: Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit.

Reproduction Steps

N/A

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.81.0

Framework Version

No response

Node.js Version

v16.17.0

OS

macOS Ventura 13.3.1

Language

Python

Language Version

No response

Other information

No response

[pahud]

Hi

If you need python:3.10 you should consider pulling from ECR public to avoid throttling.

You can select your favorite image tags here:
https://gallery.ecr.aws/docker/library/python

Let me know if it works for you.

[github-actions]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

[hoegertn]

I think this is a broader problem. assets that are bundled in the synth step (S3, Lambda Containers) do not honor the Docker credentials. This is also true for ECR logins to private registries.

[argarinpauljohn]

Hi

If you need python:3.10 you should consider pulling from ECR public to avoid throttling.

You can select your favorite image tags here: https://gallery.ecr.aws/docker/library/python

Let me know if it works for you.

Thanks @pahud! Will git it a try!

[argarinpauljohn]

I think this is a broader problem. assets that are bundled in the synth step (S3, Lambda Containers) do not honor the Docker credentials. This is also true for ECR logins to private registries.

+1!

[pahud]

I think this is a broader problem. assets that are bundled in the synth step (S3, Lambda Containers) do not honor the Docker credentials. This is also true for ECR logins to private registries.

Agree. I am making it a p2 feature request. Please help us prioritize it by upvotes 👍 on the issue.