AWS CI/CD with CodeCommit, CodeBuiild, and CodePipeline

This module utilizes tf-aws-cicd-codebuild and tf-aws-cicd-codepipeline to deliver either a same account or cross-account CI/CD pipeline. Currently, it works well with the external tf-aws-cicd-codecommit module for SCM repositories.

Prerequisites

1. A VPC, Subnets, and Security group of the pipeline to run in.
2. IAM user or role ARNs of the admins and owner of the KMS key. The KMS key will be generated in the pipeline account.
3. Deploy CodeCommit repo in a shared account or the same account where the pipeline will run using tf-aws-cicd-codecommit.
   • Push code to the repo so that at least the primary main branch gets created.
   • Save the Canonical ID of the CodeCommit Repository account if it's in a different account than the pipeline.
     • aws s3api list-buckets --query Owner.ID --output text
   • Save the Account ID of the CodeCommit Repository account if it's in a different account than the pipeline.
     • aws sts get-caller-identity --query Account --output text
   • Save the IAM Role ARN generated by the deployment of the CodeCommit module.
4. A YAML script in your deployment directory (i.e., ./scripts/plan.yml). It needs to be a path that this module can access.
5. Deploy this CI/CD module in the account where the pipeline should run.
6. Update CodeCommit Repo var.trusts[] with S3 bucket ARN, CodePipeline ARN, and KMS Key ARN outputs from this module.
   • NOTE: The first automatic execution (immediately after deployment) will fail because these trusts are not in place.

Usage

Examples of how to use this module.

Example build YAML

This script should go inyour deployment directory or a subdirdctory under it. The full path will need to be passed to this module as var.stages[x].resource.script.

version: 0.2
env:
  shell: bash
phases:
  build:
    commands:
      - echo "test:terraform init"
      - echo "test:terraform plan"

Cross-Account Terraform

This example creates a CodePipeline that pulls code from a CodeCommit repository in another account located in the same AWS Organization called the shared account.

#
# Create a CodeCommit repository in the SHARED account using tf-aws-cicd-codecommit prior to deployment.
#
module "codecommit" {
  source    = "https://github.com/clearscale/tf-aws-cicd-codecommit.git?ref=v1.0.0"

  account = {
    id = "*", name = local.account.name, provider = "aws", key = "current", region = local.region.name
  }


  prefix  = local.context.prefix
  client  = local.context.client
  project = local.context.project
  env     = local.account.name
  region  = local.region.name
  name    = "codecommit"

  repo = {
    name   = "test-shared-account"
    create = true
  }

  #
  # To be filled in and redeployed after the first successful deployment. These resources do not exist prior to module.cicd being deployed.
  # trusts = [
  #  "ARN_S3_BUCKET,
  #  "ARN_KMS_KEY", # Must be the key ARN - not an alias.
  #  "ARN_CODEPIPELINE"
  # ]
  #
}

module "cicd" {
  source = "https://github.com/clearscale/tf-aws-cicd.git?ref=v1.0.0"

  account = {
    id = "*", name = local.account.name, provider = "aws", key = "current", region = local.region.name
  }

  prefix  = local.context.prefix
  client  = local.context.client
  project = local.context.project
  env     = local.account.name
  region  = local.region.name
  name    = "cicd"


  # Optional
  iam_service_role_policies = [
    "arn:aws:iam::aws:policy/PowerUserAccess"
  ]

  vpc = {
    id              = "VPC_ID"
    subnets         = ["SUBNET_ID_1", "SUBNET_ID_2"]
    security_groups = ["SG_ID_1", "SG_ID_2"],
  }

  repo = {
    name     = "test-shared-account"
    branch   = "main"
    role_arn = "arn:aws:iam::123456789012:role/CsTffwkcs.Shared.USW1.CodeCommit.TestSharedAccount"
    account = {
      name         = "shared"
      id           = "123456789012"                                                            # Account ID of the CodeCommit account.
      id_canonical = "2B3C4D5E6F7A8B9C0D1E2F3A4B5C6D7E8F9A0B1C2D3E4F5A6B7C8D9E0F1A2B3C4D5E6F7" # Canonical ID of the CodeCommit account.
    }
  }

  stages = [{
    name = "Plan"
    action = { # https://docs.aws.amazon.com/codepipeline/latest/userguide/action-reference.html
      provider      = "CodeBuild"
      configuration = {
        ProjectName = "plan" # IAM Roles will be generated based on this name, so they need too be unique across pipelines.
      }
    }
    resource = {
      description = "MyPrefix: Plan project resources."
      script      = "${abspath(path.module)}/scripts/plan.yml"
      compute = {
        compute_type = "BUILD_GENERAL1_SMALL"
        image        = "aws/codebuild/standard:6.0-22.06.30" # "ACCOUNTID.dkr.ecr.REGION.amazonaws.com/ecr-repo:latest"
        type         = "LINUX_CONTAINER"
      }
    }
  }, {
    name   = "Apply"
    action = { # https://docs.aws.amazon.com/codepipeline/latest/userguide/action-reference.html
      provider      = "CodeBuild"
      configuration = {
        ProjectName = "apply" # IAM Roles will be generated based on this name, so they need too be unique across pipelines.
      }
    }
    resource = {
      description = "MyPrefix: Apply project resources."
      script      = "${abspath(path.module)}/scripts/plan.yml"
      compute = {
        compute_type = "BUILD_GENERAL1_SMALL"
        image        = "aws/codebuild/standard:6.0-22.06.30" # "ACCOUNTID.dkr.ecr.REGION.amazonaws.com/ecr-repo:latest"
        type         = "LINUX_CONTAINER"
      }
    }
  }]
}

Same-Account Terraform

This example creates a CodePipeline resource that pulls from a CodeCommit repository residing in the same account.

module "codecommit" {
  source    = "https://github.com/clearscale/tf-aws-cicd-codecommit.git?ref=v1.0.0"

  account = {
    id = "*", name = local.account.name, provider = "aws", key = "current", region = local.region.name
  }


  prefix  = local.context.prefix
  client  = local.context.client
  project = local.context.project
  env     = local.account.name
  region  = local.region.name
  name    = "codecommit"

  repo = {
    name   = "test-same-account"
    create = true
  }

  #
  # To be filled in and redeployed after the first successful deployment. These resources do not exist prior to module.cicd being deployed.
  # trusts = [
  #  "ARN_S3_BUCKET,
  #  "ARN_KMS_KEY", # Must be the key ARN - not an alias.
  #  "ARN_CODEPIPELINE"
  # ]
  #
}

module "cicd" {
  source = "https://github.com/clearscale/tf-aws-cicd.git?ref=v1.0.0"

  account = {
    id = "*", name = local.account.name, provider = "aws", key = "current", region = local.region.name
  }

  prefix  = local.context.prefix
  client  = local.context.client
  project = local.context.project
  env     = local.account.name
  region  = local.region.name
  name    = "cicd"


  # User account ARNs who can own and use the KMS key
  kms = {
    owners = [
      "arn:aws:iam::${module.import.vars.account.id}:role/aws-reserved/sso.amazonaws.com/AWSReservedSSO_AWSAdministratorAccess_1234567890123456"
    ]
    admins = [
      "arn:aws:iam::${module.import.vars.account.id}:role/aws-reserved/sso.amazonaws.com/AWSReservedSSO_AWSAdministratorAccess_1234567890123456"
    ]
    users = [
      "arn:aws:iam::${module.import.vars.account.id}:role/aws-reserved/sso.amazonaws.com/AWSReservedSSO_AWSAdministratorAccess_1234567890123456"
    ]
  }


  iam_service_role_policies = [
    "arn:aws:iam::aws:policy/PowerUserAccess"
  ]

  vpc = {
    id              = "VPC_ID"
    subnets         = ["SUBNET_ID_1", "SUBNET_ID_2"]
    security_groups = ["SG_ID_1", "SG_ID_2"],
  }

  repo = {
    name     = "test-same-account"
    branch   = "main"
    role_arn = module.codecommit.role.arn
  }

  stages = [{
    name = "Plan"
    action = {
      provider      = "CodeBuild"
      configuration = {
        ProjectName = "plan" # IAM Roles will be generated based on this name, so they need too be unique across pipelines.
      }
    }
    resource = {
      description = "MyPrefix: Plan project resources."
      script      = "${abspath(path.module)}/scripts/plan.yml"
      compute = {
        compute_type = "BUILD_GENERAL1_SMALL"
        image        = "aws/codebuild/standard:6.0-22.06.30" # "ACCOUNTID.dkr.ecr.REGION.amazonaws.com/ecr-repo:latest"
        type         = "LINUX_CONTAINER"
      }
    }
  }, {
    name   = "Apply"
    action = {
      provider      = "CodeBuild"
      configuration = {
        ProjectName = "apply" # IAM Roles will be generated based on this name, so they need too be unique across pipelines.
      }
    }
    resource = {
      description = "MyPrefix: Apply project resources."
      script      = "${abspath(path.module)}/scripts/plan.yml"
      compute = {
        compute_type = "BUILD_GENERAL1_SMALL"
        image        = "aws/codebuild/standard:6.0-22.06.30" # "ACCOUNTID.dkr.ecr.REGION.amazonaws.com/ecr-repo:latest"
        type         = "LINUX_CONTAINER"
      }
    }
  }]
}

Requirements

Name	Version
terraform	>= 1.6.1
aws	~> 5.0

Providers

Name	Version
aws	~> 5.0

Modules

Name	Source	Version
aws_cicd_codebuild	../tf-aws-cicd-codebuild	n/a
aws_cicd_codepipeline	../tf-aws-cicd-codepipeline	n/a
aws_cicd_codepipeline_iam	../tf-aws-cicd-codepipeline/iam	n/a
kms	terraform-aws-modules/kms/aws	2.0.1
s3_bucket	terraform-aws-modules/s3-bucket/aws	3.15.1
std	github.com/clearscale/tf-standards.git	v1.0.0
std_codepipeline	github.com/clearscale/tf-standards.git	v1.0.0

Resources

Name	Type
aws_caller_identity.current	data source
aws_canonical_user_id.current	data source

Inputs

Name	Description	Type	Default	Required
account	(Required). Cloud provider account object.	object({ key = optional(string, "current") provider = optional(string, "aws") id = optional(string, "*") name = string region = optional(string, null) })	{ "id": "*", "name": "shared" }	no
client	(Optional). Name of the client	string	"ClearScale"	no
env	(Optional). Name of the current environment.	string	"dev"	no
iam_service_role_policies	(Optional). List of IAM policy ARNs to attach to the primary service roles.	list(string)	[]	no
kms	(Optional). KMS key settings.	object({ owners = optional(list(string), []) admins = optional(list(string), []) users = optional(list(string), []) })	{}	no
name	(Optional). The name of the pipeline.	string	"default"	no
prefix	(Optional). Prefix override for all generated naming conventions.	string	null	no
project	(Optional). Name of the client project.	string	"int"	no
region	(Optional). Name of the region.	string	"us-west-1"	no
repo	(Required). Repository URL and branch name.	object({ name = string branch = string role_arn = string kms_key = optional(string, null) account = optional(object({ id = optional(number, null) id_canonical = optional(string, null) name = optional(string, null) }), null) })	n/a	yes
secrets	(Optional). List of secret names that are stored in Secrets Manager which CodeBuild should be able to read.	list(string)	[]	no
stages	(Required). List of stages for CodePipeline. configuration.ProjectName is required.	list(object({ name = string action = object({ name = optional(string, "Build") category = optional(string, "Build") provider = optional(string, "CodeBuild") version = optional(string, "1") owner = optional(string, "AWS") region = optional(string, null) input_artifacts = optional(list(string), null) configuration = optional(object({ ProjectName = string }), null) }) resource = object({ region = optional(string, null) name = optional(string, null) description = optional(string, null) script = optional(string, null) iam_service_role_policies = optional(list(string), []) # https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codebuild_project#environment # https://docs.aws.amazon.com/codebuild/latest/userguide/build-env-ref-available.html compute = optional(object({ compute_type = optional(string, "BUILD_GENERAL1_SMALL") image = optional(string, "aws/codebuild/amazonlinux2-x86_64-standard:5.0") type = optional(string, "LINUX_CONTAINER") })) # Inherits var.vpc if not set. vpc = optional(object({ id = optional(string, null) # vpc id subnets = optional(list(string), null) # ids security_groups = optional(list(string), null) # ids }), null) }) secrets = optional(list(string), []) logs = optional(list(string), []) }))	n/a	yes
vpc	(Required). Global VPC configuration for the CodeBuild projects where they are not specifically defined.	object({ id = string # vpc id subnets = list(string) # ids security_groups = list(string) # ids })	n/a	yes

Outputs

Name	Description
cache	Location path for the cached resources.
codebuild	All CodeBuild outputs.
codepipeline	All CodePipeline outputs.
kms	KMS related data.
name	The name of the CICD resources.
s3	KMS related data