How to handle multiple environments in CodePipeline?
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/continuous-delivery-codepipeline-basic-walkthrough.html
Template configuration files are applied to the CloudFormation in the CodePipeline via a parameter file like this:
{
"Parameters" : {
"DBName" : "TestWordPressDB",
"DBPassword" : "TestDBRootPassword",
"DBRootPassword" : "TestDBRootPassword",
"DBUser" : "TestDBuser",
"KeyName" : "TestEC2KeyName"
}
}
Place these files in the root of your repo and they can be referenced in at least 2 ways.
In your CodePipeline CloudFormation:
Configuration:
ActionMode: REPLACE_ON_FAILURE
RoleArn: !GetAtt [CFNRole, Arn]
StackName: !Ref TestStackName
TemplateConfiguration: !Sub "TemplateSource::${TestStackConfig}"
TemplatePath: !Sub "TemplateSource::${TemplateFileName}"
Or in the console in the Template configuration field:
It is worth noting the config file format is different from CloudFormation via cli using
-- parameters
--parameters uses this format:
[
{
"ParameterKey": "team",
"ParameterValue": "AD-Student Life Applications"
},
{
"ParameterKey": "env",
"ParameterValue": "dev"
},
{
"ParameterKey": "dataSensitivity",
"ParameterValue": "public"
},
{
"ParameterKey": "app",
"ParameterValue": "events-list-test"
}
]
CodePipeline Cloudformation template configuration files use this format:
{
"Parameters" : {
"DBName" : "TestWordPressDB",
"DBPassword" : "TestDBRootPassword",
"DBRootPassword" : "TestDBRootPassword",
"DBUser" : "TestDBuser",
"KeyName" : "TestEC2KeyName"
}
}
Check the Eric Nord's answer. It is the one you are looking for.
I asked the question on the AWS forum as well here.
Here is the solution provided by AWS:
Hi,
If your goal is to have different bucket names for staging and master, then another option is to use CloudFormation parameters.
When editing an existing pipeline if you edit an action you can expand the "Advanced" panel and enter parameter overrides to specify a different bucket prefix for each stage. You can also enter parameters as a separate .json file in your artifact.
There's more details on doing that here: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/continuous-delivery-codepipeline-parameter-override-functions.html
Here's a full walk through with a different stack configuration for test and production: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/continuous-delivery-codepipeline-basic-walkthrough.html
- Tim.
You should definitely follow the documentation provided. Here's the solution I came up with below.
Here is my own solution that I wasn't satisfied with.
I've added a script running at build time and modifying my template given the ARN of the CodeBuild agent building the project.
I've added "BRANCH_NAME" where naming collision can occur. The image_processing_sam.yml is now:
AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"
Description: Create a thumbnail for an image uploaded to S3
Resources:
ThumbnailFunction:
Type: "AWS::Serverless::Function"
Properties:
Role: !GetAtt LambdaExecutionRole.Arn
Handler: create_thumbnail.handler
Runtime: python2.7
Timeout: 30
Description: "A function computing the thumbnail for an image."
LambdaSecretEncryptionKey:
Type: "AWS::KMS::Key"
Properties:
Description: "A key used to encrypt secrets used in the Lambda functions"
Enabled: True
EnableKeyRotation: False
KeyPolicy:
Version: "2012-10-17"
Id: "lambda-secret-encryption-keyBRANCH_NAME"
Statement:
-
Sid: "Allow administration of the key"
Effect: "Allow"
Principal:
AWS: "arn:aws:iam::xxxxxxxxxxxxx:role/cloudformation-lambda-execution-role"
Action:
- "kms:Create*"
- "kms:Describe*"
- "kms:Enable*"
- "kms:List*"
- "kms:Put*"
- "kms:Update*"
- "kms:Revoke*"
- "kms:Disable*"
- "kms:Get*"
- "kms:Delete*"
- "kms:ScheduleKeyDeletion"
- "kms:CancelKeyDeletion"
Resource: "*"
-
Sid: "Allow use of the key"
Effect: "Allow"
Principal:
AWS:
- !GetAtt LambdaExecutionRole.Arn
Action:
- "kms:Encrypt"
- "kms:Decrypt"
- "kms:ReEncrypt*"
- "kms:GenerateDataKey*"
- "kms:DescribeKey"
Resource: "*"
LambdaExecutionRole:
Type: "AWS::IAM::Role"
Properties:
RoleName: "LambdaExecutionRoleBRANCH_NAME"
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service:
- "lambda.amazonaws.com"
Action:
- "sts:AssumeRole"
Policies:
-
PolicyName: LambdaKMSBRANCH_NAME
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: "Allow"
Action:
- "kms:Decrypt"
Resource: "*"
-
Effect: "Allow"
Action:
- "lambda:InvokeFunction"
Resource: "*"
ManagedPolicyArns:
- "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
UserGroup:
Type: "AWS::IAM::Group"
LambdaTriggerUser:
Type: "AWS::IAM::User"
Properties:
UserName: "LambdaTriggerUserBRANCH_NAME"
LambdaTriggerUserKeys:
Type: "AWS::IAM::AccessKey"
Properties:
UserName:
Ref: LambdaTriggerUser
Users:
Type: "AWS::IAM::UserToGroupAddition"
Properties:
GroupName:
Ref: UserGroup
Users:
- Ref: LambdaTriggerUser
Policies:
Type: "AWS::IAM::Policy"
Properties:
PolicyName: UserPolicyBRANCH_NAME
PolicyDocument:
Statement:
-
Effect: "Allow"
Action:
- "lambda:InvokeFunction"
Resource:
- !GetAtt DispatcherFunction.Arn
Groups:
- Ref: UserGroup
PackageBucket:
Type: "AWS::S3::Bucket"
Properties:
BucketName: "package-bucketBRANCH_NAME"
VersioningConfiguration:
Status: "Enabled"
Outputs:
LambdaTriggerUserAccessKey:
Value:
Ref: "LambdaTriggerUserKeys"
Description: "AWSAccessKeyId of LambdaTriggerUser"
LambdaTriggerUserSecretKey:
Value: !GetAtt LambdaTriggerUserKeys.SecretAccessKey
Description: "AWSSecretKey of LambdaTriggerUser"
The script.sh replacing the "BRANCH_NAME" in the template is:
#!/bin/bash
echo $CODEBUILD_AGENT_ENV_CODEBUILD_BUILD_ARN
if [[ "$CODEBUILD_AGENT_ENV_CODEBUILD_BUILD_ARN" == *"master"* ]]; then
sed "s/BRANCH_NAME//g" image_processing_sam.yml > generated_image_processing_sam.yml;
fi
if [[ "$CODEBUILD_AGENT_ENV_CODEBUILD_BUILD_ARN" == *"staging"* ]]; then
sed "s/BRANCH_NAME/staging/g" image_processing_sam.yml > generated_image_processing_sam.yml;
fi
The buildspec.yml is now:
version: 0.1
phases:
install:
commands:
# Install required module for python
- pip install requests -t .
- pip install simplejson -t .
- pip install Image -t .
- bash ./script.sh
# To be able to see any issue in the generated template
- cat generated_image_processing_sam.yml
# Package the generated cloudformation template in order to deploy
- aws cloudformation package --template-file generated_image_processing_sam.yml --s3-bucket piximate-package-bucket --output-template-file new_image_processing_sam.yml
artifacts:
type: zip
files:
- new_image_processing_sam.yml
I hope it can somehow help you. I would be glad if anyone can provide any improvement or documentation that could help.