Browse the Repo

file-type-icondata-stores
file-type-iconlambda
file-type-iconmgmt
file-type-iconnetworking
file-type-iconsecurity
file-type-iconservice-templates
file-type-iconservices
file-type-iconasg-service
file-type-iconbeanstalk-service
file-type-iconecs-cluster
file-type-iconmigration_guides
file-type-iconpacker
file-type-iconuser-data
file-type-iconREADME.md
file-type-iconmain.tf
file-type-iconoutputs.tf
file-type-iconvariables.tf
file-type-iconecs-service-with-alb
file-type-iconeks-cluster
file-type-iconeks-core-services
file-type-iconk8s-namespace
file-type-iconk8s-service
file-type-iconstatic-website
file-type-icontls-scripts
file-type-icon.gitignore
file-type-iconCODEOWNERS
file-type-iconREADME.md

Browse the Repo

file-type-icondata-stores
file-type-iconlambda
file-type-iconmgmt
file-type-iconnetworking
file-type-iconsecurity
file-type-iconservice-templates
file-type-iconservices
file-type-iconasg-service
file-type-iconbeanstalk-service
file-type-iconecs-cluster
file-type-iconmigration_guides
file-type-iconpacker
file-type-iconuser-data
file-type-iconREADME.md
file-type-iconmain.tf
file-type-iconoutputs.tf
file-type-iconvariables.tf
file-type-iconecs-service-with-alb
file-type-iconeks-cluster
file-type-iconeks-core-services
file-type-iconk8s-namespace
file-type-iconk8s-service
file-type-iconstatic-website
file-type-icontls-scripts
file-type-icon.gitignore
file-type-iconCODEOWNERS
file-type-iconREADME.md
Sample Infrastructure Modules

Sample Infrastructure Modules

Sample infrastructure-modules repo (multi-account setup) for fictional company Acme.

Code Preview

Preview the Code

mobile file icon

README.md

down

ECS Cluster Module

This Terraform Module launches an EC2 Container Service Cluster that you can use to run Docker containers. The cluster consists of a configurable number of instances in an Auto Scaling Group (ASG). Each instance:

  1. Runs the ECS Container Agent so it can communicate with the ECS scheduler.
  2. Authenticates with a Docker repo so it can download private images. The Docker repo auth details should be encrypted using Amazon Key Management Service (KMS) and passed in as input variables. The instances, when booting up, will use gruntkms to decrypt the data in-memory. Note that the IAM role for these instances, which uses var.cluster_name as its name, must be granted access to the Customer Master Key (CMK) used to encrypt the data.
  3. Runs the CloudWatch Logs Agent to send all logs in syslog to CloudWatch Logs. This is configured using the cloudwatch-log-aggregation-scripts module.
  4. Emits custom metrics that are not available by default in CloudWatch, including memory and disk usage. This is configured using the cloudwatch-memory-disk-metrics-scripts module.
  5. Runs the syslog module to automatically rotate and rate limit syslog so that your instances don't run out of disk space from large volumes of logs.1. Runs the ssh-grunt module so that developers can upload their public SSH keys to IAM and use those SSH keys, along with their IAM user names, to SSH to the ECS Nodes.
  6. Runs the auto-update module so that the ECS nodes install security updates automatically.

How do you use this module?

  1. Build the AMI
  2. Deploy the Terraform code

Build the AMI

Each EC2 instance in the ECS cluster should run an AMI built using the Packer template in packer/ecs-node.json. To build the AMI:

  1. Install Packer.
  2. Set your AWS credentials as the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.
  3. Set your GitHub access token as the environment variable GITHUB_OAUTH_TOKEN. Your GitHub account must have access to the Gruntwork GitHub repos mentioned in packer/ecs-node.json; if you don't have access to those, email support@gruntwork.io.
  4. Run packer build ecs-node.json.
  5. When the build completes, it'll output the id of the new AMI.

Deploy the Terraform code

  • See the root README for instructions on how to deploy the Terraform code in this repo.
  • See variables.tf for all the variables you can set on this module.

How do you deploy updates to the cluster?

If you want to update the EC2 instances running in the ECS cluster (e.g. roll out a new AMI), you must use the roll-out-ecs-cluster-update.py script in the Gruntwork ecs-module. Check out the How do you make changes to the EC2 Instances in the cluster? documentation for details.

What is an ECS Cluster?

To use ECS, you first deploy one or more EC2 Instances into a "cluster". The ECS scheduler can then deploy Docker containers across any of the instances in this cluster. Each instance needs to have the Amazon ECS Agent installed so it can communicate with ECS and register itself as part of the right cluster.

Core concepts

For more info on ECS clusters, including how to run Docker containers in a cluster, how to add additional security group rules, how to handle IAM policies, and more, check out the ecs-cluster documentation in the module-ecs repo. For info on finding your Docker container logs in CloudWatch, check out the cloudwatch-log-aggregation-scripts documentation. For info on viewing the custom metrics in CloudWatch, check out the cloudwatch-memory-disk-metrics-scripts documentation.

Questions? Ask away.

We're here to talk about our services, answer any questions, give advice, or just to chat.

Ready to hand off the Gruntwork?