Browse the Repo

file-type-icon_docs
file-type-icon_images
file-type-icon01-architecture-overview.md
file-type-icon02-whats-deployed.md
file-type-icon03-security-compliance-compatibility.md
file-type-icon04-how-code-is-organized.md
file-type-icon05-dev-environment.md
file-type-icon06-ci-cd.md
file-type-icon07-monitoring-alerting-logging.md
file-type-icon08-ssh-vpn.md
file-type-icon09-accounts-and-auth.md
file-type-icon10-gruntwork-tools.md
file-type-icon11-deploying-a-docker-service.md
file-type-icon12-migration.md
file-type-icon13-deploying-the-reference-architecture-fr...
file-type-icon14-undeploying-the-reference-architecture.md
file-type-icon15-adding-new-environments-regions-and-acc...
file-type-iconREADME.md
file-type-icondev
file-type-iconmaster
file-type-iconprod
file-type-iconsecurity
file-type-iconshared-services
file-type-iconstage
file-type-icon.gitignore
file-type-iconCODEOWNERS
file-type-iconREADME.md

Browse the Repo

file-type-icon_docs
file-type-icon_images
file-type-icon01-architecture-overview.md
file-type-icon02-whats-deployed.md
file-type-icon03-security-compliance-compatibility.md
file-type-icon04-how-code-is-organized.md
file-type-icon05-dev-environment.md
file-type-icon06-ci-cd.md
file-type-icon07-monitoring-alerting-logging.md
file-type-icon08-ssh-vpn.md
file-type-icon09-accounts-and-auth.md
file-type-icon10-gruntwork-tools.md
file-type-icon11-deploying-a-docker-service.md
file-type-icon12-migration.md
file-type-icon13-deploying-the-reference-architecture-fr...
file-type-icon14-undeploying-the-reference-architecture.md
file-type-icon15-adding-new-environments-regions-and-acc...
file-type-iconREADME.md
file-type-icondev
file-type-iconmaster
file-type-iconprod
file-type-iconsecurity
file-type-iconshared-services
file-type-iconstage
file-type-icon.gitignore
file-type-iconCODEOWNERS
file-type-iconREADME.md
Multi-account Reference Architecture

Multi-account Reference Architecture

End-to-end tech stack designed to deploy into multiple AWS accounts. Includes VPCs, EKS, ALBs, CI / CD, monitoring, alerting, VPN, DNS, and more.

Code Preview

Preview the Code

mobile file icon

01-architecture-overview.md

down

Architecture Overview

Let's start by talking about your overall architecture. Acme Multi Account's architecture is deployed on top of Amazon Web Services (AWS) using the Gruntwork Reference Architecture.

Here's a diagram that shows an overview of what the Reference Architecture looks like:

Architecture Diagram

Note that the Reference Architecture is highly customizable, so what's deployed may be a bit different than what's in the diagram. Here is an overview of what's actually deployed:

  1. Infrastructure as code
  2. Environments
  3. AWS accounts
  4. VPCs and subnets
  5. Load balancers
  6. Docker clusters (EKS)
  7. Data stores
  8. OpenVPN server
  9. Jenkins
  10. Monitoring, log aggregation, alerting
  11. DNS and TLS
  12. Static content, S3, and CloudFront
  13. Lambda
  14. Security

Infrastructure as code

All of Acme Multi Account's infrastructure is managed as code, primarily using Terraform. That is, instead of clicking around a web UI or SSHing to a server and manually executing commands, the idea behind infrastructure as code (IAC) is that you write code to define your infrastructure and you let an automated tool (e.g., Terraform) apply the code changes to your infrastructure. This has a number of benefits:

  • You can automate your entire provisioning and deployment process, which makes it much faster and more reliable than any manual process.

  • You can represent the state of your infrastructure in source files that anyone can read rather than a sysadmin's head.

  • You can store those source files in version control, which means the entire history of your infrastructure is captured in the commit log, which you can use to debug problems, and if necessary, roll back to older versions.

  • You can validate each infrastructure change through code reviews and automated tests.

  • You can package your infrastructure as reusable, documented, battle-tested modules that make it easier to scale and evolve your infrastructure. In fact, much of the infrastructure code in this architecture is powered by modules created by Gruntwork, which are called Infrastructure Packages.

For more info on Infrastructure as Code and Terraform, check out A Comprehensive Guide to Terraform.

Environments

The infrastructure is deployed across multiple environments:

  • dev (account id: 087285199408): Sandbox environment.

  • master (account id: 087285199408): Consolidated billing account

  • prod (account id: 087285199408): Production environment.

  • security (account id: 087285199408): All IAM users and permissions are defined in this account.

  • shared-services (account id: 087285199408): DevOps tooling, such as the OpenVPN server.

  • stage (account id: 087285199408): Pre-production environment.

AWS accounts

Your infrastructure is deployed across multiple AWS accounts. For example, the staging environment is in one account, the production environment in another account, the DevOps tooling in yet another account, and so on. This gives you better isolation between environments so that if you break something in one environment (e.g., staging)—or worse yet, a hacker breaks into that environment—it should have no effect on your other environments (e.g., prod). It also gives you better control over what resources each employee can access.

Check out Accounts and Auth for more info on the AWS accounts that have been set up and how to authenticate to and switch between them.

VPCs and subnets

Each environment lives in a separate Virtual Private Cloud (VPC), which is a logically isolated section within an AWS account. Each VPC defines a virtual network, with its own IP address space and rules for what can go in and out of that network. The IP addresses within each VPC are further divided into multiple subnets, where each subnet controls the routing for its IP address.

  • Public subnets are directly accessible from the public Internet.
  • Private subnets are only accessible from within the VPC.

Just about everything in this infrastructure is deployed in private subnets to reduce the surface area to attackers. The only exceptions are load balancers and the OpenVPN server, both of which are described below.

Each VPC is also configured with VPC flow logs, which can be useful for monitoring and auditing network traffic across the VPC. Each VPC publishes its flow logs to CloudWatch Logs, under the log group VPC_NAME-vpc-flow-logs, where the VPC_NAME is an input variable to the vpc-mgmt and vpc-app modules.

To learn more about VPCs and subnets, check out the Gruntwork vpc-app module documentation.

Load balancers

Traffic from the public Internet (e.g., requests from your users) initially goes to a public load balancer, which proxies the traffic to your apps. This allows you to run multiple copies of your application for scalability and high availability. The load balancers being used are:

  • Application Load Balancer (ALB): The ALB is a load balancer managed by AWS that is designed for routing HTTP and HTTPS traffic. The advantage of using a managed service is that AWS takes care of fault tolerance, security, and scaling the load balancer for you automatically. Note that in EKS, ALBs are managed by Kubernetes using Ingress resources. Check out the eks-alb-ingress-controller documentation for more information on how this works.

We also deploy an internal load balancer in the private subnets. This load balancer is not accessible to the public. Instead, it's used as a simple way to do service discovery: every backend service registers with the load balancer at a particular path, and all services know to send requests to this load balancer to talk to other services.

Docker clusters

Your application code is packaged into Docker containers and deployed across an Amazon Elactic Container Service for Kubernetes Cluster (EKS) The advantage of Docker is that it allows you to package your code so that it runs exactly the same way in all environments (dev, stage, prod). The advantage of a Docker Cluster is that it makes it easy to deploy your Docker containers across a cluster of servers, making efficient use of wherever resources are available. Moreover, EKS can automatically scale your app up and down in response to load and redeploy containers that crashed.

For a quick intro to Docker, see Running microservices on AWS using Docker, Terraform, and ECS. For more info on using EKS, see terraform-aws-eks.

Data stores

The infrastructure includes the following data stores:

  1. Mysql: Mysql is deployed using Amazon's Relational Database Service (RDS), including automatic failover, backups, and replicas. Check out module-data-storage for more info.

  2. Redis: Redis is deployed using Amazon's ElastiCache Service, including automatic failover, backups, and replicas. Check out module-cache for more info.

  3. ZooKeeper: A ZooKeeper cluster is deployed using package-zookeeper.

  4. Kafka: A Kafka cluster is deployed using package-kafka.

Lambda

We have deployed several example Lambda functions to show how you can build serverless applications. Check out the package-lambda docs for background info.

OpenVPN server

To reduce your surface area to attackers, just about all of the resources in this infrastructure run in private subnets, which are not accessible from the public Internet at all. To allow Acme Multi Account's employees to access these private resources, we expose a single server publicly: an OpenVPN server. Once you connect to the server using a VPN client, you are "in the network", and will be able to access the private resources (e.g., you will be able to SSH to your EC2 Instances).

For more info, see SSH and VPN and package-openvpn.

Jenkins

We have set up Jenkins as a Continuous Integration (CI) server. After every commit, a Jenkins job runs your build, tests, packaging, and automated deployment steps.

For more info, see Build, tests, and deployment (CI/CD) and module-ci.

Monitoring, log aggregation, alerting

You can find metrics, log files from all your servers, and subscribe to alert notifications using Amazon CloudWatch.

For more info, see Monitoring, Alerting, and Logging and module-aws-monitoring.

DNS and TLS

We are using Amazon Route 53 to configure DNS entries for all your services. We have configured SSL/TLS certificates for your domain names using Amazon's Certificate Manager (ACM), which issues certificates that are free and renew automatically.

For more info, see What's deployed.

Static content, S3, and CloudFront

All static content (e.g., images, CSS, JS) is stored in Amazon S3 and served via the CloudFront CDN. This allows you to offload all the work of serving static content from your app server and reduces latency for your users.

For more info, see What's deployed and package-static-assets.

Security

We have configured security best practices in every aspect of this infrastructure:

Check out Gruntwork Security Best Practices for more info.

Next steps

Next up, let's have a look at What's deployed.

Questions? Ask away.

We're here to talk about our services, answer any questions, give advice, or just to chat.

Ready to hand off the Gruntwork?