Read our newest book, Fundamentals of DevOps and Software Delivery, for free!

Authenticating to AWS with Instance Metadata

Headshot of Yevgeniy Brikman

Yevgeniy Brikman

AUG 6, 2018 | 7 min read
Featured Image of Authenticating to AWS with Instance Metadata
This is Part 3 of the Comprehensive Guide to Authenticating to AWS on the Command Line. In Part 2, we went over how to use Environment Variables, which were secure, but complicated to use. In this post, we’ll introduce a third option for authenticating to AWS on the Command Line: Instance Metadata.

What do you do if you want to authenticate to AWS from an EC2 Instance? For example, you have an app that needs to make API calls to AWS to download data from S3. Normally, you’d authenticate to AWS using Access Keys, but how do you get those Access Keys onto the EC2 Instance? Putting them directly in your application code or a config file is a bad idea, as that means your credentials will be in plain text, on disk, accessible to any attacker that manages to get access to the EC2 Instance or your code.A better idea is to use Instance Metadata. AWS exposes an Instance Metadata endpoint on every EC2 Instance at http://169.254.169.254. If you SSH to an EC2 Instance, you can use any HTTP client, such as curl, to get information about that Instance from its Instance Metadata endpoint:
$ curl http://169.254.169.254/latest/meta-data/ ami-id ami-launch-index ami-manifest-path block-device-mapping/ hostname iam/ instance-action instance-id instance-type local-hostname local-ipv4 mac metrics/ network/ placement/ profile public-hostname public-ipv4 public-keys/ reservation-id security-groups services/$ curl http://169.254.169.254/latest/meta-data/public-hostname ec2-203-0-113-25.compute-1.amazonaws.com
Part of the information available in EC2 Metadata are Temporary Access Keys that are automatically provided and rotated by AWS! The permissions that you get from these Temporary Access Keys are determined by the IAM Role (if any) attached to that EC2 Instance. You’ve seen earlier in this blog post that IAM Roles can be assumed by IAM Users to give them certain permissions; well, you can also create IAM Roles that can be assumed by various AWS Services, including EC2 Instances.For example, you can create an IAM Role with a Trust Policy that allows an EC2 Instance to assume that IAM Role:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "ec2.amazonaws.com"}, "Action": "sts:AssumeRole" } ] }
You can then give this IAM Role various IAM permissions, such as being able to read data from S3:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": ["s3:GetObject"], "Resource": ["arn:aws:s3:::examplebucket/*"] } ] }
Next, you can attach this IAM Role to your EC2 Instance, and the Instance Metadata endpoint will give any AWS CLI or SDK tool running on that Instance permission to read from that S3 bucket without you having to manually deal with Access Keys!

With Instance Metadata and IAM Roles, you don’t manage the Access Keys yourself, so there’s nothing to do here!

The permissions for Instance Metadata come from the IAM Role attached to your EC2 Instance, so this is handled automatically for you.

Since Instance Metadata is meant to be used by applications and automated tools, where there is no person to type in an MFA token, MFA is not supported.

If you are running Docker containers using Amazon’s Elastic Container Service (ECS), then you don’t want your Docker container relying on the Instance Metadata endpoint of the underlying EC2 Instance. Instead, you can use the Task Metadata endpoint, which serves the same purpose, but is exposed by ECS directly within your Docker container at http://169.254.170.2 (all AWS CLI and SDK tools know to check this endpoint).Likewise, instead of attaching an IAM Role to your EC2 Instance, you’ll want to attach an IAM Role directly to the ECS Task using ECS Task IAM Roles. This way, you can give your Docker containers specific IAM permissions (e.g., read access to an S3 bucket) without having to manually fuss with Access Keys.

A major drawback of the Instance Metadata endpoint is that it can be accessed by any OS user on your EC2 Instance. So if a hacker manages to break into a low-level user account, they can immediately use all of the permissions that EC2 Instance has from its IAM Role.Therefore, we strongly recommend locking down the Instance Metadata endpoint so it is only accessible to specific OS users. You can lock down IPs on Linux Instances using iptables or via the ip-lockdown script in the Infrastructure as Code Library:
ip-lockdown 169.254.169.254 root
The command above will only allow the Instance Metadata endpoint to be accessed by the root user, which makes life much harder for attackers. Moreover, if your Instance only needs IAM permissions during boot (e.g., to download a config file from S3), you could even block access to the Instance Metadata endpoint entirely after boot has completed:
# Fetch data during boot aws s3 cp s3:// ...# Block the Instance Metadata endpoint entirely ip-lockdown 169.254.169.254

  • You don’t have to manually manage Access Keys.
  • Access Keys are never stored in plaintext on disk.
  • Uses Temporary Access Keys that expire frequently and are automatically rotated by AWS.

  • Only works with AWS Services (e.g., EC2 Instances).
  • The Instance Metadata endpoint is vulnerable to attackers (unless you use something like ip-lockdown to protect it).

Thanks to Instance Metadata, you should never have to manually copy Access Keys over to your EC2 Instances. Instead, attach an IAM Role to those Instances, give that IAM Role the permissions your apps need, and everything will “just work.” However, make sure to use a tool such as ip-lockdown to protect the Instance Metadata endpoint from attackers!We hope you’ve found this guide to AWS authentication helpful. If you’re stuck or need help, feel free to ask questions in the comments!Get your DevOps superpowers at Gruntwork.io.
Share
Grunty
Resources

Explore our latest blog

Get the most up-to-date information and trends from our DevOps community.
TerraformResouces Image

Promotion Workflows with Terraform

How to configure GitOps-driven, immutable infrastructure workflows for Terraform using Gruntwork Patcher.
avatar

Jason Griffin

October 3, 2023 7 min read
TerraformResouces Image

The Impact of the HashiCorp License Change on Gruntwork Customers

How to configure GitOps-driven, immutable infrastructure workflows for Terraform using Gruntwork Patcher.
avatar

Josh Padnick

October 3, 2023 7 min read