Read our newest book, Fundamentals of DevOps and Software Delivery, for free!

Authenticating to AWS with Environment Variables

Headshot of Yevgeniy Brikman

Yevgeniy Brikman

AUG 6, 2018 | 21 min read
Featured Image of Authenticating to AWS with Environment Variables
This is Part 2 of the Comprehensive Guide to Authenticating to AWS on the Command Line. In Part 1, we went over how to use the Credentials File, but found that while easy to use, it was not particularly secure. In this post, we’ll introduce a second option for authenticating to AWS on the Command Line: Environment Variables.

Another way to authenticate to AWS on the CLI is to set your Access Keys as the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY:
export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCEXAMPLEKEY
Or if you’re on Windows:
set AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE set AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Now you can run any CLI tool and it will use these credentials automatically:
export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCEXAMPLEKEY terraform apply
Important note: Setting environment variables as shown above, at least on Linux/Unix/Mac, will store your credentials in your bash history. That means your credentials would be stored in plaintext, on disk! There are a few ways to prevent this.The first is simply to add a space before the export commands:
export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxfiCEXAMPLEKEY terraform apply
By default, commands with a leading space are not stored in bash history (note: you may need to tweak your HISTCONTROL setting to be sure, as described here).The second, and more secure option, is to store your credentials in a CLI-friendly password store, such as pass. For example, you can store the credentials as follows:
$ pass insert aws-access-key-id Enter aws-access-key-id: AKIAIOSFODNN7EXAMPLE$ pass insert aws-secret-access-key Enter aws-secret-access-key: wJalrXUtnFEMI/K7MDENG/bPxRfiCEXAMPLEKEY
When you run the commands above, pass will save the credentials in two files, aws-access-key-id and aws-secret-access-key, encrypting each file with gpg. Now, you can set your environment variables as follows, with no risk of any secrets being stored in bash history:
export AWS_ACCESS_KEY_ID=$(pass aws-access-key-id) export AWS_SECRET_ACCESS_KEY=$(pass aws-secret-access-key)
You can even take the two lines above, put them into a script called auth.sh, and set your environment variables with a single command:
. auth.shterraform apply
In the rest of this blog post, whenever you see credentials being exported as environment variables, make sure to use one of the methods above to ensure those secrets aren’t stored in your bash history!

If you have multiple sets of Access Keys (e.g., for multiple IAM Users in different AWS accounts), you just set the same environment variable names for each set of Access Keys. For example, in one terminal tab, you could authenticate using the Access Keys from your stage account:
export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE1 export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bxRfiCEXAMPLEKEY1
In another terminal, you can use the Access Keys from your prod account:
export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE2 export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bxRfiCEXAMPLEKEY2

If you want to assume IAM Roles — for example, you have an IAM User in the security account and want to assume an IAM Role in your dev account — you have two options. The first option, as discussed in the Credentials File blog post, is to do it in the CLI tool you’re using (e.g., Terraform), if it supports it.The second option, with environment variables, is quite a bit trickier. First, you need to authenticate with your normal (permanent) Access Keys:
export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCEXAMPLEKEY
Next, you call aws sts assume-role, passing it the ARN of the IAM Role you want to assume, plus a “role session name” that can be used to tell who is assuming the IAM Role and why (as the same IAM Role may be assumed by may different users):
aws sts assume-role \ --role-arn arn:aws:iam::123456789012:role/dev-full-access \ --role-session-name username@company.com
This will return a blob of JSON that contains Temporary Access Keys:
{ "Credentials": { "SecretAccessKey": "secret-access-key", "SessionToken": "temporary-session-token", "Expiration": "expiration-date-time", "AccessKeyId": "access-key-id" } }
You must now set these Temporary Access Keys as environment variables, overriding the old environment variables:
export AWS_ACCESS_KEY_ID=<Access-key-from-output> export AWS_SECRET_ACCESS_KEY=<Secret-access-key-from-output> export AWS_SESSION_TOKEN=<Session-Token-from-output>
Note that with Temporary Access Keys, you must not only set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables, but also AWS_SESSION_TOKEN. Going through this aws sts assume-role process manually each time you want to assume an IAM Role is tedious, so most teams use scripts to automate this process (more on this below).Also, take note that, by default, the Temporary Access Keys you get from aws sts assume-role expire after just 1 hour. If you plan to use those Temporary Access Keys as your credentials all day long, and don’t want to have to re-authenticate every hour, you should:
aws sts assume-role \ --role-arn arn:aws:iam::123456789012:role/dev-full-access \ --role-session-name username@company.com \ --duration-seconds 43200

Using MFA with environment variables is also a tricky, multi-step process. First, you need to authenticate with your normal (permanent) Access Keys:
export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCEXAMPLEKEY
Next, you run the aws sts get-session-token command, passing it the ARN of your MFA device and an MFA token from the Google Authenticator App or your key fob:
aws sts get-session-token \ --serial-number arn:aws:iam::123456789012:mfa/jon-doe \ --token-code 123456 \ --duration-seconds 43200
This will return a blob of JSON that contains Temporary Access Keys (note the --duration-seconds argument in the earlier command, which specifies when these Temporary Access Keys will expire):
{ "Credentials": { "SecretAccessKey": "secret-access-key", "SessionToken": "temporary-session-token", "Expiration": "expiration-date-time", "AccessKeyId": "access-key-id" } }
You must now set these Temporary Access Keys as environment variables, overriding the old environment variables:
export AWS_ACCESS_KEY_ID=<Access-key-from-output> export AWS_SECRET_ACCESS_KEY=<Secret-access-key-from-output> export AWS_SESSION_TOKEN=<Session-Token-from-output>
Note that if you want to both use MFA and assume an IAM Role, then you should call aws sts assume-role as in the previous section, but include --serial-number and --token-code parameters as in this section:
aws sts assume-role \ --role-arn arn:aws:iam::123456789012:role/dev-full-access \ --role-session-name username@company.com \ --serial-number arn:aws:iam::123456789012:mfa/jon-doe \ --token-code 123456 \ --duration-seconds 43200
Working with assume-role and get-session-token is a tedious and error-prone process, so most team use scripts and tools to automate this process, as described in the next section.

There are a several tools that make it easier to use environment variables to authenticate to AWS:

The team at 99designs has created an open source, cross-platform CLI tool called aws-vault that can:
  • 1.Securely store your AWS credentials in your operating system’s keystore (e.g., Keychain, KWallet)
  • 2.Automatically set those credentials as environment variables when executing a command.
  • 3.Handle all the aws sts commands for you when using IAM Roles or MFA.
The basic usage is as follows. First, you tell aws-vault to store your AWS Access Keys under a Named Profile (note: aws-vault uses Named Profiles, just like the Credentials File, but it does not store the credentials in plaintext in the Credentials File itself):
$ aws-vault add dev-full-access Enter Access Key Id: AKIAIOSFODNN7EXAMPLE Enter Secret Key: wJalrXUtnFEMI/K7MDENG/bPxRfiCEXAMPLEKEY
Now you can use the aws-vault exec command to set those Access Keys as environment variables when executing a CLI command:
aws-vault exec home -- terraform apply
To assume an IAM role, you can specify the role_arn parameter in your Config File in ~/.aws/config:
[profile dev-full-access] role_arn = arn:aws:iam::123456789012:role/dev-full-access
Now, next time you run aws-vault exec dev-full-access, aws-vault will automatically assume an IAM Role for you. Similarly, you can specify an mfa_serial in ~/.aws/config:
[profile dev-full-access] mfa_serial = arn:aws:iam::123456789012:mfa/jon-doe
Each time you run aws-vault exec dev-full-access, it’ll prompt you for an MFA token.

aws-auth is a bash script available in the Gruntwork Infrastructure as Code Library that automates all the aws sts steps for using environment variables. For example, to assume an IAM Role, you first set your normal (permanent) AWS Access Keys as environment variables:
export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCEXAMPLEKEY
And then you can run:
eval "$(aws-auth --role-arn arn:aws:iam::123456789011:role/my-role)"
This will automatically set new AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN environment variables for you, so you can immediately begin running commands that need AWS authentication:
eval "$(aws-auth --role-arn arn:aws:iam::123456789011:role/my-role)"terraform apply
Similarly, to use MFA, you can run:
eval "$(aws-auth \ --serial-number arn:aws:iam::123456789011:mfa/jondoe \ --token-code 123456)"terraform apply
Again, this will set the AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN environment variables for you. And, of course, you can assume an IAM Role and use MFA at the same time:
eval "$(aws-auth \ --serial-number arn:aws:iam::123456789011:mfa/jondoe \ --token-code 123456 \ --role-arn arn:aws:iam::123456789011:role/my-role)terraform apply
aws-auth works especially well when combined with a CLI-friendly password manager, such as pass. Instead of having to manually set your permanent AWS Access Keys as environment variables, and to manually pass the IAM Role ARN and MFA serial as a command-line args, you can store them securely on disk (encrypted with gpg) using pass:
$ pass insert aws-access-key-id Enter aws-access-key-id: AKIAIOSFODNN7EXAMPLE$ pass insert aws-secret-access-key Enter aws-secret-access-key: wJalrXUtnFEMI/K7MDENG/bPxRfiCEXAMPLEKEY$ pass insert aws-mfa-arn Enter aws-mfa-arn: arn:aws:iam::123456789011:mfa/jondoe$ pass insert aws-iam-role-arn Enter aws-iam-role-arn: arn:aws:iam::123456789011:role/foo
Now you can create a script called, for example, auth.sh, with the following contents:
read -p "Enter your MFA token: " token eval $(AWS_ACCESS_KEY_ID=$(pass aws-access-key-id) \ AWS_SECRET_ACCESS_KEY=$(pass aws-secret-access-key) \ aws-auth \ --serial-number $(pass aws-mfa-arn) \ --token-code "$token" \ --role-arn $(pass aws-iam-role-arn))
When you run this script, it will set permanent AWS Access Keys as environment variables, prompt you for an MFA Token, do all the aws sts calls to use the MFA Token and assume an IAM Role, and set environment variables for you with your new Temporary Access Keys, all in a single command:
eval "$(auth.sh)"terraform apply
If you’re a Gruntwork Subscriber, you can download aws-auth here (note: if you’re not a subscriber, you’ll get a 404!).

  • Your Access Keys are never stored in plaintext on disk, so this is more secure than the Credentials File.
  • Environment variables are only set in the current shell (i.e., the current terminal tab), so you’re less likely to accidentally use a default profile unintentionally.

  • You’re always using permanent Access Keys for auth rather than Temporary Access Keys that are rotated.
  • Using MFA is complicated and error prone (however, aws-vault and aws-auth make it easier).
  • Using IAM Roles is complicated and error prone (however, aws-vault and aws-auth make it easier).

Environment Variables offer more security than the Credentials File, but at the cost of usability, especially if you’re using IAM Roles or MFA. Fortunately, there are third party tools such as aws-vault and aws-auth to make your life a little easier.In the next part of the series, we’ll talk about an alternative to Environment Variables that you can use for apps running in EC2: Authenticating to AWS with Instance Metadata.Your entire infrastructure. Defined as code. In about a day. Gruntwork.io.
Share
Grunty
Resources

Explore our latest blog

Get the most up-to-date information and trends from our DevOps community.
TerraformResouces Image

Promotion Workflows with Terraform

How to configure GitOps-driven, immutable infrastructure workflows for Terraform using Gruntwork Patcher.
avatar

Jason Griffin

October 3, 2023 7 min read
TerraformResouces Image

The Impact of the HashiCorp License Change on Gruntwork Customers

How to configure GitOps-driven, immutable infrastructure workflows for Terraform using Gruntwork Patcher.
avatar

Josh Padnick

October 3, 2023 7 min read