Browse the Repo
.circleci.githubclierrorsformatkmstest
.gitignoreCODEOWNERSLICENSE.txtREADME.mdgo.modgo.summain.goBrowse the Repo
.circleci.githubclierrorsformatkmstest.gitignoreCODEOWNERSLICENSE.txtREADME.mdgo.modgo.summain.goGruntwork KMS
A command-line tool that makes it easy to encrypt and decrypt data using Amazon Key Management Service (KMS).
Code Preview
View on GitHub:
https://github.com/gruntwork-io/gruntkms/tree/v0.0.11Preview the Code
README.md
gruntkms
gruntkms is a command-line tool that makes it easy to encrypt and decrypt data using Amazon Key Management Service
(KMS). The primary use case is for storing encrypted secrets in config files.
Examples
Use the encrypt command to encrypt plaintext (note the result is prefixed with kmscrypt::):
> gruntkms encrypt --plaintext "some plaintext" --key-id alias/MyKmsKeyAlias --aws-region us-east-1
kmscrypt::abcdefg1234567adssdfsdf
Use the decrypt command to decrypt ciphertext:
> gruntkms decrypt --ciphertext "kmscrypt::abcdefg1234567adssdfsdf" --aws-region us-east-1
some plaintext
Note that the decrypt command will decrypt all text you pass to it that has the kmscrypt:: prefix. For example,
let's say you had a config.yml file with the following contents:
stage:
db:
username: admin
password: kmscrypt::abcdefg1234567adssdfsdf
prod:
db:
username: admin
password: kmscrypt::dkfk1ksdkfkkdkdk34k4k43
You could decrypt the entire file as follows (note how the decrypt command reads from stdin if --ciphertext is not
specified):
> cat config.yml | gruntkms decrypt --aws-region us-east-1
stage:
db:
username: admin
password: decrypted-password-stage
prod:
db:
username: admin
password: decrypted-password-prod
You can use this strategy to make the decrypted config content available to your apps without ever writing it to disk:
DECRYPTED_CONFIG=$(cat config.yml | gruntkms decrypt --aws-region us-east-1)
echo "$DECRYPTED_CONFIG" | run-my-app -config=/dev/stdin
Install
To install gruntkms, go to the Releases Page, download the
binary for your OS, rename it to gruntkms, and add it to your PATH.
Alternatively, you can use the Gruntwork Installer, which is especially convenient when installing gruntkms in a Packer template or Dockerfile:
gruntwork-install --binary-name 'gruntkms' --repo 'https://github.com/gruntwork-io/gruntkms' --tag 'v0.0.7'
Usage
Authenticate
gruntkms uses the standard authentication methods supported by all AWS CLI apps, including:
- Environment variables
AWS_ACCESS_KEY_IDandAWS_SECRET_ACCESS_KEY - Credentials stored in
~/.aws/credentials - IAM role if running this tool on an EC2 instance
Your IAM user must have access to whatever customer master keys (CMK) you are using for encryption or decryption.
Encrypt
The encrypt command encrypts whatever text is passed in via stdin or the --plaintext option and writes the
resulting ciphertext to stdout. It supports the following options:
--key-id(Required): The ID of the customer master key (CMK) to use for encryption. This value can be a globally unique identifier (e.g.12345678-1234-1234-1234-123456789012), a fully specified ARN (e.g.arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012), or an alias name prefixed by "alias/" (e.g.alias/MyAliasName). Follow the Creating Keys documentation to create a CMK if you don't have one already.--aws-region(Required): The AWS region where your customer master key (CMK) is defined (e.g. us-east-1). Defaults to the value of the environment variableDEFAULT_AWS_REGION.--plaintext(Optional): The plaintext to encrypt. If you don't specify this argument, you must pass the plaintext via stdin.--prefix(Optional): The prefix to prepend to the returned ciphertext. Default:kmscrypt::.--role-arn(Optional): The ARN of an IAM role to assume. Useful to use KMS keys in another AWS account. Can also be specified by the environment variableROLE_ARN.
Example:
> gruntkms encrypt --plaintext "some plaintext" --key-id alias/MyKmsKeyAlias --aws-region us-east-1
kmscrypt::abcdefg1234567adssdfsdf
Here is the same example as above, but this time passing the plaintext via stdin:
> echo "some plaintext" | gruntkms encrypt --key-id alias/MyKmsKeyAlias --aws-region us-east-1
kmscrypt::abcdefg1234567adssdfsdf
Decrypt
The decrypt command reads in any text that is passed in via stdin or the --ciphertext option, finds and decrypts all
text that has a kmscrypt:: prefix, and writes the result to stdout. The decrypt command supports the following
options:
--aws-region(Required): The AWS region where the customer master key (CMK) used to encrypt the ciphertext is defined (e.g. us-east-1). Defaults to the value of the environment variableDEFAULT_AWS_REGION.--ciphertext(Optional): The ciphertext to decrypt. If you don't specify this argument, you must specify the ciphertext via stdin. Note that only text that starts with the prefixkmscrypt::and is base64 encoded will be decrypted; the rest of the text will be returned unchanged.--prefix(Optional): The prefix that indicates the text immediately after it should be decrypted. The text after this prefix must be base 64 encoded. If you set the prefix to an empty string, everything that is base64 encoded will be decrypted. Default:kmscrypt::.--role-arn(Optional): The ARN of an IAM role to assume. Useful to use KMS keys in another AWS account. Can also be specified by the environment variableROLE_ARN.
Example:
> gruntkms decrypt --ciphertext "kmscrypt::abcdefg1234567adssdfsdf" --aws-region us-east-1
some plaintext
Here is the same example as above, but this time passing the ciphertext via stdin:
> echo "kmscrypt::abcdefg1234567adssdfsdf" | gruntkms decrypt --aws-region us-east-1
some plaintext
Note that any text that isn't preceded by the kmscrypt:: prefix is returned unchanged:
> echo "this-will-be-returned-unchanged kmscrypt::abcdefg1234567adssdfsdf this-will-also-be-unchanged" | gruntkms decrypt --aws-region us-east-1
this-will-be-returned-unchanged some plaintext this-will-also-be-unchanged
gruntkms vs other tools
How does gruntkms compare to tools such as Vault,
AWS Secrets Manager, and AWS Parameter
Store?
-
With the
gruntkmsapproach, the idea is to put the ciphertext of the secrets directly into your version control system so that the secrets are versioned, code reviewed, tested, and deployed exactly like the rest of your code. That’s because, contrary to popular belief, "config" and "secrets" are code and are just as risky to change as the rest of your code. By putting secrets in your code, every secret change goes through your full CI/CD pipline and all changes are versioned with your app. Moreover, a dev can easily see that there are config files for multiple environments sitting next to each other (e.g.,config-dev.json,config-stage.json,config-prod.json), so it’s less likely that you’ll add a config for one env, and forget another (which is a very common source of bugs). However, the downside is that rotating secrets in version control is harder—especially if a key got compromised and you have to update everything in a hurry! -
The alternative to the
gruntkmsapproach is to store your secrets in an external secret store such as Vault, AWS Secrets Manager, or AWS Parameter Store. The advantage of this approach is that the secrets are centrally managed, so you can rotate them all in one place. You also have support for single-use secrets, auditing, and many other advanced features (especially in Vault). Moreover, you can use not only a CLI tool, but also a UI to encrypt/decrypt secrets. The downside is that the secrets are not versioned, reviewed, tested, or deployed the way the rest of your code is, so you’re more likely to have bugs in this area.
It's up to you to decide which approach you believe is a better fit for your team. If you do go with option #2, for most teams, we'd recommend using AWS Secrets Manager, as AWS seems to be focusing on it more these days (as opposed to Parameter Store), it has a passable UI, and you don’t have to run it yourself (unlike Vault). We'd only recommend Vault (using our terraform-aws-vault module) for larger teams that have people/time you can dedicated to deploying, managing, and operationalizing Vault.
Developing gruntkms
Running locally
To run gruntkms locally, install Go (minimum version 1.13), and then use the
go run command:
go run main.go
Running tests
Before running the tests, you must configure your AWS credentials as explained in the Authenticate section.
To run all the tests:
go test -v -parallel 128 ./...
To run only the tests in a specific package, such as the package kms:
cd kms
go test -v -parallel 128
And to run a specific test, such as TestFoo in package kms:
cd kms
go test -v -parallel 128 -run TestFoo
Debug logging
If you set the GRUNTKMS_DEBUG environment variable to "true", the stack trace for any error will be printed to
stdout when you run the app.
Error handling
In this project, we try to ensure that:
- Every error has a stacktrace. This makes debugging easier.
- Every error generated by our own code (as opposed to errors from Go built-in functions or errors from 3rd party libraries) has a custom type. This makes error handling more precise, as we can decide to handle different types of errors differently.
To accomplish these two goals, we have created an errors package that has several helper methods, such as
errors.WithStackTrace(err error), which wraps the given error in an Error object that contains a stacktrace. Under
the hood, the errors package is using the go-errors library, but this may
change in the future, so the rest of the code should not depend on go-errors directly.
Here is how the errors package should be used:
- Any time you want to create your own error, create a custom type for it, and when instantiating that type, wrap it
with a call to
errors.WithStackTrace. That way, any time you call a method defined in thegruntkmscode, you know the error it returns already has a stacktrace and you don't have to wrap it yourself. - Any time you get back an error object from a function built into Go or a 3rd party library, immediately wrap it with
errors.WithStackTrace. This gives us a stacktrace as close to the source as possible. - If you need to get back the underlying error, you can use the
errors.IsErroranderrors.Unwrapfunctions.
Releasing new versions
To release a new version, just go to the Releases Page and create a new release. The CircleCI job for this repo has been configured to:
- Automatically detect new tags.
- Build binaries for every OS using that tag as a version number.
- Upload the binaries to the release in GitHub.
See circle.yml for details.
TODO
- Add support for data keys
- Add support for encryption context
License
Please see LICENSE.txt for details on how the code in this repo is licensed.
Questions? Ask away.
We're here to talk about our services, answer any questions, give advice, or just to chat.
