aws

Gruntwork Newsletter, Summer 2021

Once a month (ish), we send out a newsletter to all Gruntwork customers that describes all the updates we’ve made in the last month, news…
Gruntwork Newsletter, Summer 2021
Amanda Ohmer
Published September 17, 2021

Once a month (ish), we send out a newsletter to all Gruntwork customers that describes all the updates we’ve made in the last month, news in the DevOps industry, and important security updates. Note that many of the links below go to private repos in the Gruntwork Infrastructure as Code Library and Reference Architecture that are only accessible to customers.

Hello Grunts,

In the last few months we launched the Gruntwork Developer Portal, opened up early access to our HIPAA compliance solution, upgraded all of our modules to work with Terraform 1.0, updated Gruntwork Pipelines with first-class support for destroy, added a new API Gateway Proxy module, made progress on the Ubuntu 20.04 upgrade, updated several of our deployment guides, and much more!

As always, if you have any questions or need help, email us at support@gruntwork.io!

Gruntwork Updates

Introducing: The Gruntwork Developer Portal

Motivation: For years, if you were a Gruntwork customer, there was no central place to discover Gruntwork products or manage your subscription: for example, if you wanted to add or remove users, you had to email us; after we processed that email, those new users would get a bunch of random links and emails; and later on, if you forgot about those emails, figuring out how to use Gruntwork could be a challenge.

Solution: We’re excited to announce the Gruntwork Developer Portal, a centralized, self-service solution for Gruntwork customers! It’s available at app.gruntwork.io, runs on top of our own Reference Architecture (dogfooding!), and you can use it to manage your subscription, add and remove users, link your GitHub account, and discover all of Gruntwork’s products!

What to do about it: If you’re a subscriber, you’ve either gotten a welcome email already, or will get one shortly in the next few weeks. We’d love to hear how it works for you!

Introducing: HIPAA compliance early access

Motivation: For the last few years, Gruntwork has offered infrastructure that is compliant out-of-the-box with the CIS AWS Foundations Benchmark, but many customers have requested out-of-the-box support for additional standards, including HIPAA compliance.

Solution: We are excited to offer early access to our HIPAA compliance solution! This includes:

  • A Reference Architecture that is HIPAA-compliant, out-of-the-box.
  • Application templates that show how to build apps (e.g., Spring Boot) that are HIPAA-compliant (e.g., properly handle PHI) and work seamlessly in the Reference Architecture.
  • HIPAA audit assistance, where we’ll go through an audit with you, and fix any issues your auditor identifies in the infrastructure.

What to do about it: If your company needs to achieve HIPAA compliance on AWS, request early access now!

It’s time to upgrade to Terraform 1.0!

Motivation: In June, HashiCorp released version 1.0 of Terraform. This is a significant milestone that indicates Hashicorp’s commitment to language stability for Terraform across releases from this point on. The first 1.0 release is actually a rebranding of version 0.15.5 , which means that most of our modules should work out of the box with Terraform 1.0. Nevertheless we have to go through all the modules in our IaC library to update our tests to make sure there are no unexpected backward incompatible changes.

Solution: We have tested and updated all of our modules to work with Terraform 1.0! Check out our How to Update to Terraform 1.x guide for instructions on how to update your code.

What to do about it: Follow the instructions in our upgrade guide and let us know how it works for you!

[NEW FEATURE] First-class support for undeploying infrastructure in Gruntwork Pipelines

Motivation: In our CI / CD solution, called Gruntwork Pipelines, many customers have requested support for running destroy on Terraform modules.

Solution: We’re excited to announce that we now have first-class support for destroy! To signal that you want to destroy some infrastructure, delete the folder the code is in, and commit the changes to a branch in your infrastructure-live repo. Gruntwork Pipelines will detect that you want to destroy the modules in question and will show you the output of terragrunt plan -destroy. If the plan looks good, merge the branch, and Gruntwork Pipelines will run terragrunt plan -destroy one more time, prompt you for a final confirmation via Slack, and if you approve, it’ll run terragrunt apply -destroy. So in short, delete the folder, commit, and Gruntwork Pipelines takes care of the rest!

What to do about it: First, read more about this feature in this blog post. If you don’t have Gruntwork Pipelines, read our guide on how to set it up. If you want a Reference Architecture, talk to us! If you have one already, follow the migration guide to upgrade it to include the new destroy support!

Here’s a quick demo of Gruntwork Pipelines in action:

[video]

A teaser on how Gruntwork Pipelines works

[NEW MODULE] API Gateway Proxy

Motivation: Using the modules in Gruntwork’s terraform-aws-lambda repo, it’s easy to deploy and manage Lambda functions. But what if you want to build a serverless web app by setting up API Gateway in front of your Lambda functions? One option is to do a bunch of complicated routing configuration in API Gateway itself, or use something like the Serverless Framework, but what if you (a) want to forward all API requests to (b) a Lambda function running a standard web framework, such as Express.js?

Solution: This month we released a new module called api-gateway-proxy that allows you to deploy API Gateway in proxy mode, forwarding all requests under a given path to a lambda function running your serverless application. This provides an easy way to expose an app built on top of a standard web framework as a serverless web app. For example, if you built an app using Express.js, you can adapt it to run in a Lambda function using the serverless-express plugin, and deploy it as a serverless web app in just a few lines of Terraform:

# Deploy your Express.js app as a Lambda function
module "lambda" {
source = "git::git@github.com:gruntwork-io/terraform-aws-lambda.git//modules/lambda?ref=v0.12.0"
name        = "example-express-app"
source_path = "./example-express-app"
runtime     = "nodejs14.x"
handler     = "index.handler"
memory_size = 512
timeout     = 300
}
# Create an API Gateway to proxy all requests to the Lambda function
module "api_gateway" {
source = "git::git@github.com:gruntwork-io/terraform-aws-lambda.git//modules/api-gateway-proxy?ref=v0.12.0"
api_name = "example-api"
lambda_functions = {
# The "" key means proxy all routes to the function
"" = module.lambda.function_name
}
}

What to do about it: Try deploying the new module to front your Lambda function and take it for a spin!

Ubuntu 20.04 progress

Motivation: Last month we announced our plans to update to Ubuntu 20.04, which is the latest LTS release for Ubuntu since April 23rd, 2020.

Solution: We continue to work towards supporting Ubuntu 20.04 in our library. This month, the following modules have been confirmed to work with Ubuntu 20.04 (and the release where the confirmation was made):

The following modules are on deck:

  • terraform-aws-vault
  • terraform-aws-nomad
  • terraform-aws-couchbase

What to do about it: Keep an eye out for our releases and the monthly newsletter for continuous updates on the progress of Ubuntu 20.04 compatibility in our modules!

Updated Guides: Gruntwork AWS Landing Zone and CIS

Motivation: We launched the Gruntwork Service Catalog a while back, but had not yet updated all of our production deployment guides to use it.

Solution: We have now updated the Landing Zone and CIS guides to use the Service Catalog instead of a custom copy of the code in an infrastructure-modules repo, so that you can spin up your architecture that much faster and maintain 10x less code!

What to do about it: Read the Gruntwork AWS Landing Zone guide as well as the CIS Compliance guide! We’ll update the other guides as soon as we can too.

Service Catalog Updates

terraform-aws-service-catalog

  • v0.40.4: You can now configure the subnet spacing / sizing in the vpc module using the new input variables subnet_spacing, private_subnet_spacing, persistence_subnet_spacing, public_subnet_bits, private_subnet_bits, and persistence_subnet_bits.
  • v0.40.5: You can now override the sources of the external-dns app in eks-core-services
  • v0.41.0: AMIs updated to use Ubuntu 20.04 as base.
  • v0.41.1: Update dependency gruntwork-io/bash-commons to v0.1.4; Update dependency gruntwork-io/terraform-aws-load-balancer to v0.26.0.
  • v0.41.2: Update various dependencies to latest version. Refer to the release notes for more info.
  • v0.41.3: Update various dependencies to latest version. Refer to the release notes for more info.
  • v0.41.4: Allow setting cluster_autoscaler_version in eks-core-services; Update all packer template names to use the format COMPONENT-OS.json .
  • v0.42.0: Update eks dependency to v0.41.0; Update kubergrunt dependency to v0.7.1; Create a new module eks-workers that lets you manage EKS worker groups (self-managed ASGs and Manged Node Groups) separately from the EKS cluster; Add support for deploying Managed Node Groups. IMPORTANT: This is a major backward incompatible release for eks-cluster. A naive update will redeploy all worker nodes and cause downtime. Refer to the migration guide for strategies to avoid the downtime.
  • v0.42.1: Update dependency gruntwork-io/bash-commons to v0.1.7; [ec2-baseline] Make sure each log file managed by cloudwatch-agent goes to separate streams.
  • v0.43.0: Updates dependency gruntwork-io/terraform-aws-monitoring to v0.28.0 (metrics are now being reported by cloudwatch-agent); CloudWatch Logs group names are now configurable for ECS cluster; Updated the for-production/infrastructure-live examples with many bug fixes and updates; Setting sse_algorithm to null will now disable encryption on S3 buckets.
  • v0.44.0: Jenkins module backup function is now converted to use AWS Data Lifecycle Manager instead of a custom lambda function. If you wish to continue to use the lambda based backup function, you can set backup_using_lambda = true; The dashboard widgets and alarms for EC2 and ASG based modules have been updated to work with the new CloudWatch agent instead of cloudwatch-memory-disk-metrics. To ensure compatibility, make sure to rebuild your server AMIs to align with this version.
  • v0.44.1: You can now get the ID of the default security group from the vpc module using the new default_security_group_id output variable. Updated the for-production examples to the latest.
  • v0.44.2: You can now expose the type of traffic to capture in VPC flow logs in the vpc module using the new traffic_type input variable.
  • v0.44.4: Expose several new variables in the Landing Zone modules (account-baseline-app, account-baseline-root, account-baseline-security) for configuring CloudTrail.
  • v0.44.5: You can now avoid creating the default ACM certificate in the route53 module by setting provision_certificates on the input parameter.
  • v0.44.6: k8s-service: add support for custom resources
  • v0.44.7: Fix bug where eks-cluster required both worker types.
  • NOTE: Due to an error in releasing, we skipped v0.45–v0.49 accidentally. The next release after v0.44.7 is correctly v0.50.0.
  • v0.50.0:Add docs on why we are not using module count/for_each for the eks-worker modules; Add docs on the IAM roles and aws-auth for eks-workers; Update dependency gruntwork-io/terraform-aws-security to v0.50.0.
  • v0.50.1: Fix bug in services/ec2-instance where ip-lockdown referenced a non-existant user; Expose root_volume_size configuration option for services/ec2-instance; Fix bug where the build permissions were not being attached to the Jenkins IAM role; Fix various typos in docs and comments.
  • v0.50.2: Add new module for managing IAM Users and Groups only in Landing Zone. Refer to the module docs for more information on when you might want to use this over the same feature in account-baseline-security.
  • v0.50.3: You can now attach custom security group rules to the EKS worker pool managed by eks-cluster and eks-workers. For eks-workers module, use the new custom_ingress_security_group_rules and custom_egress_security_group_rules input variables. For eks-cluster, use the new custom_worker_ingress_security_group_rules and custom_worker_egress_security_group_rules input variables.
  • v0.50.4: Jenkins server AMI now contains aws-auth.
  • v0.50.5: k8s-service now exposes a way to configure side car containers.
  • v0.50.6: Replaced gofmt with goimports and update versions of underlying modules.
  • v0.50.7: Updated dependency gruntwork-io/terraform-aws-eks to v0.42.1. With this change, you can now configure self managed workers to use multiple instance types for a single pool (using the newly exposed multi_instances_overrides attribute).
  • v0.50.8: Fixed a bug where eks-cluster errors out when trying to lookup IAM role for Managed Node Groups or Self Managed Workers after having none; route53 module now supports creating NS records for creating delegated public hosted zones; updated versions of various dependencies.
  • v0.51.0: Added the ability to associate multiple VPCs to private route 53 hosted zone. This is a backward incompatible change. Refer to the Release notes for information on how to update to this version.
  • v0.51.1: Reduced the footprint of a test file so that hopefully future test runs will take less time. Added support for custom endpoints in the Elasticsearch module. Made it easier to pass in EC2 instance type to the ECS packer template.
  • v0.51.2: Exposed the ability to pass through volumes (including EFS volumes) to the wrapped ecs-service module.
  • v0.51.3: Added ability to configure replica_backup_retention_period. This can be used to enable automated backups of RDS read replica instances. Updated various dependencies.
  • v0.51.4: All packer templates have been updated to take in an availability_zone variable, which can be used to specify which AZ to use when spinning up the build instance. This is useful to avoid problematic AZs that do not have standard instance types available. Also updated dependency gruntwork-io/terraform-aws-eks to v0.42.2.
  • v0.52.0: Updated dependency gruntwork-io/terragrunt to v0.31.1. Updated dependency gruntwork-io/terraform-aws-vpc to v0.16.0. This is a backward incompatible update. Refer to the underlying release note for more information on the update.
  • v0.53.0: Update account-baseline-xxx modules to new multi-region approach. This reduces the number of providers that Terraform must instantiate, making the account-baseline-xxx modules much faster and more stable to use. It also gives you full control over how to authenticate to your various AWS accounts. Also, update dependency versions for a number of dependencies in this repo.
  • v0.54.0: Updated various dependencies. Updated to use kubernetes provider v2.x.
  • v0.55.0: We have verified that this repo is compatible with Terraform 1.0.x, and we are now testing against Terraform 1.0.x. Also fixed a bug with ec2-instance service module where the EBS volume properties were not being set correctly.
  • v0.56.0: This release updates all the Packer templates to HCL2. See the Getting started guide from HashiCorp for details on HCL2. This release is backward incompatible — refer to the migration guide for more information.
  • v0.56.1: Exposed additional_security_group_ids in ec2-instance module.
  • v0.56.2: Added support for configuring Horizontal Pod Autoscaler (via the horizontal_pod_autoscaler input variable) and overriding chart inputs (via the override_chart_inputs input variable).
  • v0.57.0: The default Kubernetes versions of modules have been updated to 1.21. The module dependency terraform-aws-eks has also been updated to v0.44.3.
  • v0.58.0: Added the ability to track external Fargate Profile executor IAM roles in the aws-auth configmap. Fixed bug where managed node groups could not be updated post deployment due to an error message about MIME format. Fixed bug where using managed node groups sometimes caused an error with IAM roles for self managed ASGs. Fixed bug where baseline IAM policies for various services were not being attached to managed node groups.
  • v0.58.2: When you set dns_zone_is_private to true, the ec2-instance module will now associate the private IP of the instance with the Route 53 private zone.
  • v0.58.3: Added support for passing CORS Rules via var.cors_rules to s3-bucket.
  • v0.58.4: Added ability to configure additional filters on fluent-bit in eks-core-services module. Updated various dependencies (see the release notes for the full list).
  • v0.59.0: Set MFA Delete to false by default on S3 buckets. Adding apply_default_nacl_rules to the VPC module.
  • v0.59.1: Updated eks-workers module to allow specifying per ASG --kublet-extra-args. You can now configure eks_kubelet_extra_args on each ASG group to override the extra args that should be passed to the underlying kubelet process. You can also configure different user data boot scripts for each worker by setting the cloud_init_parts field on the ASG configuration.
  • v0.59.2: Updated eks-workers module to allow specifying per Managed Node Group (MNG) --kublet-extra-args. You can now configure eks_kubelet_extra_args on each MNG group to override the extra args that should be passed to the underlying kubelet process. You can also configure different user data boot scripts for each worker by setting the cloud_init_parts field on the MNG configuration.
  • v0.59.3: Optionally create service-linked roles for security account using var.service_linked_roles.
  • v0.59.4: Updated ecs-service module with newly added inputs to configure App Mesh behavior.
  • v0.60.0: Updated eks-core-services to configure fluent-bit for log aggregation for Fargate pods. Updated k8s-service module to expose the target-type parameter on the Ingress resource. Updated openvpn and eks-worker packer templates to expose region_kms_key_ids variable for encrypting multi region images. Update various dependencies — refer to the release notes for more details.
  • v0.60.1: All packer templates now support configuring region_kms_key_ids for encrypting multi region AMIs.
  • v0.60.2: Updated the example code in examples/for-production/infrastructure-live with support for destroying infrastructure. Updates to vpc and ecs-service modules.

terraform-aws-cis-service-catalog

  • v0.21.0: Updates versions of underlying modules.
  • v0.22.0: Introduces new folder layout.
  • v0.22.1: Fixes a bug in password policies. Amend maximum unused credentials age to 45 days. Update versions of underlying modules.
  • v0.22.2: Adds a locking mechanism to SecurityHub tests, to prevent a race condition that happened during concurrent runs of these tests. Adds for-production examples. Updates variable description for the SecurityHub’s email. Cleans up unused variables in account-baseline-root. Updates log filters to meet CIS 1.4 recommendations. Updates version references from v1.3 to v1.4 throughout the codebase.
  • v0.23.0: Removes the custom Python code that we had for managing SecurityHub member invitations, and replaces it with the native Terraform approach using the aws_securityhub_invite_accepter resource. Refer to the release notes for migration information.
  • v0.23.1: Updates versions of underlying modules.
  • v0.23.2: Improves testing infrastructure, updates for-production examples, and updates versions of underlying modules.
  • v0.23.3: Adds missing variables for Bucket configs at Account Baseline Root and removes unused code from SecurityHub codegen script.
  • v0.23.4: Update underlying modules terraform-aws-monitoring and terraform-aws-service-catalog ; Replace gofmt with goimports ; Remove paralellism limit for go tests.
  • v0.23.5: Fix bug where VPC module did not plumb through the variables for configuring NACL rules for peering.
  • v0.23.6: Added ability to control password policy variables that CIS has no preference on; Added validation logic to ensure minimum password length can not be set below 14.
  • v0.24.0: Update dependency gruntwork-io/terraform-aws-security to v0.50.0 and gruntwork-io/terraform-aws-service-catalog to v0.50.0
  • v0.24.1: Override renovate.json ignorePaths so that it won’t ignore examples or tests. vpc: Expose default security group ID in outputs.
  • v0.25.0: Update modules to new multi-region approach. This reduces the number of providers that Terraform must instantiate, making the modules much faster and more stable to use. It also gives you full control over how to authenticate to your various AWS accounts. Also, update dependency versions for a number of dependencies in this repo.
  • v0.25.1: Add a module for deploying and configuring Amazon Macie. This release also configures the RenovateBot not to update this repo itself, as well as updates a number of dependencies.
  • v0.26.0: We have verified that this repo is compatible with Terraform 1.0.x, and we are now testing against Terraform 1.0.x.
  • v0.26.1: macie: Add support for multi-account setup. This release also updates a number of dependencies, as well as for-production examples. See release notes for more information.
  • v0.26.2: Integrates Macie into the Landing Zone modules. This release also makes the buckets_to_analyze variable optional and defaults it to empty. When buckets_to_analyze has no entry for a particular region, the resource aws_macie2_classification_job will not be created in that region.
  • v0.26.3: Updates a bunch of underlying dependencies: terraform-aws-vpc , terraform-aws-security and terraform-aws-service-catalog
  • v0.27.0: Updates MFA Delete default value to false . This release reverts v0.24.0.
  • v0.27.1: This release enables passing additional data logging resources to the cloudtrail module, via the new additional_data_logging_resources variable. This release also updates all the README badges to point to the v1.4.0 of the CIS Benchmark. It also updates the versions of a number of dependencies.
  • v0.27.2: Updates the example code in examples/for-production/infrastructure-live with support for destroying infrastructure.
  • v0.27.3: This release introduces changes in variables of account-baseline-root: pass cloudtrail_kms_key_arn to base module and make cloudtrail_should_create_s3_bucket and config_should_create_s3_bucket configurable. It also updates the versions of a number of dependencies.

Open Source Updates

Terragrunt

  • v0.29.9: You can now configure the --terragrunt-include-external-dependencies setting via the environment variable TERRAGRUNT_INCLUDE_EXTERNAL_DEPENDENCIES.
  • v0.29.10: Fixed a bug where yamldecode is unable decode certain forms of yaml.
  • v0.30.0: aws-provider-patch now supports additional data types. Previously aws-provider-patch only supported patching strings, which made it impossible to patch provider attributes that are not strings (e.g., the allowed_account_ids attribute of the aws provider, which is list(string) type).
  • v0.30.1: Updated documentation and help text in hclfmt command to clarify that it works on all files with hcl extension, not just terragrunt.hcl.
  • v0.30.2: Improved error messages in aws-provider-patch when the json input is malformed.
  • v0.30.3: Improved error messaging when multiple errors are returned.
  • v0.30.4: You can now access values from included config. E.g., if you want to access a local var region defined in the parent terragrunt config, you can reference include.locals.region in the child config.
  • v0.30.5: Fixed a bug where exposing include did not work when referencing in locals blocks.
  • v0.30.6: Fixed a the way the remote_state block handles the AWS partition settings so that it works correctly with GovCloud.
  • v0.30.7: Fixed a bug where using an exposed include with local in the same expression did not work when referencing in locals blocks.
  • v0.31.0: Updated to start testing Terragrunt against Terraform 1.0. Terraform functions have been updated to the versions shipped with Terraform 0.15.3 (previously the functions were pulled in from Terraform 0.12.24). These may include backward incompatibilities. Refer to the terraform release notes for more information.
  • v0.31.1: Addressed bug where plan file args were not always passed to the end of the arg list when calling terraform. Now terragrunt will check all the args and determine if an arg is a plan file (a filename that exists on disk and ends with extension tfplan), and if it is, feed it to the end of the args list.
  • v0.31.2: Added new attribute to include: merge_strategy. merge_strategy indicates how the included parent config should be merged with the child config. Currently, this only supports no_merge and shallow. When omitted, the merge strategy defaults to shallow (the same strategy as previous versions).
  • v0.31.3: Added deep merge strategy for include. Refer to the updated documentation for more information on what deep merge means.
  • v0.31.4: Added merge_state_and_mocks_outputs option. When true, dependency fetching will always merge in mock_outputs into the dependency output.
  • v0.31.5: Added support for fetching modules from any Terraform Registry using the new tfr:// protocol syntax for the source attribute. See the updated docs on source for more details.
  • v0.31.6: Fix a bug where run_cmd could be executed more than once.
  • v0.31.9: Terragrunt will now support Terraform registries, such as GitLab, that return relative paths. Terragrunt will now detect and show an error for generate blocks with duplicate names.
  • v0.31.10: Terragrunt will now show an error if source or dependencies are referencing a folder that doesn't exist.

Terratest

  • v0.35.2: Added new functions to load and save SSH KeyPairs directly (as opposed to through Ec2KeyPair.
  • v0.35.3: Added new function to allow explicitly using in-cluster auth (k8s.NewKubectlOptionsWithInClusterAuth).
  • v0.35.4: Introduced additional functions for retrieving Azure Resource Groups: azure.GetAResourceGroup and azure.ListResourceGroupsByTag.
  • v0.35.5: Adapted count parser to tf 0.15.5; Updated example az keyvault cert.
  • v0.35.6: Fixed bug where run-all commands were all unconditionally adding in -lock=false.
  • v0.36.0: Fixed a bug in the GetBucketTagging module where it wouldn’t handle buckets correctly that had recently been deleted, but were still being returned by the ListBuckets API. Fix a number of bugs in the ValidateAllTerraformModules function, including copying code to a temp folder before executing, skipping .terraform hidden folders, and refactoring to avoid an import cycle.
  • v0.36.1: Fixed bugs with how Terratest handles the -no-color and -var/-var-file options with Terraform. In particular, we (a) no longer append -var or -var-file when calling apply with a plan file and (b) append -no-color at the end of the args, where Terraform expects it.
  • v0.36.3: For each SSM method XXX, added an XXXWithClient method that lets you pass in an SSM client, rather than a new one being created each time. terraform.Options now supports the Reconfigure and MigrateState params.
  • v0.36.4: Updated terraform-json to latest version.
  • v0.36.5: Allowed initializing KV Authorizer using environment variables.
  • v0.36.6: Namespace flag is now always included in helm functions.
  • v0.36.7: Added methods to handle jobs in terratest: k8s.ListJob, k8s.GetJob, k8s.WaitUntilJobSucceed
  • v0.36.8: Added support for Azure FunctionApp. Refer to appService.go for the supported functions.
  • v0.37.0: We have verified that this repo is compatible with Terraform 1.0.x, and we are now testing against Terraform 1.0.x.
  • v0.37.1: Added helper function terraform.InitAndPlanAndShowWithStructNoLogTempPlanFile for extracting plan struct without logging and without specifying a plan file.
  • v0.37.2: This release adds support for Packer v1.7.0. Refer to the linked PR below for details. This change is backward compatible with previous versions of packer.
  • v0.37.3: Added support for Azure Container Registry. Refer to the functions in the containers.go file for more info.
  • v0.37.4: Updated docker.Build to support multiarchitecture builds. When options.Architectures is set, the Build function will use docker buildx to build a multiarchitecture image. Note that the buildx plugin must be installed and initialized for this functionality to work. Updated docker.Build to support pushing the image after build (when options.Push is set to True). Added functions for pushing a docker image (docker.Push and docker.PushE)
  • v0.37.5: Added helper functions for interacting with Azure Container Instances.
  • v0.37.7: The terraform.GetVariableAsXXX methods (e.g., terraform.GetVariableAsStringFromVarFile) can now parse .tfvars.json files in addition to .tfvars.
  • v0.37.8: Fix the log formatting of the shell commands so that they aren't accidentally trying to process interpolations (e.g., %s) that happen to be in the log output.

cloud-nuke

  • v0.2.0: Cloud nuke will now delete secrets manager secrets. If you wish to avoid nuking secrets manager entries, you can either pass in --exclude-resource-type secretsmanager, or specify a config file.
  • v0.3.0: Cloud nuke will now delete NAT Gateways. If you wish to avoid nuking NAT Gateways, you can either pass in --exclude-resource-type nat-gateway, or specify a config file.

bash-commons

  • v0.1.5: os_create_user and os_change_dir_owner now have the ability to run respective commands using sudo.
  • v0.1.6: Fixed a bug introduced with using sudo for the os_create_user and os_change_dir_owner functions.
  • v0.1.7: Introduced assert_user_has_sudo_perms which checks if user has permissions to run sudo. Introduced string_substr which can extract a substring by index.

go-commons

  • v0.9.2: Add new functionality to allow locking of resources when they’re being used from multiple places simultaneously. We expect this to be useful especially when testing AWS resources/services which can only be enabled once for the whole AWS Account — such as SecurityHub, GuardDuty, or any others.
  • v0.10.0: Add new function to upload string contents to S3 bucket; Refactor shell functions to use a common underlying runtime interface; New shell run command functions RunShellCommandAndGetOutputStruct and RunShellCommandAndGetOutputStructAndStreamOutput which will return a struct that captures stdout, stderr, and merged outputs so you can access all of those outputs; Refactor implementation of streaming and capturing outputs.

gruntwork-installer

  • v0.0.37: Added support for installing arm64 binaries.

kubergrunt

  • v0.7.2: Fixed a bug in the core components update for Kubernetes version 1.20, where coredns required additional permissions. The sync-core-components command will now patch the ClusterRole with the necessary permissions.
  • v0.7.3: This adds support for Kubernetes version 1.21 in the sync-core-components command.
  • v0.7.4: Fixed a bug where eks deploy gets stuck waiting for load balancers to register when the Service load balancer is using a NLB.
  • v0.7.5: Fixed a bug where eks deploy crashes with NLBs that have more than one target group.
  • v0.7.6: Updated eks sync-core-components command to use the correct version of coredns deployed on EKS clusters with kubernetes version 1.21 (CoreDNS v1.8.4).
  • v0.7.7: Fixed a bug in eks sync-core-components where the eksbuild tag to use for coredns and kube-proxy depended on the region of the EKS cluster. kubergrunt will now identify the correct eksbuild tag to use for the specific region of the cluster.
  • v0.7.8: Fixed a bug in eks deploy where it did not correctly identify ALBs managed by the AWS LoadBalancer Controller.
  • v0.7.9: Updated eks sync-core-components to deploy aws-vpc-cni version 1.9.

helm-kubernetes-services

  • v0.2.2: You can now override the fullname that appears in the resources managed by the k8s-service helm chart using the fullnameOverride input variable.
  • v0.2.3: Updated configMaps and secrets to support injecting with envFrom so that all keys are automatically loaded as environment variables.
  • v0.2.4: Added ability to configure initContainers on the Deployment using the new input initContainers.

terraform-kubernetes-namespace

  • v0.4.0: We have verified that this repo is compatible with Terraform 1.0.x, and we are now testing against Terraform 1.0.x.

terraform-aws-utilities

  • v0.6.0: We have verified that this repo is compatible with Terraform 1.0.x, and we are now testing against Terraform 1.0.x.

terraform-aws-nomad

  • v0.9.1: The install-nomad script now supports ARM processors!
  • v0.10.0: We have verified that this repo is compatible with Terraform 1.0.x!

terraform-google-security

  • v0.2.1: Replaced gofmt with goimport, added a Terraform validate test, and updated the CircleCI image for the test suite.

terraform-google-load-balancer

  • v0.4.2: Added a Terraform validate test, and updated the CircleCI image for the test suite.

terraform-google-network

  • v0.8.2: Replaced gofmt with goimport, added a Terraform validate test, and updated the CircleCI image for the test suite.

terraform-google-sql

  • v0.5.1: Replace gofmt with goimport, add a Terraform validate test, and update the CircleCI image for the test suite.

terraform-google-static-assets

  • v0.5.1: Replaced gofmt with goimport, added a Terraform validate test, and updated the CircleCI image for the test suite.

terraform-google-gke

  • v0.9.1: Replaced gofmt with goimport, added a Terraform validate test, and updated the CircleCI image for the test suite.

terraform-aws-vault

  • v0.17.0: We have verified that this repo is compatible with Terraform 1.0.x!

Other updates

gruntwork CLI

  • v0.2.2: Added a new gruntwork aws reset-password feature so you can reset your IAM user password and when you receive your Reference Architecture.

terraform-aws-eks

  • v0.40.1: Added support for skipping individual components during cluster upgrades. Note that you will need kubergrunt version v0.7.1 and above to take advantage of the skip feature.
  • v0.41.0: Made various fixes to managed node groups module. Refer to the release notes for more details.
  • v0.41.1: Added ability to update the aws-for-fluent-bit version that is installed (var.aws_for_fluent_bit_version).
  • v0.42.0: Fixed undocumented variable multi_instance_overrides so you can also set weighted_capacity in eks-workers module.
  • v0.42.1: Fixed RBAC permissions for aws-auth-merger so that it can create a new aws-auth ConfigMap when it doesn't exist.
  • v0.42.2: Added ability to pass in an IAM role ARN directly to avoid depending on a data source in the workers modules, which can be a source of perpetual diffs.
  • v0.42.3: Added new variable pod_resources which can be used to control the resource allocation for the cluster-autoscaler.
  • v0.43.0: We have verified that this repo is compatible with Terraform 1.0.x, and we are now testing against Terraform 1.0.x. Updated required kubernetes provider version to 2.x.
  • v0.44.0: The default Kubernetes version deployed by the control plane module has been updated to 1.21. If you wish to maintain backward compatibility with your existing setup, you will want to configure the kubernetes_version parameter to the version of Kubernetes you are currently using. Note that 1.21 requires kubergrunt version 0.7.3 and above.
  • v0.44.1: Upgraded dependencies of aws-auth-merger. Updated examples to use packer 1.7 with HCL2. Bumped reference kubergrunt version to 0.7.4.
  • v0.44.2: Removed usage of the deprecated template provider and replaced them with HashiCorp recommended replacements.
  • v0.44.3: Updated Cluster Autoscaler IAM permissions to allow describing launch templates.
  • v0.44.4: Exposed extraFilters helm chart input value with the extra_filters var in the eks-container-logs module.
  • v0.44.5: Bump default kubergrunt download URL to the latest version.
  • v0.44.6: Updated the kubergrunt version that gets automatically installed to v0.7.9
  • v0.44.7: You can now set the expanderPriorities and priorityConfigMapAnnotations properties of the cluster-auto-scaler Helm chart using the new expander_priorities and priority_config_map_annotations input variables, respectively. These settings are useful when If scaling_strategy is set to priority.
  • v0.44.8: Added new module (eks-fargate-container-logs) for setting up log aggregation to AWS resources (CloudWatch Logs, Kinesis, or Elasticsearch) for Fargate pods.
  • v0.45.0: Updated the AWS Load Balancer Controller helm chart to use v2.2.3 (previously v2.1.3). v2.2.0 introduced several breaking changes.

terraform-aws-security

  • v0.49.1: Added a new feature to the custom-iam-entity module to make it easier to create an IAM group that only has permissions to assume one or more IAM roles.
  • v0.49.2: Added a new AWS Config rule for checking unused credentials.
  • v0.49.3: Made optional SSE for S3 buckets.
  • v0.49.4: You can now attach inline custom IAM policies on the IAM group/role managed by custom-iam-entity.
  • v0.50.0: Set mfa_delete = true for S3 buckets. This release was reverted in v0.53.1.
  • v0.51.0: We have refactored all our multi-region modules (the ones that have -multi-region in the name) to no longer create nested provider blocks. Instead, providers must be passed in now via the providers map. This reduces the number of providers that Terraform must instantiate, making the multi-region modules much faster and more stable to use. It also gives you full control over how to authenticate to your various AWS accounts.
  • v0.52.0: We have removed the opt_out_regions input variable from all the multi-region modules (e.g., aws-config-multi-region). We have updated the examples to not include ap-northeast-3 in opt_in_regions. We’ve updated custom-iam-entity with a backward compatibility fix for the changes in #502.
  • v0.53.0: We have verified that this repo is compatible with Terraform 1.0.x, and we are now testing against Terraform 1.0.x.
  • v0.53.1: Updated the default value of mfa_delete to false. This release reverts the change made at v0.50.0.
  • v0.53.2: Added Cloudtrail tags to the Cloudwatch Log Group. Improved docs for MFA Delete (private-s3-bucket).
  • v0.53.3: Introduced enable_all_regions_for_config_aggregator which can be used to configure whether AWS should set the config aggregator to all regions regardless of opt_in_regions.
  • v0.53.4: Updated the aws-config module aggregator functionality to work with the aws_region data source and module depends_on. For details, see #509.
  • v0.53.5: Removed references to the deprecated template provider and replaced with official recommendations.
  • v0.53.6: You can now control if the aws-config module tries to attach IAM policies to the IAM role using the new should_attach_sns_policy input variable.
  • v0.54.0: Remove variable enable_lifecycle_rules , introduced at v0.53.1.
  • v0.55.0: adds the capability for the cloudtrail module to handle multiple data logging resources via a new variable data_logging_resources . This is a backwards incompatible release, please see the release notes for the detailed migration guide.
  • v0.55.1: adds a new input to the custom-iam-entity and iam-users module to allow attaching raw JSON-formatted IAM policies to IAM groups/roles and users, respectively.

terraform-aws-ci

  • v0.37.0: The jenkins module now supports Ubuntu 20.04. Note that starting this release, support for Ubuntu 16.04 is dropped.
  • v0.37.1: Added toggles for backup routines in Jenkins example
  • v0.37.2: infrastructure-deployer now supports AWS SSO and ~/.aws/config.
  • v0.37.3: You can now enable deletion protection for the ALB that is created for the Jenkins server.
  • v0.37.4: You can now filter in or out deleted folders when calling git-updated-folders. This can be used to implement destroy workflows in your pipelines. Refer to the updated documentation for more information about this feature.
  • v0.37.5: Replaced gofmt with goimports in the pre-commit configuration.
  • v0.37.6: Mainly bumped versions in the ecs-deploy-runner Dockerfile so that it installs newer packages.
  • v0.37.7: Added a new module monorepo-helpers that contains scripts that help with adapting CI/CD pipelines for infrastructure code to monorepo setups. These scripts can be used to setup pipelines that only run tests on the infrastructure modules that changed (as opposed to always running all tests on every change). Refer to the module docs for more info.
  • v0.37.8: Updated default terraform-aws-ci version tag used in deploy-runner image to use 0.37.2.
  • v0.38.0: We have verified that this repo is compatible with Terraform 1.0.x, and we are now testing against Terraform 1.0.x.
  • v0.38.1: Gracefully handle error messages for starting the deploy runner task.
  • v0.38.2: Updated build-packer-artifact to be compatible with provider download specifications from packer 1.7. The script will now call packer init if the target template is non-json and the underlying packer version supports init.
  • v0.38.3: ECS Deploy Runner now defaults to installing packer 1.7.4 and terraform-aws-ci version 0.38.2
  • v0.38.4: build-packer-artifact now supports HCL Packer templates. The ecs-deploy-runner Dockerfile has been updated to include hcl2json which is needed by the updated build-packer-artifact.
  • v0.38.5: Support destroy in the CI / CD pipeline. The ecs-deploy-runner-standard-configuration has been updated to support running destroy, plan -destroy, and apply -destroy.
  • v0.38.6: Updates the ecs-deploy-runner Dockerfile to use the correct version of terraform-aws-ci with updated and fixed build scripts.
  • v0.38.7: Updates edrhelpers test assertion with a new expected string value. No functional impact for users.
  • v0.38.8: Bump to latest kubergrunt version in ecs-deploy-runner container.

terraform-aws-zookeeper

  • v0.11.0: Added support for Ubuntu 20.04. Note that starting this release, support for Ubuntu 16.04 is dropped.
  • v0.11.1: Replaced local readonly for local -r in the bash scripts. Add Terraform Validate test. Replace gofmt with goimports .
  • v0.12.0: We have verified that this repo is compatible with Terraform 1.0.x, and we are now testing against Terraform 1.0.x.

terraform-aws-kafka

  • v0.10.0: Added support for Ubuntu 20.04 and Amazon Linux 2. Note that starting this release, support for Ubuntu 16.04 and Amazon Linux 1 is dropped.
  • v0.10.1: Replace go fmt in the pre-commit configuration with goimports
  • v0.11.0: We have verified that this repo is compatible with Terraform 1.0.x, and we are now testing against Terraform 1.0.x.

terraform-aws-elk

  • v0.10.0: Added support for Ubuntu 20.04. Note that starting this release, support for Ubuntu 16.04 is dropped.
  • v0.11.0: We have verified that this repo is compatible with Terraform 1.0.x, and we are now testing against Terraform 1.0.x.

terraform-aws-server

  • v0.12.2: Fixed error message when describing vols by tag; Add retry logic when pulling new interface ID in attach-eni script; Add sleep at end of attach-eni script to give kernel a chance to boot up the newly configured interface.
  • v0.12.3: Added Terraform validate test and replace gofmt with goimports
  • v0.12.4: Added ability to set secondary_private_ips variable.
  • v0.13.0: We have verified that this repo is compatible with Terraform 1.0.x, and we are now testing against Terraform 1.0.x.
  • v0.13.1: You can now configure the single-server module to point the Route 53 DNS record at the private IP of the EIP rather than the public IP by setting the new dns_uses_private_ip variable to true.
  • v0.13.2: Removed references to deprecated template provider.

terraform-aws-asg

  • v0.14.2: Fixed a bug where the IAM permissions were not being attached before the ASG was created.
  • v0.15.0: We have verified that this repo is compatible with Terraform 1.0.x, and we are now testing against Terraform 1.0.x.
  • v0.15.1: Removed references to deprecated template provider and replaced with official replacements.

terraform-aws-monitoring

  • v0.28.0: Fixed a bug in agents/cloudwatch-agent module where the metrics were not being reported under the InstanceId dimension. The logs/cloudwatch-log-aggregation-scripts and metrics/cloudwatch-memory-disk-metrics-scripts modules have been removed, as they are now functionally replaced by agents/cloudwatch-agent. Refer to the release notes for migration information.
  • v0.28.1: Fixed a wrong error message in configure-cloudwatch-agent.sh .
  • v0.29.0: The CloudWatch Agent is now configured to report disk usage percent and memory usage percent metrics; The EC2 and ASG alarms have been adjusted to be consistent cloudwatch-agent. Refer to the migration guide for more info on retaining backward compatibility with the old scripts.
  • v0.29.1: You can now disable metrics reporting using the new --disable-cpu-metrics, --disable-mem-metrics, and --disable-disk-metrics args of the configure-cloudwatch-agent.sh script.
  • v0.29.2: The elasticache-redis-alarms module now supports several additional alarms: Engine CPU usage, cluster swap usage, cluster memory usage, connection count, replication lag.
  • v0.29.3: Replaced gofmt with goimports
  • v0.30.0: We have verified that this repo is compatible with Terraform 1.0.x, and we are now testing against Terraform 1.0.x.
  • v0.30.1: Removed references to deprecated template provider and replaced with official Hashicorp alternatives.
  • v0.30.2: Fix copy/paste error in the curr_connections and replication-lag alarm names in elasticache-redis-alarms.

terraform-aws-data-storage

  • v0.20.1: You can now configure timeouts in the rds module using the new creating_timeout, updating_timeout, and deleting_timeout input variables.
  • v0.20.2: The apply_immediately flag now propagates to the replica instances for the rds module. Previously it was only being set on the leader instance.
  • v0.20.3: Replace go fmt in the pre-commit configuration file with goimports
  • v0.20.4: Added ability to set backup_retention_period on RDS read replicas via the replica_backup_retention_period input variable.
  • v0.20.5: You can now allow major version upgrades on the aurora module by setting the new allow_major_version_upgrade input variable.
  • v0.21.0: We have verified that this repo is compatible with Terraform 1.0.x, and we are now testing against Terraform 1.0.x.
  • v0.21.1: Removed references to template provider and replaced with official alternatives.

terraform-aws-vpc

  • v0.15.5: Updated the vpc-flow-logs module to add the necessary IAM permissions to allow the VPC flow logs service to write to the S3 bucket.
  • v0.15.6: Added Terraform validate test and replace gofmt with goimports.
  • v0.15.7: Added the ability to specify availability zones to use for the subnets directly to vpc-app module; Added the ability to configure timeouts on route resources; Minor internal changes to vpc-app implementation which will have no effect on existing infrastructure.
  • v0.16.0: Fixed bug with configuring default NACLs, where default NACLs were applied and configured even when apply_default_nacl_rules was false. Now the default NACLs are only touched and updated if apply_default_nacl_rules is true.
  • v0.17.0: We have verified that this repo is compatible with Terraform 1.0.x, and we are now testing against Terraform 1.0.x.
  • v0.17.1: Added support for EBS and Lambda interface endpoints. Removed usage of the deprecated template provider.
  • v0.17.2: The vpc-interface-endpoint module can now automatically create a security group that allows HTTPS ingress to the endpoints from your VPC if you set create_https_security_group to true.
  • v0.17.4: Add support for EFS interface endpoints to the vpc-interface-endpoint module.

terraform-aws-ecs

  • v0.29.2: Added terraform validate test and replace gofmt with goimports
  • v0.29.3: Made load_balancing_algorithm_type configurable in ELB target groups.
  • v0.30.0: We have verified that this repo is compatible with Terraform 1.0.x, and we are now testing against Terraform 1.0.x.
  • v0.30.1: Added ability to configure associate_public_ip_address in the Launch Configuration used to manage the ASG for the ECS cluster.
  • v0.30.2: You can now enable the ECS “circuit breaker” feature via the new deployment_circuit_breaker input variable.
  • v0.30.3: Removed references to the deprecated template provider and replaced with official recommendation.
  • v0.30.4: Updated ecs-service to expose proxy_configuration subblock for the aws_ecs_task_definition resource to support App Mesh.
  • v0.31.0: Added support for protocol_version in the ECS service target group.
  • v0.31.1: The configure-ecs-instance script no longer calls docker login for ECS, as the ECS Agent does that automatically nowadays.https://github.com/gruntwork-io/terraform-aws-ecs/releases/tag/v0.31.2:
  • v0.31.2: Updated configure-ecs-instance script to handle setting arbitrary ECS configurations. You can now set arbitrary configuration values using the --custom-config option.

terraform-aws-load-balancer

  • v0.26.2: Fixed a timeout bug when acm-tls-certificates with Subject Alternative Names are created with verification records.
  • v0.26.3: Fixed a bug that was introduced in v0.26.2, where the domain validation options filter was case sensitive, when the domain names are not.
  • v0.27.0: We have verified that this repo is compatible with Terraform 1.0.x, and we are now testing against Terraform 1.0.x.
  • v0.27.1: Examples have been updated to not use the deprecated template provider. No changes to modules.

terraform-aws-sam

  • v0.5.1: Added terraform validate test and replace gofmt with goimports
  • v0.6.0: We have verified that this repo is compatible with Terraform 1.0.x, and we are now testing against Terraform 1.0.x.

terraform-aws-cache

  • v0.15.1: Added terraform validate test and replace gofmt with goimports
  • v0.16.0: We have verified that this repo is compatible with Terraform 1.0.x, and we are now testing against Terraform 1.0.x.
  • v0.16.1: Removed references to deprecated template provider

terraform-aws-messaging

  • v0.6.2: You can now configure deduplication and FIFO throughput limits on the sqs module using the new deduplication_scope and fifo_throughput_limit input variables, respectively.
  • v0.7.0: We have verified that this repo is compatible with Terraform 1.0.x, and we are now testing against Terraform 1.0.x.
  • v0.7.1: You can now configure consumer deletion in the kinesis module using the new enforce_consumer_deletion input variable.
  • v0.7.2: Removed references to the deprecated template provider

terraform-aws-lambda

  • v0.12.0: Added new modules for configuring API Gateway. Refer to the release notes for more info.
  • v0.13.0: We have verified that this repo is compatible with Terraform 1.0.x, and we are now testing against Terraform 1.0.x.
  • v0.13.1: Removed usage of the template provider which is now deprecated.
  • v0.13.2: You can now have the lambda module use an existing IAM role, rather than creating a new one, by passing in the IAM role's ARN via the new existing_role_arn input variable.
  • v0.13.3: Add a required_providers block to the lambda-edge module so you can pass in a custom provider and not get warnings in Terraform 0.15 and above.

terraform-aws-openvpn

  • v0.16.0: We have verified that this repo is compatible with Terraform 1.0.x, and we are now testing against Terraform 1.0.x.
  • v0.16.1: Removed references to deprecated template provider in examples (no changes to underlying modules).

terraform-aws-static-assets

  • v0.11.0: We have verified that this repo is compatible with Terraform 1.0.x, and we are now testing against Terraform 1.0.x.
  • v0.12.1: You can now disable access logging in the s3-cloudfront module using the new disable_logging input variable. This is useful in regions where CloudFront access logging isn't supported.

DevOps News

EKS improvements: Kubernetes 1.21, VPC CNI limits, upgrading node groups in parallel, and EKS anywhere

What happened: Amazon has announced four improvements to EKS, their managed Kubernetes service:

  1. EKS now supports Kubernetes 1.21.
  2. The VPC CNI plugin has increased the limits on the number of pods you can run per node.
  3. Managed node groups can now upgrade nodes in parallel.
  4. EKS Anywhere allows you to run EKS in your own data center.

Why it matters: Here is why these four improvements matter:

  1. Kubernetes 1.21: This release includes Cronjobs and Immutable Secrets and ConfigMaps reaching stable status, and Graceful Node Shutdown graduating to beta.
  2. VPC CNI plugin: One of the problems with the VPC CNI plugin is that it assigns an IP address for every Pod you run, but the number of IPs any given EC2 instance can have is limited, so you’d often hit these limits long before the instance ran out of other resources (e.g., CPU, memory). The new VPC CNI plugin allows you to work around these limits on AWS Nitro based EC2 instances.
  3. Managed node groups: Before, you could only upgrade one node at a time, so rolling out upgrades would take a long time. Being able to run upgrades in parallel will make upgrades go much faster.
  4. EKS anywhere: You can now use EKS to deploy and manage Kubernetes clusters in your own data centers, as well as in AWS.

What to do about it: See the Kubernetes 1.21 release notes, VPC CNI plugin announcement, managed node groups announcement, and EKS Anywhere page for all the details.

Amazon Managed Grafana is now generally available

What happened: Amazon has announced that it’s Managed Grafana service is now generally available to all users.

Why it matters: Grafana is a popular open source visualization and querying platform for metrics, logs, and traces. In the past, if you wanted to use it, you’d have to run it and maintain it yourself. With Amazon Managed Grafana, you get a fully managed service to do all the heavy lifting for you.

What to do about it: Give Amazon Managed Grafana a shot and let us know what you think and if we should have first class support for it at Gruntwork!

Amazon Elasticsearch is now Amazon OpenSearch

What happened: Amazon has announced that their managed Elasticsearch service is now called OpenSearch and will support Elasticsearch up to version 7.10 and OpenSearch 1.0 and beyond.

Why it matters: As announced previously, Amazon forked the Elasticsearch project into a new project called OpenSearch. Now, they have taken their managed Elasticsearch service, and updated it to run OpenSearch as well.

What to do about it: Depending on whether you believe Amazon forking Elasticsearch is a good thing or a bad thing, this announcement will either make you excited or you’ll need to find an alternative managed Elasticsearch service (such as elastic.co). Read the announcement for more details.

Security Updates

Below is a list of critical security updates that may impact your services. We notify Gruntwork customers of these vulnerabilities as soon as we know of them via the Gruntwork Security Alerts mailing list. It is up to you to scan this list and decide which of these apply and what to do about them, but most of these are severe vulnerabilities, and we recommend patching them ASAP.

OpenSSL

Two high-severity vulnerabilities were found and fixed in OpenSSL. Some more resources on the vulnerabilities below can be found here (Security Week summary) & here (analysis by Sophos).

1. SM2 Decryption Buffer Overflow (CVE-2021–3711) (High severity)

The first vulnerability is SM2 Decryption Buffer Overflow (CVE-2021–3711). This vulnerability means that when the SM2 data is being decrypted, one of the API calls incorrectly calculates the size of the data it expects back to be bigger, which leaves the space for buffer overflow. This could open the opportunity to a malicious actor presenting valid SM2 data for decryption and overflow the buffer, which in its turn could cause the change of behaviour for the application leading it to crash.

What’s SM2? In short, SM2 is an algorithm that supports signature and encryption functionality. Read more here.

Who’s affected? The affected versions of OpenSSL are everything below and including OpenSSL versions 1.1.1k.

What’s the fix? Upgrade to OpenSSL 1.1.1l. OpenSSL 1.0.2 is not impacted, and OpenSSL 3.0 alpha/beta releases will be updated before the final release.

2. Read buffer overruns processing ASN.1 strings (CVE-2021–3712) (Moderate severity)

This vulnerability allows for ASN.1 strings to be presented in a different format (still a string structure, but not of the specific ASN1_STRING type). The ASN1_STRING structure indicates its data and expected buffer size with two fields, and when it terminates it sets the value to NUL. However, when this string is constructed directly, it doesn’t check if the values for those fields have been set, and this can result in the processor thinking the buffer is longer in size, and thus keep reading the buffer. If exploited, this could lead to a service outage (denial of service) or data leakage (through memory leakage) of private data such as keys, certificates, other plaintext private data, etc.

What’s ASN.1? ASN.1 is a data notation standard that allows end devices to agree on the encryption and data format of how messages will be sent over the communication protocols.

What is buffer over-read? When an application reads a memory buffer but doesn’t respect the boundary and thus starts reading into another memory’s segment boundaries. See here for more details.

What this means in real life? In the vulnerability description and announcement, they’ve given a great example where this vulnerability is shown to affect the X509 certificate functions.

Who’s affected? All versions below and including OpenSSL versions 1.1.1k & OpenSSL 1.0.2 users, as well as other OpenSSL users.

What’s the fix: OpenSSL users of 1.1.1k should upgrade to OpenSSL 1.1.1l. There’s support only for the premium customers of OpenSSL 1.0.2 — they should upgrade to 1.0.2za. All other users should upgrade to 1.1.1l.