Improvements to Pipelines Change Detection

We're excited to introduce significant improvements to how Gruntwork Pipelines detects and processes changes in your infrastructure code. These enhancements make Pipelines more predictable and responsive to the full range of modifications in your codebase.

We've added several opt-in feature flags that you can enable by setting environment variables in your Pipelines configuration. You can enable these independently, or set PIPELINES_FEATURE_EXPERIMENT_ALL to true to enable the full set of functionality described below.

We expect to enable these behaviors by default in a future major version of Pipelines. Check out our feature flags documentation for more details on how to implement these improvements in your workflow in the meantime.

Improved Unit File Change Detection

Feature Flag: PIPELINES_FEATURE_EXPERIMENT_COLOCATED_FILE_UNIT_CHANGE_DETECTION

This flag enhances how Pipelines detects changes within Terragrunt Units (any non-root directory of the repo that contains a terragrunt.hcl file) to better align with our semantic notion of a unit.

When enabled, any file modified within a Terragrunt Unit will trigger a ModuleChanged event for that unit, causing Pipelines to run a Terragrunt Plan/Apply operation for the affected directory. This includes files like terragrunt.lock.hcl and other support files within the unit. Previously, these would trigger a FileChanged event, which disregarded the semantic connection to the unit being affected.

For example, consider a unit containing these files:

tags.yml
dir1/terragrunt.hcl
dir1/myvars.json

With this new flag enabled:

  • A change to dir1/myvars.json is detected as a ModuleChanged event for dir1, triggering a Terragrunt Plan/Apply for this directory.
  • A change to tags.yml is still detected as a FileChanged event, causing Pipelines to run Terragrunt with queue-include-units-reading for this file.

Smarter Change Consolidation

Feature Flag: PIPELINES_FEATURE_EXPERIMENT_AGGRESSIVE_CONSOLIDATION

This flag improves how Pipelines consolidates and executes Terragrunt operations to improve dependency handling and minimize redundant change execution. When enabled, Pipelines will consolidate as many Terragrunt plan/apply changes as possible into a single run --all job. This reduces duplicated work when, for example, both FileChanged and EnvCommon events would trigger plans for the same unit.

Minimizing Blast Radius

Feature Flag: PIPELINES_FEATURE_EXPERIMENT_MINIMIZE_BLAST_RADIUS

Safety is a core principle of Gruntwork Pipelines. In that spirit, we’re making improvements to ensure that the blast radius of your changes remain as localized as possible by default. With this flag enabled, Pipelines will execute Terragrunt run --all with the --queue-strict-include option, which helps reduce the blast radius of per-unit changes as dependencies will no longer run automatically when dependents are modified. This ensures that run --all behaviors are consistent with runs for individual units.