The Problem: State Tied to File Paths

Terragrunt's default behavior links remote state keys directly to the file path of the Terraform module. This design choice simplifies initial setup but creates a significant hurdle when reorganizing repositories. Renaming a folder, restructuring project groups, or correcting early naming mistakes typically forces a state migration, a process that is often complex, error-prone, and time-consuming. This is a common pain point for teams managing numerous Terragrunt-managed infrastructure components.

For instance, a typical Terragrunt configuration might look like this:

# root.hcl
remote_state {
  backend = "s3"
  generate = {
    path      = "backend.tf"
    if_exists = "overwrite"
  }
}

In this setup, the remote state location, often an S3 bucket, is dynamically generated. The key within that state file is derived from the directory structure. If you have a project at infrastructure/production/webserver, its state might be stored under a key like infrastructure/production/webserver/terraform.tfstate. If you decide to move this project to infrastructure/staging/webserver, Terragrunt, by default, would expect the state to be in the new location, effectively losing track of the existing state. This forces a manual intervention, usually involving copying the state file to the new location and updating Terragrunt's configuration to point to it. This is not ideal for maintaining a clean and automated infrastructure-as-code workflow.

The Solution: Explicit State Key Configuration

The key to avoiding state migration when renaming directories lies in explicitly defining the remote state key. Instead of letting Terragrunt infer the key from the file path, you can hardcode it. This decouples the state’s identity from its physical location in the repository.

The solution involves modifying the remote_state block within your terragrunt.hcl files. Specifically, you need to add a config attribute that specifies the exact key for your remote state. This attribute allows you to provide a custom configuration for the backend. For S3, this means defining the key parameter within the backend configuration.

Consider the following updated configuration:

# terragrunt.hcl
remote_state {
  backend = "s3"
  generate = {
    path      = "backend.tf"
    if_exists = "overwrite"
  }
  config = {
    key = "my-custom-state-key/terraform.tfstate"
  }
}

In this example, the remote state will always be stored under the key my-custom-state-key/terraform.tfstate, regardless of where the terragrunt.hcl file resides in the directory structure. This is a powerful technique for refactoring.

Implementing the Change

To apply this change across your projects:

  1. Identify Projects to Refactor: Determine which directories you intend to rename or move.
  2. Update terragrunt.hcl: For each project, locate its terragrunt.hcl file. Add or modify the remote_state block to include the config with a fixed key. Ensure the key is unique per project.
  3. Commit Changes: Commit the updated terragrunt.hcl files.
  4. Rename Directories: Now, you can safely rename or move the directories containing these projects. Terragrunt will continue to look for the state file at the explicitly defined key, not the old path.
  5. Run terragrunt plan: Execute terragrunt plan in the new directory location. Terragrunt will find the existing state at the specified key and show no differences related to state location.

This approach effectively treats the remote state key as an immutable identifier for your infrastructure component, rather than a dynamic pointer tied to its filesystem location. This is analogous to how unique resource IDs work in cloud environments – the ID persists even if you move the resource conceptually within your management tools.

Broader Implications and Best Practices

While this method solves the immediate problem of refactoring without state migration, it's essential to consider its implications. Hardcoding state keys means that the key itself becomes a critical piece of information that must be managed. If you need to move the state to a completely different backend or rename the key for organizational reasons, you will still need to perform a state migration, albeit a more controlled one.

Best Practices:

  • Centralized Configuration: For consistency, consider managing these explicit state key definitions in a common configuration file or module that your individual project terragrunt.hcl files can inherit from.
  • Naming Conventions: Establish clear and consistent naming conventions for your state keys. This aids in organization and prevents accidental key collisions.
  • Documentation: Clearly document the chosen state key strategy and any refactoring steps taken.
  • Testing: Always run terragrunt plan after renaming directories to verify that Terragrunt correctly identifies the existing state.

This technique empowers infrastructure teams to maintain a cleaner, more organized repository structure without the disruptive overhead of manual state management. It shifts the responsibility of state key management from the filesystem to explicit configuration, providing greater flexibility in how infrastructure is organized and refactored over time.