terragrunt unable to get the outputs from dependency

[srinathjamboju92]

I'm seeing a problem with terragrunt not being able to get the outputs from modules that needs to be passed as inputs to dependent modules.

Below is the list of resources that are being attempted to create. Please note that this problem happens when terragrunt tries to create both parent and dependent resources for the first time.

Parent Resource:

• Module source: tfr:///terraform-aws-modules/vpc/aws//.?version=5.0.0

Dependent Resource:

• Module source: tfr:///terraform-aws-modules/eks/aws//.?version=19.14.0

I've below on the EKS Module:

dependencies {
  paths = ["../vpc"]
}
dependency "vpc" {
  config_path                             = "../vpc"
  mock_outputs_allowed_terraform_commands = ["validate", "init", "plan", "apply"]
  mock_outputs                            = {
    vpc_id          = "vpc-123456"
    private_subnets = ["subnet-123456", "subnet-123456", "subnet-123456"]
  }
  mock_outputs_merge_strategy_with_state  = "shallow"
}

inputs = {
 vpc_id                           = dependency.vpc.outputs.vpc_id
 subnet_ids                  = dependency.vpc.outputs.private_subnets
}

Now when I run terragrunt run-all apply, I get the below Groups to be applied.

Group 1

• test/vpc

Group2

• test/eks_cluster

And when confirmed "y" to apply, terragrunt ran inside the test/vpc directory and created the VPC resources and did output the resource attributes such as vpc_id, private_subnets and others.

Now when terragrunt runs inside eks_cluster, it never gets the outputs from the parent module i.e., test/vpc. It always gets the values from mock_outputs while apply is in progress.

I think the issue here is, terragrunt before plans to apply, it gets the outputs from the parent modules and outputs as a json in the dependent(eks in this case) folder and use it to parse the outputs?

If this is the case, run-all apply will never work across the modules, that both parent(eks) and eks gets deployed at same time. This will be a major problem. It's not just VPC that is always a dependency to others. At least in my case, there are quite a few dependency modules, and I need to run terragrunt apply in each parent folder before applying in other sub-folders.

Is there any work-around for this? I already ready most of the GitHub pages and the below links, none of them could solve the problem of run-all apply at first time when terraform state is empty.

This link explains about unapplied dependency, but doesn't explain that it can or cannot pass outputs to dependent modules while applying for the first time.

• https://terragrunt.gruntwork.io/docs/features/execute-terraform-commands-on-multiple-modules-at-once/#dependencies-between-modules

The comment here explicitly says, when run-all apply is made, outputs from vpc is written to state and they are used for dependent modules at the same time, but that never worked for me with above config.

• CI/CD Best Practice for plan-all output? terragrunt#720 (comment)

There is one more workaround, this is again for run-all plan, never expected to be used for apply, as I need real outputs for apply.

• dependency mock_outputs workflow for output changes terragrunt#940 (comment)

NOTE: If I remove apply I know that the mock_outputs are not considered for run-all apply or apply commands, but when did so, I get the error that the outputs are not present in the parent module. And when apply is used, terragrunt never looks at state to get the outputs generated in the dependent module to use instead.

terragrunt version: v0.50.16
terraform version: v1.5.5

---

Tracked in ticket #110448

Below is the error log from the terminal output:

test/eks_cluster has finished with an error: Cannot process module Module test/eks_cluster (excluded: false, assume applied: false, dependencies: [test/vpc]) because one of its dependencies, finished with an error: test/vpc/terragrunt.hcl is a dependency of test/eks_cluster/terragrunt.hcl but detected no outputs. Either the target module has not been applied yet, or the module has no outputs. If this is expected, set the skip_outputs flag to true on the dependency block.  prefix=[test/eks_cluster] 

[denis256]

Hello,
if will be applied vpc module first, the issue will continue to appear?

[srinathjamboju92]

there are 2 answers for your question @denis256 .
Case 1:
If I do run-all apply,
even with vpc created first, and the outputs are generated into state file, eks module does not consume them. Instead eks still shows the mock outputs. Now if I open an another terminal and use terragrunt apply inside eks, (remember that run-all apply still waiting for me to apply eks on another terminal), then the outputs are consumed. This means, the outputs from across the modules are fetched at the initial phase, before applying the changes and that outputs are used, and obviously empty, in case of creating the infra for the very first time.

Case 2:
If I do terragrunt apply inside vpc module first,
And then inside eks, then the real outputs are consumed from vpc.

[denis256]

Will be helpful to share an example repo where the issue happens, so far my tests return real output values
https://github.com/denis256/terragrunt-tests/tree/master/issue-2714

[wakeful]

hey @srinathjamboju92 did you try to run the above without the dependencies block? just with dependency instead both of them? e.q.

dependency "vpc" {
...

maybe there is a race condition and terragrunt is not requesting outputs as the dependencies block does not run terraform output after the apply.