terragrunt and policy checking not working

[connaryscott]

This policy set in /etc/atlantis/repos.yaml

policies:
  owners:
    users:
      - cscott
  policy_sets:
    - name: terraform_version
      path: policies/terraform_version
      source: local

appears to be failing since a terraform show w/o knowledge of the backend fails. We are using terragrunt.

In the log we see:

"terraform/terraform_client.go:277","msg":"running \"/usr/local/bin/terraform show -no-color -json /atlantis-data/repos/qadium/shared-infra-automation-staging/9/default/aws-ent/qadium-dev/s3/exp-staging-test-bucket3/default.tfplan

The specific error is not shown in atlantis log however I believe it's because we need terragrunt to run the show sub command, not terraform:

$  kubectl -n atlantis-staging logs atlantis-staging-0 |tail -1 |jq '.'
{
  "level": "error",
  "ts": 1620341815.7039707,
  "caller": "terraform/terraform_client.go:277",
  "msg": "running \"/usr/local/bin/terraform show -no-color -json /atlantis-data/repos/qadium/shared-infra-automation-staging/9/default/aws-ent/qadium-dev/s3/exp-staging-test-bucket3/default.tfplan\" in \"/atlantis-data/repos/qadium/shared-infra-automation-staging/9/default/aws-ent/qadium-dev/s3/exp-staging-test-bucket3\": exit status 1",
  "json": {
    "repo": "qadium/shared-infra-automation-staging",
    "pull": "9"
  },
  "stacktrace": "github.com/runatlantis/atlantis/server/events/terraform.(*DefaultClient).RunCommandWithVersion\n\t/home/circleci/project/server/events/terraform/terraform_client.go:277\ngithub.com/runatlantis/atlantis/server/events/runtime.(*ShowStepRunner).Run\n\t/home/circleci/project/server/events/runtime/show_step_runner.go:42\ngithub.com/runatlantis/atlantis/server/events/runtime.(*PlanTypeStepRunnerDelegate).Run\n\t/home/circleci/project/server/events/runtime/plan_type_step_runner_delegate.go:64\ngithub.com/runatlantis/atlantis/server/events/runtime.(*MinimumVersionStepRunnerDelegate).Run\n\t/home/circleci/project/server/events/runtime/minimum_version_step_runner_delegate.go:43\ngithub.com/runatlantis/atlantis/server/events.(*DefaultProjectCommandRunner).runSteps\n\t/home/circleci/project/server/events/project_command_runner.go:381\ngithub.com/runatlantis/atlantis/server/events.(*DefaultProjectCommandRunner).doPolicyCheck\n\t/home/circleci/project/server/events/project_command_runner.go:246\ngithub.com/runatlantis/atlantis/server/events.(*DefaultProjectCommandRunner).PolicyCheck\n\t/home/circleci/project/server/events/project_command_runner.go:145\ngithub.com/runatlantis/atlantis/server/events.runProjectCmds\n\t/home/circleci/project/server/events/project_command_pool_executor.go:47\ngithub.com/runatlantis/atlantis/server/events.(*PolicyCheckCommandRunner).Run\n\t/home/circleci/project/server/events/policy_check_command_runner.go:59\ngithub.com/runatlantis/atlantis/server/events.(*PlanCommandRunner).runAutoplan\n\t/home/circleci/project/server/events/plan_command_runner.go:144\ngithub.com/runatlantis/atlantis/server/events.(*PlanCommandRunner).Run\n\t/home/circleci/project/server/events/plan_command_runner.go:221\ngithub.com/runatlantis/atlantis/server/events.(*DefaultCommandRunner).RunAutoplanCommand\n\t/home/circleci/project/server/events/command_runner.go:160"
}

which will fail due to our terragrunt abstraction of the terraform backend.

I believe the following occurs:

terraform show -json default.tfplan
Backend reinitialization required. Please run "terraform init".
Reason: Initial configuration of the requested backend "s3"
...
Error: Initialization required. Please see the error message above.

bash-5.0$ echo $?
1

When running terragrunt, this issue does not occur:

$ terragrunt show -json default.tfplan   |jq '.'  |head
{
  "format_version": "0.1",
  "terraform_version": "0.13.3",
  "variables": {
    "acl": {
      "value": "private"
    },
    "attach_policy": {
      "value": "true"
    },

Is there a way to run policy checking when using terragrunt or are we going to need to shell out to a script to do this?

[joshmyers]

Also hitting this, exact same issue.

[mhennecke]

The following workflow step (here as json) should resolve the issue:

{
  "run": "terragrunt show -no-color -json \"$PLANFILE\" > \"${PLANFILE/%.tfplan/.json}\""
}

Just add the step after your terragrunt apply. Beware, that the show filename must contain the project name, if you are using them in your config. Refer to GetShowResultFileName()

@lkysow Maybe adding the result file name to the run step environment, e.g. as SHOWFILE or SHOWRESULT, would simplify the terragrunt use case!?

[joshmyers]

@mhennecke How does #1611 close this issue? This issue is about getting the in built policy checking working when using something like Terragrunt in a custom workflow. You can already shell out for conftest if things are in a custom workflow:

- run: terragrunt show -json \$PLANFILE >tfplan.json
- run: ../../../../../scripts/conftest tfplan.json

[mhennecke]

Sure, this will work as well. But instead of running conftest with a run step, you could use the default policy_check as your terragrunt show step puts the json plan in the correct place already (where policy_checkassumes it to be). The PR just adds a new environment variable for the expected json plan file location. Note: It must contain the atlantis project name if you are working with those in your config.

So even without #1611, the following should work for you:

policy_check:
      steps:
        - run: terragrunt show -no-color -json "$PLANFILE" > "${PLANFILE/%.tfplan/.json}
        - policy_check