Repository-side configuration should extend/augment the server side one

[Ninir]

As per the documentation:

Once an atlantis.yaml file exists in a repo, Atlantis won’t try to determine where to run plan automatically. Instead it will just follow the project configuration. This means that you’ll need to define each project in your repo.

If you have many directories with Terraform configuration, each directory will need to be defined.

For instance, supposing we have 3 folder:

• A with default workspace
• B with dev and prod workspaces
• C with test workspace

and the following repo side config:

version: 3

projects:
- name: B-dev
  dir: B
  workspace: dev
  workflow: dev

- name: B-prod
  dir: B
  workspace: prod
  workflow: prod

- name: C-test
  dir: C
  workspace: test
  workflow: dev

workflows:
  dev:
    plan:
      steps:
      - init
      - plan:
          extra_args: ["-var-file", "terraform.tfvars"]

  prod:
    plan:
      steps:
      - init
      - plan:
          extra_args: ["-var-file", "terraform.prod.tfvars"]

I would expect a plan for dir A that would be the default workflow, 2 plans for B as per the config, and one for C as per the config also.

It is not obvious to me, reading the RFC, why it should one or the other?

I feel we could keep the directory discovery as default, and the repository configuration file could extend that.

[lkysow]

This would break backwards compatibility because all directories users make changes to would now cause Atlantis to autoplan whereas previously they wouldn't unless listed in atlantis.yaml.

I didn't actually think of changing the behaviour when implementing server-side workflows, but thinking about it now, I don't think I would change it to keep backwards compatibility.

[Ninir]

Say you have dozens of directories to handle and you are using workspaces, you now need to have X * Y entries in the atlantis.yaml file, where X is the number of projects and Y the number of workspaces at least per directory.

Talking about backward compatibility, how about bumping the API version to 4 and allow it / add a new config like extend_default_config: true?

[lkysow]

I'm not sure I understand. If you have workspaces then you need to tell Atlantis their names for each directory. Having the default workflow used for non-specified directories wouldn't solve the problem of needing config for each directory that does use workspaces.

[Ninir]

Let me reword: you have 100 directories. 50 are using the default workflow, 50 are not. It means I have to define 100 entries at least, with additional ones for workspaces I need.

That means the file starts becoming unreadable, while, if we specify the directories that use workspaces only, it's less entries and way more readable.

Is your first message stating there is no way to change this behavior and it's either one defines everything or nothing?

[Ninir]

@lkysow For the record, I've been creating a atlantis.yaml file that contains all of the entries, and everything plans perfectly.

I was thinking, what about allowing glob patterns in name/directories?

...

# all of the children
- name: children
  dir: apps/*

# all of the sub-children
- name: sub-children
  dir: apps/**/*

For instance, the first case could be matching apps/myapp1 and apps/myapp2 while case 2 would be matching apps/project/subapp1 and apps/project/subapp2

(sorry If it was discussed already and I missed the conversation)

[lkysow]

Hey sorry I didn't respond to your last, must have missed it. I think what we really need is #500. The globbing solution might work for your specific case but if anyone has directories inside those dirs they don't want Atlantis to respond to then they can't use it.

I think a mechanism for dynamically generating the atlantis.yaml is the most flexible and long term solution.

[Ninir]

No worries.
That make sense, will follow this topic closely :)

Closing!