Merge pull request #14 from runatlantis/comment-syntax

Add -w workspace and -d directory flags to plan/apply comments

Author: lkysow

README.md

@@ -45,13 +45,13 @@ Read about [Why We Built Atlantis](https://www.atlantis.run/blog/atlantis-releas
 - Optionally, require a **review and approval** prior to running `apply`
 
 ➜ Also
-- No more **copy-pasted code across workspaces/environments**. Atlantis supports using an `env/{env}.tfvars` file per workspace/environment so you can write your base configuration once
 - Support **multiple versions of Terraform** with a simple project config file
 
 ## Atlantis Works With
 * GitHub (public, private or enterprise) and GitLab (public, private or enterprise)
 * Any Terraform version (see [Terraform Versions](#terraform-version))
 * Can be run with a [single binary](https://github.com/runatlantis/atlantis/releases) or with our [Docker image](https://hub.docker.com/r/runatlantis/atlantis/)
+* Any repository structure
 
 ## Getting Started
 Download from [https://github.com/runatlantis/atlantis/releases](https://github.com/runatlantis/atlantis/releases)
@@ -71,15 +71,42 @@ If you're ready to permanently set up Atlantis see [Production-Ready Deployment]
 ## Pull/Merge Request Commands
 Atlantis currently supports three commands that can be run via pull request comments (or merge request comments on GitLab):
 
+![Help Command](./docs/pr-comment-help.png)
 #### `atlantis help`
 View help
 
-#### `atlantis plan [workspace]`
-Runs `terraform plan` for the changes in this pull request. If `[workspace]` is specified, will switch to that workspace, before running `plan`. Any additional arguments passed to `atlantis plan` will be passed on to `terraform plan`. For example if you'd like to run `terraform plan -target={target}` then you can comment `atlantis plan -target={target}`.
+---
+![Plan Command](./docs/pr-comment-plan.png)
+#### `atlantis plan [options] -- [terraform plan flags]`
+Runs `terraform plan` for the changes in this pull request.
+
+Options:
+* `-d directory` Which directory to run plan in relative to root of repo. Use '.' for root. If not specified, will attempt to run plan for all Terraform projects we think were modified in this changeset.
+* -w workspace` Switch to this [Terraform workspace](https://www.terraform.io/docs/state/workspaces.html) before planning. Defaults to 'default'. If not using Terraform workspaces you can ignore this.
+* `--verbose` Append Atlantis log to comment.
+
+Additional Terraform flags:
+
+If you need to run `terraform plan` with additional arguments, like `-target=resource` or `-var 'foo-bar'`
+you can append them to the end of the comment after `--`, ex.
+```
+atlantis plan -d dir -- -var 'foo=bar'
+```
+If you always need to append a certain flag, see [Project-Specific Customization](#project-specific-customization).
+
+---
+![Apply Command](./docs/pr-comment-apply.png)
+#### `atlantis apply [options] -- [terraform apply flags]`
+Runs `terraform plan` for the changes in this pull request.
+
+Options:
+* `-d directory` Apply the plan for this directory, relative to root of repo. Use '.' for root. If not specified, will run apply against all plans created for this workspace.
+* -w workspace` Apply the plan for this [Terraform workspace](https://www.terraform.io/docs/state/workspaces.html). Defaults to 'default'. If not using Terraform workspaces you can ignore this.
+* `--verbose` Append Atlantis log to comment.
+
+Additional Terraform flags:
 
-#### `atlantis apply [workspace]`
-Runs `terraform apply` for the plan generated by `atlantis plan`. If `[workspace]` is specified, will switch to that workspace.
-Any additional arguments passed to `atlantis apply` will be passed on to `terraform apply`.
+Same as with `atlantis plan`.
 
 ## Project Structure
 Atlantis supports several Terraform project structures:
@@ -131,23 +158,24 @@ or
     │   └── staging.tfvars
     └── main.tf
 ```
-With the above project structure you can de-duplicate your Terraform code between workspaces/environments without requiring extensive use of modules. At Hootsuite we've found this project format to be very successful and use it in all of our 100+ Terraform repositories.
+With the above project structure you can de-duplicate your Terraform code between workspaces/environments without requiring extensive use of modules. At Hootsuite we found this project format to be very successful and use it in all of our 100+ Terraform repositories.
 
 ## Workspaces/Environments
 Terraform introduced [Workspaces](https://www.terraform.io/docs/state/workspaces.html) in 0.9. They allow for
 > a single directory of Terraform configuration to be used to manage multiple distinct sets of infrastructure resources
 
-If you're using a Terraform version >= 0.9.0, Atlantis supports workspaces through an additional argument to the `atlantis plan` and `atlantis apply` commands.
+If you're using a Terraform version >= 0.9.0, Atlantis supports workspaces through the `-w` flag.
 For example,
 ```
-atlantis plan staging
+atlantis plan -w staging
 ```
 
 If a workspace is specified, Atlantis will use `terraform workspace select {workspace}` prior to running `terraform plan` or `terraform apply`.
 
 If you're using the `env/{env}.tfvars` [project structure](#project-structure) we will also append `-tfvars=env/{env}.tfvars` to `plan` and `apply`.
 
-If no workspace is specified, terraform will use the `default` workspace by default.
+If no workspace is specified, we'll use the `default` workspace by default.
+This replicates Terraform's default behaviour which also uses the `default` workspace.
 
 ## Terraform Versions
 By default, Atlantis will use the `terraform` executable that is in its path. To use a specific version of Terraform just install that version on the server that Atlantis is running on.
@@ -209,13 +237,13 @@ extra_arguments:
 ```
 
 When running the `pre_plan`, `post_plan`, `pre_apply`, and `post_apply` commands the following environment variables are available
-- `WORKSPACE`: if a workspace argument is supplied to `atlantis plan` or `atlantis apply`, ex `atlantis plan staging`, this will
+- `WORKSPACE`: if a workspace argument is supplied to `atlantis plan` or `atlantis apply`, ex `atlantis plan -w staging`, this will
 be the value of that argument. Else it will be `default`
 - `ATLANTIS_TERRAFORM_VERSION`: local version of `terraform` or the version from `terraform_version` if specified, ex. `0.10.0`
 - `DIR`: absolute path to the root of the project on disk
 
 ## Locking
-When `plan` is run, the [project](#project) and [workspace](#workspaceenvironment) are **Locked** until an `apply` succeeds **and** the pull request/merge request is merged.
+When `plan` is run, the [project](#project) and [workspace](#workspaceenvironment) (**but not the whole repo**) are **Locked** until an `apply` succeeds **and** the pull request/merge request is merged.
 This protects against concurrent modifications to the same set of infrastructure and prevents
 users from seeing a `plan` that will be invalid if another pull request is merged.
 
@@ -463,7 +491,7 @@ A Terraform workspace. See [terraform docs](https://www.terraform.io/docs/state/
 ## FAQ
 **Q: Does Atlantis affect Terraform [remote state](https://www.terraform.io/docs/state/remote.html)?**
 
-A: No. Atlantis does not interfere with Terraform remote state in anyway. Under the hood, Atlantis is simply executing `terraform plan` and `terraform apply`.
+A: No. Atlantis does not interfere with Terraform remote state in any way. Under the hood, Atlantis is simply executing `terraform plan` and `terraform apply`.
 
 **Q: How does Atlantis locking interact with Terraform [locking](https://www.terraform.io/docs/state/locking.html)?**
 

docs/pr-comment-apply.png

@@ -46,22 +46,39 @@ func (a *ApplyExecutor) Execute(ctx *CommandContext) CommandResponse {
 	// Plans are stored at project roots by their workspace names. We just
 	// need to find them.
 	var plans []models.Plan
-	err = filepath.Walk(repoDir, func(path string, info os.FileInfo, err error) error {
+	// If they didn't specify a directory, we apply all plans we can find for
+	// this workspace.
+	if ctx.Command.Dir == "" {
+		err = filepath.Walk(repoDir, func(path string, info os.FileInfo, err error) error {
+			if err != nil {
+				return err
+			}
+			// Check if the plan is for the right workspace,
+			if !info.IsDir() && info.Name() == ctx.Command.Workspace+".tfplan" {
+				rel, _ := filepath.Rel(repoDir, filepath.Dir(path))
+				plans = append(plans, models.Plan{
+					Project:   models.NewProject(ctx.BaseRepo.FullName, rel),
+					LocalPath: path,
+				})
+			}
+			return nil
+		})
 		if err != nil {
-			return err
+			return CommandResponse{Error: errors.Wrap(err, "finding plans")}
 		}
-		// Check if the plan is for the right workspace,
-		if !info.IsDir() && info.Name() == ctx.Command.Workspace+".tfplan" {
-			rel, _ := filepath.Rel(repoDir, filepath.Dir(path))
-			plans = append(plans, models.Plan{
-				Project:   models.NewProject(ctx.BaseRepo.FullName, rel),
-				LocalPath: path,
-			})
+	} else {
+		// If they did specify a dir, we apply just the plan in that directory
+		// for this workspace.
+		path := filepath.Join(repoDir, ctx.Command.Dir, ctx.Command.Workspace+".tfplan")
+		stat, err := os.Stat(path)
+		if err != nil || stat.IsDir() {
+			return CommandResponse{Error: errors.Wrapf(err, "finding plan for dir %q and workspace %q", ctx.Command.Dir, ctx.Command.Workspace)}
 		}
-		return nil
-	})
-	if err != nil {
-		return CommandResponse{Error: errors.Wrap(err, "finding plans")}
+		rel, _ := filepath.Rel(repoDir, filepath.Dir(path))
+		plans = append(plans, models.Plan{
+			Project:   models.NewProject(ctx.BaseRepo.FullName, filepath.Dir(rel)),
+			LocalPath: path,
+		})
 	}
 	if len(plans) == 0 {
 		return CommandResponse{Failure: "No plans found for that workspace."}

docs/pr-comment-help.png

@@ -3,12 +3,14 @@ package events
 import (
 	"errors"
 	"fmt"
+	"path/filepath"
 	"strings"
 
 	"github.com/google/go-github/github"
 	"github.com/lkysow/go-gitlab"
 	"github.com/runatlantis/atlantis/server/events/models"
 	"github.com/runatlantis/atlantis/server/events/vcs"
+	"github.com/spf13/pflag"
 )
 
 const gitlabPullOpened = "opened"
@@ -20,6 +22,10 @@ type Command struct {
 	Workspace string
 	Verbose   bool
 	Flags     []string
+	// Dir is the path relative to the repo root to run the command in.
+	// If empty string then it wasn't specified. "." is the root of the repo.
+	// Dir will never end in "/".
+	Dir string
 }
 
 type EventParsing interface {
@@ -60,52 +66,82 @@ func (e *EventParser) DetermineCommand(comment string, vcsHost vcs.Host) (*Comma
 		return nil, err
 	}
 
-	workspace := "default"
-	verbose := false
-	var flags []string
-
 	vcsUser := e.GithubUser
 	if vcsHost == vcs.Gitlab {
 		vcsUser = e.GitlabUser
 	}
 	if !e.stringInSlice(args[0], []string{"run", "atlantis", "@" + vcsUser}) {
 		return nil, err
 	}
-	if !e.stringInSlice(args[1], []string{"plan", "apply", "help"}) {
+	if !e.stringInSlice(args[1], []string{"plan", "apply", "help", "-help", "--help"}) {
 		return nil, err
 	}
-	if args[1] == "help" {
+
+	command := args[1]
+	if command == "help" || command == "-help" || command == "--help" {
 		return &Command{Name: Help}, nil
 	}
-	command := args[1]
 
-	if len(args) > 2 {
-		flags = args[2:]
-
-		// if the third arg doesn't start with '-' then we assume it's a
-		// workspace, not a flag
-		if !strings.HasPrefix(args[2], "-") {
-			workspace = args[2]
-			flags = args[3:]
+	var workspace string
+	var dir string
+	var verbose bool
+	var extraArgs []string
+	var flagSet *pflag.FlagSet
+	var name CommandName
+
+	// Set up the flag parsing depending on the command.
+	const defaultWorkspace = "default"
+	if command == "plan" {
+		name = Plan
+		flagSet = pflag.NewFlagSet("plan", pflag.ContinueOnError)
+		flagSet.StringVarP(&workspace, "workspace", "w", defaultWorkspace, fmt.Sprintf("Switch to this Terraform workspace before planning. Defaults to '%s'", defaultWorkspace))
+		flagSet.StringVarP(&dir, "dir", "d", "", "Which directory to run plan in relative to root of repo. Use '.' for root. If not specified, will attempt to run plan for all Terraform projects we think were modified in this changeset.")
+		flagSet.BoolVarP(&verbose, "verbose", "", false, "Append Atlantis log to comment.")
+	} else if command == "apply" {
+		name = Apply
+		flagSet = pflag.NewFlagSet("apply", pflag.ContinueOnError)
+		flagSet.StringVarP(&workspace, "workspace", "w", defaultWorkspace, fmt.Sprintf("Apply the plan for this Terraform workspace. Defaults to '%s'", defaultWorkspace))
+		flagSet.StringVarP(&dir, "dir", "d", "", "Apply the plan for this directory, relative to root of repo. Use '.' for root. If not specified, will run apply against all plans created for this workspace.")
+		flagSet.BoolVarP(&verbose, "verbose", "", false, "Append Atlantis log to comment.")
+	} else {
+		return nil, fmt.Errorf("unknown command %q – this is a bug", command)
+	}
+
+	// Now parse the flags.
+	if err := flagSet.Parse(args[2:]); err != nil {
+		return nil, err
+	}
+	// We only use the extra args after the --. For example given a comment:
+	// "atlantis plan -bad-option -- -target=hi"
+	// we only append "-target=hi" to the eventual command.
+	// todo: keep track of the args we're discarding and include that with
+	//       comment as a warning.
+	if flagSet.ArgsLenAtDash() != -1 {
+		extraArgs = flagSet.Args()[flagSet.ArgsLenAtDash():]
+	}
+
+	// If dir is specified, must ensure it's a valid path.
+	if dir != "" {
+		validatedDir := filepath.Clean(dir)
+		// Join with . so the path is relative. This helps us if they use '/',
+		// and is safe to do if their path is relative since it's a no-op.
+		validatedDir = filepath.Join(".", validatedDir)
+		// Need to clean again to resolve relative validatedDirs.
+		validatedDir = filepath.Clean(validatedDir)
+		// Detect relative dirs since they're not allowed.
+		if strings.HasPrefix(validatedDir, "..") {
+			return nil, fmt.Errorf("relative path %q not allowed", dir)
 		}
 
-		// check for --verbose specially and then remove any additional
-		// occurrences
-		if e.stringInSlice("--verbose", flags) {
-			verbose = true
-			flags = e.removeOccurrences("--verbose", flags)
-		}
+		dir = validatedDir
 	}
-
-	c := &Command{Verbose: verbose, Workspace: workspace, Flags: flags}
-	switch command {
-	case "plan":
-		c.Name = Plan
-	case "apply":
-		c.Name = Apply
-	default:
-		return nil, fmt.Errorf("something went wrong parsing the command, the command we parsed %q was not apply or plan", command)
+	// Because we use the workspace name as a file, need to make sure it's
+	// not doing something weird like being a relative dir.
+	if strings.Contains(workspace, "..") {
+		return nil, errors.New("workspace can't contain '..'")
 	}
+
+	c := &Command{Name: name, Verbose: verbose, Workspace: workspace, Dir: dir, Flags: extraArgs}
 	return c, nil
 }
 
@@ -308,14 +344,3 @@ func (e *EventParser) stringInSlice(a string, list []string) bool {
 	}
 	return false
 }
-
-// nolint: unparam
-func (e *EventParser) removeOccurrences(a string, list []string) []string {
-	var out []string
-	for _, b := range list {
-		if b != a {
-			out = append(out, b)
-		}
-	}
-	return out
-}