Skip to content

Commit

Permalink
Add pre workflow custom hooks to run scripts before workflow executio…
Browse files Browse the repository at this point in the history
…n(plan, apply, etc) (#1255)

* Updated runatlantis.io/docs to have `pre-workflow-hooks` use cases and examples
  • Loading branch information
msarvar authored Dec 14, 2020
1 parent 5a1e191 commit 3456dc9
Show file tree
Hide file tree
Showing 22 changed files with 1,015 additions and 101 deletions.
1 change: 1 addition & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -411,6 +411,7 @@ golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAG
golang.org/x/oauth2 v0.0.0-20181106182150-f42d05182288/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190402181905-9f3314589c9a/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45 h1:SVwTIAaPC2U/AvvLNZ2a7OVsmBpC8L5BlwK1whH3hm0=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20191122200657-5d9234df094c h1:HjRaKPaiWks0f5tA6ELVF7ZfqSppfPwOEEAvsrKUTO4=
golang.org/x/oauth2 v0.0.0-20191122200657-5d9234df094c/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
Expand Down
1 change: 1 addition & 0 deletions runatlantis.io/.vuepress/config.js
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ module.exports = {
['configuring-atlantis', 'Overview'],
'server-configuration',
'server-side-repo-config',
'pre-workflow-hooks',
'custom-workflows',
'repo-level-atlantis-yaml',
'upgrading-atlantis-yaml',
Expand Down
52 changes: 52 additions & 0 deletions runatlantis.io/docs/pre-workflow-hooks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Pre Workflow Hooks

Pre workflow hooks can be defined to run scripts right before default or custom
workflows are executed.

[[toc]]

## Usage
Pre workflow hooks can only be specified in the Server-Side Repo Config under
`repos` key.
::: tip Note
`pre-workflow-hooks` do not prevent Atlantis from executing its
workflows(`plan`, `apply`) even if a `run` command exits with an error.
:::

## Use Cases
### Dynamic Repo Config Generation
If you want generate your `atlantis.yaml` before Atlantis can parse it. You
can add a `run` command to `pre_workflow_hooks`. Your Repo config will be generated
right before Atlantis can parse it.

```yaml
repos:
- id: /.*/
pre_workflow_hooks:
- run: ./repo-config-genarator.sh
```
### Reference
#### Custom `run` Command
This is very similar to [custom workflow run
command](custom-workflows.html#custom-run-command).
```yaml
- run: custom-command
```
| Key | Type | Default | Required | Description |
|-----|--------|---------|----------|----------------------|
| run | string | none | no | Run a custom command |

::: tip Notes
* `run` commands are executed with the following environment variables:
* `BASE_REPO_NAME` - Name of the repository that the pull request will be merged into, ex. `atlantis`.
* `BASE_REPO_OWNER` - Owner of the repository that the pull request will be merged into, ex. `runatlantis`.
* `HEAD_REPO_NAME` - Name of the repository that is getting merged into the base repository, ex. `atlantis`.
* `HEAD_REPO_OWNER` - Owner of the repository that is getting merged into the base repository, ex. `acme-corp`.
* `HEAD_BRANCH_NAME` - Name of the head branch of the pull request (the branch that is getting merged into the base)
* `BASE_BRANCH_NAME` - Name of the base branch of the pull request (the branch that the pull request is getting merged into)
* `PULL_NUM` - Pull request number or ID, ex. `2`.
* `PULL_AUTHOR` - Username of the pull request author, ex. `acme-user`.
* `DIR` - The absolute path to the root of the cloned repository.
* `USER_NAME` - Username of the VCS user running command, ex. `acme-user`. During an autoplan, the user will be the Atlantis API user, ex. `atlantis`.
:::

21 changes: 20 additions & 1 deletion runatlantis.io/docs/server-side-repo-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ If you don't wish to write a config file to disk, you can use the
`--repo-config-json` flag or `ATLANTIS_REPO_CONFIG_JSON` environment variable
to specify your config as JSON. See [--repo-config-json](server-configuration.html#repo-config-json)
for an example.

## Example Server Side Repo
```yaml
# repos lists the config for specific repos.
Expand Down Expand Up @@ -48,6 +48,10 @@ repos:
# workflows. If false (default), the repo can only use server-side defined
# workflows.
allow_custom_workflows: true

# pre_workflow_hooks defines arbitrary list of scripts to execute before workflow execution.
pre_workflow_hooks:
- run: my-pre-workflow-hook-command arg1

# id can also be an exact match.
- id: github.com/myorg/specific-repo
Expand Down Expand Up @@ -153,6 +157,21 @@ projects:
apply_requirements: []
```

### Running Scripts Before Atlantis Workflows
If you want to run scripts that would execute before Atlantis can run default or
custom workflows, you can create a `pre-workflow-hooks`:

```yaml
repos:
- id: /.*/
pre_workflow_hooks:
- run: my custom command
- run: |
my bash script inline
```
See [Pre Workflow Hooks](pre-workflow-hooks.html) for more details on writing
pre workflow hooks.

### Change The Default Atlantis Workflow
If you want to change the default commands that Atlantis runs during `plan` and `apply`
phases, you can create a new `workflow`.
Expand Down
112 changes: 112 additions & 0 deletions server/events/mocks/mock_pre_workflows_hooks_command_runner.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

19 changes: 19 additions & 0 deletions server/events/models/models.go
Original file line number Diff line number Diff line change
Expand Up @@ -535,3 +535,22 @@ func (c CommandName) String() string {
}
return ""
}

// PreWorkflowHookCommandContext defines the context for a pre_worklfow_hooks that will
// be executed before workflows.
type PreWorkflowHookCommandContext struct {
// BaseRepo is the repository that the pull request will be merged into.
BaseRepo Repo
// HeadRepo is the repository that is getting merged into the BaseRepo.
// If the pull request branch is from the same repository then HeadRepo will
// be the same as BaseRepo.
HeadRepo Repo
// Log is a logger that's been set up for this context.
Log logging.SimpleLogging
// Pull is the pull request we're responding to.
Pull PullRequest
// User is the user that triggered this command.
User User
// Verbose is true when the user would like verbose output.
Verbose bool
}
129 changes: 129 additions & 0 deletions server/events/pre_workflow_hooks_command_runner.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
package events

import (
"fmt"

"github.com/runatlantis/atlantis/server/events/models"
"github.com/runatlantis/atlantis/server/events/runtime"
"github.com/runatlantis/atlantis/server/events/vcs"
"github.com/runatlantis/atlantis/server/events/yaml/valid"
"github.com/runatlantis/atlantis/server/logging"
"github.com/runatlantis/atlantis/server/recovery"
)

//go:generate pegomock generate -m --use-experimental-model-gen --package mocks -o mocks/mock_pre_workflows_hooks_command_runner.go PreWorkflowHooksCommandRunner

type PreWorkflowHooksCommandRunner interface {
RunPreHooks(
baseRepo models.Repo,
headRepo models.Repo,
pull models.PullRequest,
user models.User,
)
}

// DefaultPreWorkflowHooksCommandRunner is the first step when processing a workflow hook commands.
type DefaultPreWorkflowHooksCommandRunner struct {
VCSClient vcs.Client
Logger logging.SimpleLogging
WorkingDirLocker WorkingDirLocker
WorkingDir WorkingDir
GlobalCfg valid.GlobalCfg
Drainer *Drainer
PreWorkflowHookRunner *runtime.PreWorkflowHookRunner
}

// RunPreHooks runs pre_workflow_hooks when PR is opened or updated.
func (w *DefaultPreWorkflowHooksCommandRunner) RunPreHooks(
baseRepo models.Repo,
headRepo models.Repo,
pull models.PullRequest,
user models.User,
) {
if opStarted := w.Drainer.StartOp(); !opStarted {
if commentErr := w.VCSClient.CreateComment(baseRepo, pull.Num, ShutdownComment, "pre_workflow_hooks"); commentErr != nil {
w.Logger.Log(logging.Error, "unable to comment that Atlantis is shutting down: %s", commentErr)
}
return
}
defer w.Drainer.OpDone()

log := w.buildLogger(baseRepo.FullName, pull.Num)
defer w.logPanics(baseRepo, pull.Num, log)

log.Info("running pre hooks")

unlockFn, err := w.WorkingDirLocker.TryLock(baseRepo.FullName, pull.Num, DefaultWorkspace)
if err != nil {
log.Warn("workspace is locked")
return
}
log.Debug("got workspace lock")
defer unlockFn()

repoDir, _, err := w.WorkingDir.Clone(log, headRepo, pull, DefaultWorkspace)
if err != nil {
log.Err("unable to run pre workflow hooks: %s", err)
return
}

preWorkflowHooks := make([]*valid.PreWorkflowHook, 0)
for _, repo := range w.GlobalCfg.Repos {
if repo.IDMatches(baseRepo.ID()) && len(repo.PreWorkflowHooks) > 0 {
preWorkflowHooks = append(preWorkflowHooks, repo.PreWorkflowHooks...)
}
}

ctx := models.PreWorkflowHookCommandContext{
BaseRepo: baseRepo,
HeadRepo: headRepo,
Log: log,
Pull: pull,
User: user,
Verbose: false,
}

err = w.runHooks(ctx, preWorkflowHooks, repoDir)

if err != nil {
log.Err("pre workflow hook run error results: %s", err)
}
}

func (w *DefaultPreWorkflowHooksCommandRunner) runHooks(
ctx models.PreWorkflowHookCommandContext,
preWorkflowHooks []*valid.PreWorkflowHook,
repoDir string,
) error {

for _, hook := range preWorkflowHooks {
_, err := w.PreWorkflowHookRunner.Run(ctx, hook.RunCommand, repoDir)

if err != nil {
return nil
}
}

return nil
}

func (w *DefaultPreWorkflowHooksCommandRunner) buildLogger(repoFullName string, pullNum int) *logging.SimpleLogger {
src := fmt.Sprintf("%s#%d", repoFullName, pullNum)
return w.Logger.NewLogger(src, true, w.Logger.GetLevel())
}

// logPanics logs and creates a comment on the pull request for panics.
func (w *DefaultPreWorkflowHooksCommandRunner) logPanics(baseRepo models.Repo, pullNum int, logger logging.SimpleLogging) {
if err := recover(); err != nil {
stack := recovery.Stack(3)
logger.Err("PANIC: %s\n%s", err, stack)
if commentErr := w.VCSClient.CreateComment(
baseRepo,
pullNum,
fmt.Sprintf("**Error: goroutine panic. This is a bug.**\n```\n%s\n%s```", err, stack),
"",
); commentErr != nil {
logger.Err("unable to comment: %s", commentErr)
}
}
}
Loading

0 comments on commit 3456dc9

Please sign in to comment.