"Atlantis apply" fails where "atlantis apply -p project" works #2168

MrLuje · 2022-03-28T15:09:36Z

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Overview of the Issue

I have a multi-projects repository on gitlab.

Autoplan works fine
atlantis apply -p project also works fine
atlantis apply fails with message : running git ls-files . --others: : fork/exec /usr/bin/git: not a directory

Reproduction Steps

Create a Merge Request in the multi-projects repository
Autoplan triggers and detects changes in multiple projects
Send a atlantis apply through Merge Request comment
Apply fails with message : running git ls-files . --others: : fork/exec /usr/bin/git: not a directory

Logs

{"level":"info","ts":"2022-03-28T13:50:14.469Z","caller":"events/events_controller.go:417","msg":"parsed comment as command=\"apply\" verbose=false dir=\"\" workspace=\"\" project=\"\" flags=\"\"","json":{}}
{"level":"info","ts":"2022-03-28T13:50:19.914Z","caller":"runtime/pre_workflow_hook_runner.go:50","msg":"successfully ran \"curl -sS https://webinstall.dev/jq | bash\" in \"/atlantis-data/repos/test-repo/4/default\"","json":{"repo":"test-repo","pull":"4"}}
{"level":"info","ts":"2022-03-28T13:50:21.413Z","caller":"runtime/pre_workflow_hook_runner.go:50","msg":"successfully ran \"curl -L https://github.com/mikefarah/yq/releases/download/v4.22.1/yq_linux_amd64 -o /home/atlantis/.local/yq && chmod +x /home/atlantis/.local/yq\" in \"/atlantis-data/repos/test-repo/4/default\"","json":{"repo":"test-repo","pull":"4"}}
{"level":"info","ts":"2022-03-28T13:50:21.414Z","caller":"runtime/pre_workflow_hook_runner.go:50","msg":"successfully ran \"cp /etc/config/gitlab/env.sh ../env.sh\" in \"/atlantis-data/repos/test-repo/4/default\"","json":{"repo":"test-repo","pull":"4"}}
{"level":"info","ts":"2022-03-28T13:50:24.514Z","caller":"runtime/pre_workflow_hook_runner.go:50","msg":"successfully ran \"chmod +x ../env.sh && ../env.sh\" in \"/atlantis-data/repos/test-repo/4/default\"","json":{"repo":"test-repo","pull":"4"}}
{"level":"warn","ts":"2022-03-28T13:50:30.512Z","caller":"events/apply_command_runner.go:111","msg":"unable to update commit status: POST https://private-gitlab/api/v4/projects/test-repo/statuses/09ffc07d3f7722a9d655a9dec7da82079af52f1b: 400 {message: Cannot transition status via :run from :running (Reason(s): Status cannot transition via \"run\")}","json":{"repo":"test-repo","pull":"4"},"stacktrace":"github.com/runatlantis/atlantis/server/events.(*ApplyCommandRunner).Run\n\tgithub.com/runatlantis/atlantis/server/events/apply_command_runner.go:111\ngithub.com/runatlantis/atlantis/server/events.(*DefaultCommandRunner).RunCommentCommand\n\tgithub.com/runatlantis/atlantis/server/events/command_runner.go:259"}
{"level":"error","ts":"2022-03-28T13:50:30.851Z","caller":"events/pull_updater.go:14","msg":"running git ls-files . --others: : fork/exec /usr/bin/git: not a directory","json":{"repo":"test-repo","pull":"4"},"stacktrace":"github.com/runatlantis/atlantis/server/events.(*PullUpdater).updatePull\n\tgithub.com/runatlantis/atlantis/server/events/pull_updater.go:14\ngithub.com/runatlantis/atlantis/server/events.(*ApplyCommandRunner).Run\n\tgithub.com/runatlantis/atlantis/server/events/apply_command_runner.go:121\ngithub.com/runatlantis/atlantis/server/events.(*DefaultCommandRunner).RunCommentCommand\n\tgithub.com/runatlantis/atlantis/server/events/command_runner.go:259"}

Environment details

Atlantis version: 0.19.2

Atlantis server-side config file:

repoConfig: |
  ---
  repos:
  - id: /.*/
    pre_workflow_hooks:
     - run: curl -sS https://webinstall.dev/jq | bash
     - run: curl -L https://github.com/mikefarah/yq/releases/download/v4.22.1/yq_linux_amd64 -o /home/atlantis/.local/yq && chmod +x /home/atlantis/.local/yq
     - run: cp /etc/config/gitlab/env.sh ../env.sh
     - run: chmod +x ../env.sh && ../env.sh                <--- only used to generate atlantis.yaml file
    apply_requirements: [approved, mergeable, undiverged]
    allowed_overrides: [workflow]
    allow_custom_workflows: true

Repo atlantis.yaml file:

version: 3
automerge: true
projects:
  - name: infra
    dir: src/infra
    autoplan:
      when_modified: ["*.tf", "../modules/**/*.tf"]
      enabled: true
    workflow: infra_src/infra
  - name: vault
    dir: src/k8s/vault/terraform
    autoplan:
      when_modified: ["*.tf", "../modules/**/*.tf"]
      enabled: true
    workflow: infra_src/k8s/vault/terraform
  - name: test-terraform
    dir: src/k8s/test-terraform
    autoplan:
      when_modified: ["*.tf", "../modules/**/*.tf"]
      enabled: true
    workflow: infra_src/k8s/test-terraform

workflows:
    infra_src/infra:
        plan:
            steps:
            - env:
                name: TF_STATE
                value: src_infra
            - env:
                name: TF_HTTP_USERNAME
                command: 'echo "${ATLANTIS_GITLAB_USER}"'
            - env:
                name: TF_HTTP_PASSWORD
                command: 'echo "${ATLANTIS_GITLAB_TOKEN}"'
            - env:
                name: TF_VAR_project_name
                value: 'eu-west-0'
            - init
            - plan

        apply:
            steps:
            - env:
                name: TF_STATE
                value: src_infra
            - env:
                name: TF_HTTP_USERNAME
                command: 'echo "${ATLANTIS_GITLAB_USER}"'
            - env:
                name: TF_HTTP_PASSWORD
                command: 'echo "${ATLANTIS_GITLAB_TOKEN}"'
            - env:
                name: TF_VAR_project_name
                value: 'eu-west-0'
            - apply

    infra_src/k8s/vault/terraform:
        plan:
            steps:
            - env:
                name: TF_STATE
                value: src_infra
            - env:
                name: TF_HTTP_USERNAME
                command: 'echo "${ATLANTIS_GITLAB_USER}"'
            - env:
                name: TF_HTTP_PASSWORD
                command: 'echo "${ATLANTIS_GITLAB_TOKEN}"'
            - env:
                name: TF_VAR_project_name
                value: 'eu-west-0'
            - init
            - plan

        apply:
            steps:
            - env:
                name: TF_STATE
                value: src_infra
            - env:
                name: TF_HTTP_USERNAME
                command: 'echo "${ATLANTIS_GITLAB_USER}"'
            - env:
                name: TF_HTTP_PASSWORD
                command: 'echo "${ATLANTIS_GITLAB_TOKEN}"'
            - env:
                name: TF_VAR_project_name
                value: 'eu-west-0'
            - apply
    infra_src/k8s/test-terraform:
        plan:
            steps:
            - env:
                name: TF_STATE
                value: src_infra
            - env:
                name: TF_HTTP_USERNAME
                command: 'echo "${ATLANTIS_GITLAB_USER}"'
            - env:
                name: TF_HTTP_PASSWORD
                command: 'echo "${ATLANTIS_GITLAB_TOKEN}"'
            - env:
                name: TF_VAR_project_name
                value: 'eu-west-0'
            - init
            - plan

        apply:
            steps:
            - env:
                name: TF_STATE
                value: src_infra
            - env:
                name: TF_HTTP_USERNAME
                command: 'echo "${ATLANTIS_GITLAB_USER}"'
            - env:
                name: TF_HTTP_PASSWORD
                command: 'echo "${ATLANTIS_GITLAB_TOKEN}"'
            - env:
                name: TF_VAR_project_name
                value: 'eu-west-0'
            - apply

Additional Context

if I connect to the pod running atlantis & cd to my repo in /atlantis-data, git commands work fine

bash-5.1$ /usr/bin/git status
On branch test/atlantis
Your branch is up to date with 'origin/test/atlantis'.

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        atlantis.yaml
        src/infra/infra-default.tfplan
        src/k8s/test-terraform/test-terraform-default.tfplan
        src/k8s/vault/terraform/vault-default.tfplan

nothing added to commit but untracked files present (use "git add" to track)

bash-5.1$ git ls-files . --others
atlantis.yaml
[...]

Project structure

.
+--- .gitlab-ci.yml
+--- src
|   +--- infra
|   |   +--- main.tf
|   |   +--- .terraform.lock.hcl
|   +--- k8s
|   |   +--- vault
|   |   |   +--- terraform
|   |   |   |   +--- main.tf
|   |   |   |   +--- .terraform.lock.hcl
|   |   +--- test-terraform
|   |   |   +--- main.tf
+--- .gitignore
+--- atlantis.yaml

Global apply

Per project apply

The text was updated successfully, but these errors were encountered:

Dilergore · 2022-05-02T09:48:19Z

I am running into the same or similar issue:

2022-05-02T09:41:16.560622935Z {"level":"error","ts":"2022-05-02T09:41:16.560Z","caller":"events/pull_updater.go:14","msg":"running git ls-files . --others: fatal: not a git repository (or any of the parent directories): .git\n: exit status 128","json":{"repo":"orgname/projectname/reponame","pull":"109777"},"stacktrace":"github.com/runatlantis/atlantis/server/events.(*PullUpdater).updatePull\n\t/home/runner/work/atlantis/atlantis/server/events/pull_updater.go:14\ngithub.com/runatlantis/atlantis/server/events.(*ApplyCommandRunner).Run\n\t/home/runner/work/atlantis/atlantis/server/events/apply_command_runner.go:121\ngithub.com/runatlantis/atlantis/server/events.(*DefaultCommandRunner).RunCommentCommand\n\t/home/runner/work/atlantis/atlantis/server/events/command_runner.go:259"}

Apply works if I specify the folder like atlantis apply -d xxx/yyy.

Dilergore · 2022-05-03T11:42:57Z

I was able to debug this. This error message appears in case you create an additional folder one level above the repo root.

For example in case your repo root is:
/mnt/persistance/repos/devops-org-name/devops-project-name/devops-repo-name/pr-number/default

and you create an additional folder like:
/mnt/persistance/repos/devops-org-name/devops-project-name/devops-repo-name/pr-number/xxx

You get the above error.

jaysonsantos · 2022-05-06T17:16:58Z

@Dilergore that is funny because if you create workspaces, they always become a sibling of default folder

ckite · 2022-07-01T20:15:31Z

I'm having the same problem as well.

{ "level": "error", "ts": "2022-07-01T19:21:01.691Z", "caller": "events/instrumented_project_command_builder.go:27", "msg": "Error building apply commands: running git ls-files . --others: fatal: unsafe repository ('/home/atlantis/.atlantis/repos/company/group/terraform/null_resource/10/default' is owned by someone else)\nTo add an exception for this directory, call:\n\n\tgit config --global --add safe.directory /home/atlantis/.atlantis/repos/company/group/terraform/null_resource/10/default\n: exit status 128", "json": {}, "stacktrace": "github.com/runatlantis/atlantis/server/events.(*InstrumentedProjectCommandBuilder).BuildApplyCommands\n\tgithub.com/runatlantis/atlantis/server/events/instrumented_project_command_builder.go:27\ngithub.com/runatlantis/atlantis/server/events.(*ApplyCommandRunner).Run\n\tgithub.com/runatlantis/atlantis/server/events/apply_command_runner.go:116\ngithub.com/runatlantis/atlantis/server/events.(*DefaultCommandRunner).RunCommentCommand\n\tgithub.com/runatlantis/atlantis/server/events/command_runner.go:277" }

I'm fairly new to Atlantis; besides adding -p dev or -d terraform/ to my comments, has anyone found a workaround?

jamengual · 2022-08-26T04:02:20Z

is this still happening with v0.19.8?

MrLuje · 2022-08-26T08:25:53Z

@jamengual yes, updated a few minutes ago

jamengual · 2022-08-26T15:33:31Z

are you running a custom image?

…

On Fri, Aug 26, 2022, 1:26 a.m. MrLuje ***@***.***> wrote: @jamengual <https://github.com/jamengual> yes, updated a few minutes ago [image: image] <https://user-images.githubusercontent.com/632075/186857926-84dc93fb-8ee8-4307-b486-534cb69371ff.png> — Reply to this email directly, view it on GitHub <#2168 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAQ3ERGES5QN6NFMNHT5Z5LV3B5RZANCNFSM5R3O53RA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

MrLuje · 2022-08-29T08:38:32Z

No, I'm using the stock one (atlantis/atlantis:v0.19.8)

ilyaluk · 2022-09-07T19:12:24Z

I'm having a similar issue, and was able to debug it:

The error says that something is not a directory. The command git ls-files . --others is called only in one place: (*DefaultPendingPlanFinder).findWithAbsPaths.

This function lists all entries in the PR folder (like /home/atlantis/data/repos/owner/repo/1234), and then runs git ls-files . --others for each entry, setting cmd.Dir to this entry.

In my case, some steps in the workflow were creating files in aforementioned PR directory, hence breaking this logic and Atlantis trying to run git ls-files . --others with file path in cmd.Dir.

I'm not sure whether you are having the same issue, but judging by the error and some steps that copy files outside of current dir, this could be it. Hope this helps!

This could be fixed at Atlantis level: for example, by reading workspace list from atlantis.yaml, and not by listing the PR directory, or by ignoring all non-directories in PR dir.

Gabryel8818 · 2022-10-06T19:53:45Z

Same thing in version 0.19.9

MrLuje · 2023-01-07T15:51:48Z

@jamengual Can I work on this one ?
As @jilyaluk stated, the issue is about extra files generated by pre_hooks at pullDir level. Can be fixed by only considering folders in pullDir (since there should only be workspaces, here)

jamengual · 2023-01-07T18:17:14Z

@MrLuje Fabiano is already working on this one, he created a pull request.

if you are interested you can take a look at it get familiar with the code and maybe give some feedback.

jamengual · 2023-01-07T18:19:22Z

#2921

MrLuje added the bug Something isn't working label Mar 28, 2022

jamengual added the waiting-on-response Waiting for a response from the user label Aug 26, 2022

jamengual added help wanted Good feature for contributors and removed waiting-on-response Waiting for a response from the user labels Aug 29, 2022

nitrocode mentioned this issue Jan 7, 2023

feat: Clone only once per PR #2921

Closed

mmokijewski mentioned this issue Jan 26, 2024

fix: improve git error log while running 'atlantis apply' #4177

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

"Atlantis apply" fails where "atlantis apply -p project" works #2168

"Atlantis apply" fails where "atlantis apply -p project" works #2168

MrLuje commented Mar 28, 2022 •

edited

Loading

Dilergore commented May 2, 2022 •

edited

Loading

Dilergore commented May 3, 2022

jaysonsantos commented May 6, 2022

ckite commented Jul 1, 2022 •

edited

Loading

jamengual commented Aug 26, 2022

MrLuje commented Aug 26, 2022

jamengual commented Aug 26, 2022 via email

MrLuje commented Aug 29, 2022

ilyaluk commented Sep 7, 2022 •

edited

Loading

Gabryel8818 commented Oct 6, 2022

MrLuje commented Jan 7, 2023

jamengual commented Jan 7, 2023

jamengual commented Jan 7, 2023

"Atlantis apply" fails where "atlantis apply -p project" works #2168

"Atlantis apply" fails where "atlantis apply -p project" works #2168

Comments

MrLuje commented Mar 28, 2022 • edited Loading

Community Note

Overview of the Issue

Reproduction Steps

Logs

Environment details

Additional Context

Project structure

Global apply

Per project apply

Dilergore commented May 2, 2022 • edited Loading

Dilergore commented May 3, 2022

jaysonsantos commented May 6, 2022

ckite commented Jul 1, 2022 • edited Loading

jamengual commented Aug 26, 2022

MrLuje commented Aug 26, 2022

jamengual commented Aug 26, 2022 via email

MrLuje commented Aug 29, 2022

ilyaluk commented Sep 7, 2022 • edited Loading

Gabryel8818 commented Oct 6, 2022

MrLuje commented Jan 7, 2023

jamengual commented Jan 7, 2023

jamengual commented Jan 7, 2023

MrLuje commented Mar 28, 2022 •

edited

Loading

Dilergore commented May 2, 2022 •

edited

Loading

ckite commented Jul 1, 2022 •

edited

Loading

ilyaluk commented Sep 7, 2022 •

edited

Loading