Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chaosmonkey can not find eligible instances to terminate #108

Open
wen15th opened this issue Mar 9, 2025 · 1 comment
Open

chaosmonkey can not find eligible instances to terminate #108

wen15th opened this issue Mar 9, 2025 · 1 comment

Comments

@wen15th
Copy link

wen15th commented Mar 9, 2025

Issue Description

When I tried to terminate instances using this command:
chaosmonkey terminate my-chaosmonkey-app default

  • "my-chaosmonkey-app" is my application name
  • "default" is the account name

I got this error:

[57965] 2025/03/09 00:04:03 WARNING: eligible.Instances failed for InstanceGroup{app=my-chaosmonkey-app account=default region= stack= cluster=}: Name 'deployment my-chaosmonkey-app' does not have any labeled or unlabeled matches
[57965] 2025/03/09 00:04:03 No eligible instances in group, nothing to terminate: InstanceGroup{app=my-chaosmonkey-app account=default region= stack= cluster=}

Chaosmonkey is running on my macOS, cloud provider is Kubernetes, and the Spinnaker is managed by Halyard on docker.

Here are some information of the deployments:

Image Image

The configuration of the monkey for the application is also valid:

Image

The chaosmonkey.toml:

[chaosmonkey]
enabled = true
schedule_enabled = true
leashed = false
accounts = ["default","spinnakerk8s"]
region = "us-west-2"

start_hour = 9      # time during day when starts terminating
end_hour = 15       # time during day when stops terminating

# location of command Chaos Monkey uses for doing terminations
term_path = "./chaosmonkey-terminate.sh"

# cron file that Chaos Monkey writes to each day for scheduling kills
cron_path = "./cron.d/chaosmonkey-schedule"

[database]
host = "127.0.0.1"
name = "chaosmonkey"
user = "root"
encrypted_password = "root"

[spinnaker]
endpoint = "http://127.0.0.1:8084"

And information about my Spinnaker:
This is the application:

Image

This is the clusters info:

Image

This is the pipeline:

Image

This is the Manifest configuration of my pipeline:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-chaosmonkey-app
  namespace: default
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-chaosmonkey-app
  template:
    metadata:
      labels:
        app: my-chaosmonkey-app
    spec:
      containers:
        - image: 'wen15th/my-chaosmonkey-app:v1'
          name: my-chaosmonkey-app
          ports:
            - containerPort: 5000

I also find two similar issues:

@wen15th
Copy link
Author

wen15th commented Mar 9, 2025

Updates on 2025-03-09:

According to spinnaker/spinnaker#3671, it might be the name convention issue, so I re-configured my Manifest of the pipeline, and re-deployed the clusters, now they are showing below (the name has been changed from "deployment my-chosmonkey-app" to "my-chaosmonkey-app"):

Image

I think this should match the "app-stack-detail" name convention, but when I execute the chaosmonkey eligible command below, I still can't get any available instances returned:

chaosmonkey eligible my-chaosmonkey-app default
chaosmonkey eligible my-chaosmonkey-app spinnakerk8s
Image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant