Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add sound and browser notifications for agent state changes #6530

Merged
merged 74 commits into from
Feb 27, 2025

Conversation

xingyaoww
Copy link
Collaborator

@xingyaoww xingyaoww commented Jan 29, 2025

Add notification features to improve user experience when the agent needs attention:

  • Sound notifications when agent needs input (with toggle)
  • Browser tab title flashing when window is not focused
  • Tooltips for all trajectory buttons
  • Proper SSR handling and i18n support

Sound button:

image

Browser tab notification:

image

The notifications trigger when agent state changes to:

  • Awaiting user input
  • Awaiting confirmation
  • Task finished

To run this PR locally, use the following command:

docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:5b95c3c-nikolaik   --name openhands-app-5b95c3c   docker.all-hands.dev/all-hands-ai/openhands:5b95c3c

openhands-agent and others added 30 commits January 29, 2025 22:18
- Reverted vite.config.ts to main branch version
- Reverted package.json and package-lock.json to main branch versions
- Reverted Dockerfile to main branch version to maintain consistent Node.js version
- Added @types/react-router-dom package
- Created custom type declaration for use-sound
- Fixed linting issues
@xingyaoww
Copy link
Collaborator Author

image image

Copy link
Member

@amanape amanape left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very satisfying, but maybe we don't want to play the audio if the user is already focus on the tab?

@xingyaoww
Copy link
Collaborator Author

maybe we don't want to play the audio if the user is already focus on the tab?

hmm good point, although I think a lot of time, I put openhands in a separate screen with tab open, while working on the other screen. We'd actually want openhands to remind us in this case, right?

@amanape
Copy link
Member

amanape commented Feb 27, 2025

I believe that you'd be "focusing" on a screen or tab if you click it. I haven't tried but this should mean that even if you have OpenHands on another screen, the tab will be considered "unfocused" if you're doing stuff on the other screen.

You've provided the option to disable the audio too so IDM the current implementation either way

@xingyaoww
Copy link
Collaborator Author

@amanape Yeah.. I think another potential use case would be, I focus on the tab but I'm away in the kitchen cooking stuff 🤣.. In this usecase (that actually happens a lot to me) sound notification while in-focus could still be useful.

But anyway, since we allow the user to toggle off this notification - it should be fine? if ppl are annoyed, they can simply turn this off.

@amanape
Copy link
Member

amanape commented Feb 27, 2025

Then in that case maybe we can default to false

@xingyaoww
Copy link
Collaborator Author

done @amanape

@xingyaoww xingyaoww enabled auto-merge (squash) February 27, 2025 18:15
@xingyaoww xingyaoww merged commit 616ff49 into main Feb 27, 2025
15 checks passed
@xingyaoww xingyaoww deleted the feature/notifications branch February 27, 2025 18:40
zchn pushed a commit to zchn/OpenHands that referenced this pull request Feb 28, 2025
adityasoni9998 pushed a commit to adityasoni9998/OpenHands that referenced this pull request Mar 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants