Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set SO_REUSEADDR again for wasmtime serve on unix #8738

Merged

Conversation

alexcrichton
Copy link
Member

@alexcrichton alexcrichton commented Jun 3, 2024

This effectively reverts #7863 which was a misunderstanding on my part about SO_REUSEADDR. I think I mixed it up with SO_REUSEPORT. Without SO_REUSEADDR it's possible to have this happen:

  • Start a wasmtime serve session
  • Connect to the session with nc
  • Kill the server
  • Restarting the server no longer works

Due to the active TCP connection at the time the server was killed the socket/address is in the TIME_WAIT state which apparently prevents rebinding the port until the OS has a chance to clean up everything. Setting SO_REUSEADDR enables rebinding the address quickly.

Now in #7863 that was trying to fix #7852 which said that it was possible to have multiple wasmtime serve instances binding the same port. That can lead to confusion and is generally something we don't want. That being said #7863 didn't actually change anything on either macOS or Unix, although it did fix the issue on Windows. Turns out the fix for both these issues was to only turn off SO_REUSEADDR on Windows, but leave it on for Unix.

This PR additionally adds two new tests, both for rebinding an in-use port quickly and additionally ensuring the same port can't be listened to twice.

@alexcrichton alexcrichton requested a review from a team as a code owner June 3, 2024 18:59
@alexcrichton alexcrichton requested review from elliottt and removed request for a team June 3, 2024 18:59
@alexcrichton alexcrichton changed the title Set SO_REUSEADDR again for wasmtime serve Set SO_REUSEADDR again for wasmtime serve on unix Jun 3, 2024
This reverts part of bytecodealliance#7863 which was a misunderstanding on my part
about `SO_REUSEADDR`. I think I mixed it up with `SO_REUSEPORT`. Without
`SO_REUSEADDR` it's possible to have this happen on Unix:

* Start a `wasmtime serve` session
* Connect to the session with `nc`
* Kill the server
* Restarting the server no longer works

Due to the active TCP connection at the time the server was killed the
socket/address is in the `TIME_WAIT` state which apparently prevents
rebinding the port until the OS has a chance to clean up everything.
Setting `SO_REUSEADDR` enables rebinding the address quickly.

Now in bytecodealliance#7863 that was trying to fix bytecodealliance#7852 which said that it was
possible to have multiple `wasmtime serve` instances binding the same
port. That can lead to confusion and is generally something we don't
want. That being said bytecodealliance#7863 only fixed the issue for Windows but ended
up making Unix worse. This PR restores more reasonable behavior for both
Unix and Windows by conditionally setting the `SO_REUSEADDR` based on
the platform.

This PR additionally adds two new tests, both for rebinding an in-use
port quickly and additionally ensuring the same port can't be listened
to twice.
@alexcrichton
Copy link
Member Author

Ok turns out the issue was that we want SO_REUSEADDR on Unix, but not Windows. The option means different things on different platforms. The two new tests assert we have the desired behavior on all platforms now, though.

@alexcrichton alexcrichton enabled auto-merge June 3, 2024 19:35
@alexcrichton alexcrichton added this pull request to the merge queue Jun 3, 2024
Merged via the queue into bytecodealliance:main with commit bda1a64 Jun 3, 2024
36 checks passed
@alexcrichton alexcrichton deleted the serve-port-options branch June 3, 2024 19:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

wasmtime serve stuck when the port is used
2 participants