Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support grpc stream for data plane #352

Draft
wants to merge 15 commits into
base: main
Choose a base branch
from
Draft

Conversation

hwjiangkai
Copy link
Contributor

What problem does this PR solve?

transport layer performance optimization

Issue Number: close #xxx

Problem Summary

What is changed and how does it work?

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

@hwjiangkai hwjiangkai marked this pull request as draft December 17, 2022 08:12
@codecov
Copy link

codecov bot commented Dec 17, 2022

Codecov Report

Merging #352 (73b43c0) into main (09f3748) will decrease coverage by 3.70%.
The diff coverage is 51.84%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #352      +/-   ##
==========================================
- Coverage   62.72%   59.02%   -3.71%     
==========================================
  Files         133      172      +39     
  Lines       12128    14220    +2092     
==========================================
+ Hits         7607     8393     +786     
- Misses       3977     5225    +1248     
- Partials      544      602      +58     
Impacted Files Coverage Δ
internal/controller/eventbus/server/instance.go 72.04% <0.00%> (ø)
internal/controller/trigger/storage/secret.go 48.48% <0.00%> (ø)
...nternal/controller/trigger/storage/subscription.go 54.71% <0.00%> (ø)
...ernal/controller/trigger/storage/trigger_worker.go 63.41% <0.00%> (ø)
...rnal/primitive/interceptor/errinterceptor/error.go 0.00% <0.00%> (ø)
.../primitive/interceptor/memberinterceptor/member.go 0.00% <0.00%> (ø)
internal/primitive/vanus/id.go 19.68% <0.00%> (-0.65%) ⬇️
internal/raft/log/recovery.go 52.27% <ø> (ø)
internal/raft/log/wal.go 100.00% <ø> (ø)
internal/store/block/entry.go 70.96% <ø> (ø)
... and 164 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6b23e5a...73b43c0. Read the comment docs.

@hwjiangkai hwjiangkai marked this pull request as ready for review December 17, 2022 13:12
Signed-off-by: jyjiangkai <[email protected]>
@hwjiangkai hwjiangkai requested a review from xdlbdy as a code owner December 19, 2022 16:26
Copy link
Collaborator

@ifplusor ifplusor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't serialize on a stream.

@hwjiangkai hwjiangkai marked this pull request as draft December 20, 2022 03:32
@hwjiangkai hwjiangkai marked this pull request as ready for review December 21, 2022 06:10
@ifplusor ifplusor changed the title feat: transport layer performance optimization feat: support grpc stream for data plane Dec 22, 2022
Signed-off-by: jyjiangkai <[email protected]>
if stderr.Is(err, io.EOF) {
s.appendStream.CloseSend()
s.appendStream = nil
c, _ := s.appendCallbacks.LoadAndDelete(opaqueID)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

delete for all error

Signed-off-by: jyjiangkai <[email protected]>
@hwjiangkai hwjiangkai force-pushed the clientstream-v2 branch 2 times, most recently from c527fad to e3f573e Compare December 28, 2022 12:43
Signed-off-by: jyjiangkai <[email protected]>
@hwjiangkai hwjiangkai force-pushed the clientstream-v2 branch 2 times, most recently from 1460868 to 1f7ad23 Compare January 6, 2023 09:46
Comment on lines 214 to 215
errCode = errpb.ErrorCode_UNKNOWN
errMsg = "unknown"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remains detail of ErrorType.

errCode = errpb.ErrorCode_APPEND_TO_BLOCK_FAILED
errMsg = "append to block failed"
}
log.Error(ctx, "append to block failed", map[string]interface{}{
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use Warning, not Error

request, err := stream.Recv()
if err != nil {
if stderr.Is(err, io.EOF) {
log.Warning(ctx, "append stream closed", map[string]interface{}{
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EOF is ok, don't use Warning and Error

Signed-off-by: jyjiangkai <[email protected]>
@ifplusor ifplusor marked this pull request as draft February 14, 2023 02:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants