queue: 基于semaphore的并发阻塞队列实现

[uzziahlin]

-主要改造了concurrent_array_blocking_queue的实现，将原来基于cond的实现改造成基于semaphore的实现

[flycash]

从逻辑上来看并不会引起什么问题，之前我还担心引发死锁的问题，目前来看也不会。
我简单总结一下两种实现的问题：

• 之前我的实现，问题就出在广播这里。意味着它会唤醒所有的 goroutine，但是最终只有一个 goroutine 拿到数据，其它的都会再次陷入沉睡。并发数越高，问题越大；
• 而改成 semaphore 之后，不存在广播的问题，但是问题变成了会有非常多的 channel，也就是问题出在 channel 这个开销了。之前我们在设计延时队列的时候，也考虑过类似的设计。容量越大，内存开支越大；

前者是一个 CPU 的问题，也就是 goroutine 被唤醒然后又沉睡；后者更加像是一个内存分配的问题，因为 semaphore 一次只会唤醒一个。

你们总体上倾向于哪种？

[flycash]

要是问题你解决了就记得点一下 resolve

[codecov]

Codecov Report

Merging #129 (4a9132d) into dev (3de3e1b) will increase coverage by 0.06%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##              dev     #129      +/-   ##
==========================================
+ Coverage   95.17%   95.24%   +0.06%     
==========================================
  Files          34       34              
  Lines        1699     1723      +24     
==========================================
+ Hits         1617     1641      +24     
  Misses         64       64              
  Partials       18       18              

Impacted Files	Coverage Δ
queue/concurrent_array_blocking_queue.go	100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[flycash]

我还是把这个合并进去了。核心就是：

• 广播的实现，其实也会有内存问题。因为每一次调用广播，都是要创建一个 channel，所以实际上相比之下我觉得这个实现可能会更加好

[longyue0521]

我还是把这个合并进去了。核心就是：

• 广播的实现，其实也会有内存问题。因为每一次调用广播，都是要创建一个 channel，所以实际上相比之下我觉得这个实现可能会更加好

@flycash 广播的实现,channel与监听的协程比是1:N,semaphore的实现是1:1. 我好奇我们所担心的channel开销多低有多大.

所以我修改了一下semaphore包内semaphore_bench_test.go的BenchmarkAcquireSeq基准测试 发现一个不太符合我们直觉的现象——开销为0, 我们担心的channel创建开销并不存在, 不知道是我Go版本的问题?(我用的是1.19)还是我测的不对?或者真的就是没有这个开销.

我记得在课程也出现过这个现象,——开销就是为0

goos: darwin
goarch: amd64
cpu: Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
BenchmarkAcquireSeq
BenchmarkAcquireSeq/Weighted-acquire-1-1-1
BenchmarkAcquireSeq/Weighted-acquire-1-1-1-16         	35998287	        28.93 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/Weighted-tryAcquire-1-1-1
BenchmarkAcquireSeq/Weighted-tryAcquire-1-1-1-16      	41879769	        27.04 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/semChan-acquire-1-1-1
BenchmarkAcquireSeq/semChan-acquire-1-1-1-16          	24851871	        45.41 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/semChan-tryAcquire-1-1-1
BenchmarkAcquireSeq/semChan-tryAcquire-1-1-1-16       	26257648	        43.67 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/Weighted-acquire-2-1-1
BenchmarkAcquireSeq/Weighted-acquire-2-1-1-16         	43242860	        28.76 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/Weighted-tryAcquire-2-1-1
BenchmarkAcquireSeq/Weighted-tryAcquire-2-1-1-16      	45190890	        26.94 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/semChan-acquire-2-1-1
BenchmarkAcquireSeq/semChan-acquire-2-1-1-16          	28821824	        43.82 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/semChan-tryAcquire-2-1-1
BenchmarkAcquireSeq/semChan-tryAcquire-2-1-1-16       	27721966	        44.96 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/Weighted-acquire-16-1-1
BenchmarkAcquireSeq/Weighted-acquire-16-1-1-16        	44187162	        27.92 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/Weighted-tryAcquire-16-1-1
BenchmarkAcquireSeq/Weighted-tryAcquire-16-1-1-16     	42152251	        26.67 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/semChan-acquire-16-1-1
BenchmarkAcquireSeq/semChan-acquire-16-1-1-16         	28329458	        43.75 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/semChan-tryAcquire-16-1-1
BenchmarkAcquireSeq/semChan-tryAcquire-16-1-1-16      	28469403	        43.39 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/Weighted-acquire-128-1-1
BenchmarkAcquireSeq/Weighted-acquire-128-1-1-16       	43165717	        28.53 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/Weighted-tryAcquire-128-1-1
BenchmarkAcquireSeq/Weighted-tryAcquire-128-1-1-16    	44714756	        27.37 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/semChan-acquire-128-1-1
BenchmarkAcquireSeq/semChan-acquire-128-1-1-16        	28282513	        43.44 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/semChan-tryAcquire-128-1-1
BenchmarkAcquireSeq/semChan-tryAcquire-128-1-1-16     	27839509	        42.90 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/Weighted-acquire-2-2-1
BenchmarkAcquireSeq/Weighted-acquire-2-2-1-16         	45012165	        29.21 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/Weighted-tryAcquire-2-2-1
BenchmarkAcquireSeq/Weighted-tryAcquire-2-2-1-16      	46323514	        26.54 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/semChan-acquire-2-2-1
BenchmarkAcquireSeq/semChan-acquire-2-2-1-16          	14097836	        81.13 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/semChan-tryAcquire-2-2-1
BenchmarkAcquireSeq/semChan-tryAcquire-2-2-1-16       	14871572	        80.53 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/Weighted-acquire-16-2-8
BenchmarkAcquireSeq/Weighted-acquire-16-2-8-16        	 5685538	       212.2 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/Weighted-tryAcquire-16-2-8
BenchmarkAcquireSeq/Weighted-tryAcquire-16-2-8-16     	 5542713	       214.1 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/semChan-acquire-16-2-8
BenchmarkAcquireSeq/semChan-acquire-16-2-8-16         	 1754398	       634.1 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/semChan-tryAcquire-16-2-8
BenchmarkAcquireSeq/semChan-tryAcquire-16-2-8-16      	 1889330	       636.1 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/Weighted-acquire-128-2-64
BenchmarkAcquireSeq/Weighted-acquire-128-2-64-16      	  730034	      1708 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/Weighted-tryAcquire-128-2-64
BenchmarkAcquireSeq/Weighted-tryAcquire-128-2-64-16   	  687126	      1731 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/semChan-acquire-128-2-64
BenchmarkAcquireSeq/semChan-acquire-128-2-64-16       	  212145	      5204 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/semChan-tryAcquire-128-2-64
BenchmarkAcquireSeq/semChan-tryAcquire-128-2-64-16    	  209847	      5137 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/Weighted-acquire-2-1-2
BenchmarkAcquireSeq/Weighted-acquire-2-1-2-16         	22397176	        53.51 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/Weighted-tryAcquire-2-1-2
BenchmarkAcquireSeq/Weighted-tryAcquire-2-1-2-16      	22724324	        55.06 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/semChan-acquire-2-1-2
BenchmarkAcquireSeq/semChan-acquire-2-1-2-16          	13908943	        85.60 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/semChan-tryAcquire-2-1-2
BenchmarkAcquireSeq/semChan-tryAcquire-2-1-2-16       	13333144	        84.69 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/Weighted-acquire-16-8-2
BenchmarkAcquireSeq/Weighted-acquire-16-8-2-16        	20401054	        53.34 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/Weighted-tryAcquire-16-8-2
BenchmarkAcquireSeq/Weighted-tryAcquire-16-8-2-16     	20004867	        53.01 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/semChan-acquire-16-8-2
BenchmarkAcquireSeq/semChan-acquire-16-8-2-16         	 1970730	       611.3 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/semChan-tryAcquire-16-8-2
BenchmarkAcquireSeq/semChan-tryAcquire-16-8-2-16      	 1941979	       647.4 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/Weighted-acquire-128-64-2
BenchmarkAcquireSeq/Weighted-acquire-128-64-2-16      	21613214	        55.71 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/Weighted-tryAcquire-128-64-2
BenchmarkAcquireSeq/Weighted-tryAcquire-128-64-2-16   	23113664	        54.97 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/semChan-acquire-128-64-2
BenchmarkAcquireSeq/semChan-acquire-128-64-2-16       	  251800	      4888 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/semChan-tryAcquire-128-64-2
BenchmarkAcquireSeq/semChan-tryAcquire-128-64-2-16    	  236616	      4836 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/Weighted-acquire-1000000-1-100000
BenchmarkAcquireSeq/Weighted-acquire-1000000-1-100000-16         	     436	   2641038 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/Weighted-tryAcquire-1000000-1-100000
BenchmarkAcquireSeq/Weighted-tryAcquire-1000000-1-100000-16      	     432	   2735235 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/semChan-acquire-1000000-1-100000
BenchmarkAcquireSeq/semChan-acquire-1000000-1-100000-16          	     270	   4184122 ns/op	       0 B/op	       0 allocs/op
BenchmarkAcquireSeq/semChan-tryAcquire-1000000-1-100000
BenchmarkAcquireSeq/semChan-tryAcquire-1000000-1-100000-16       	     280	   4395449 ns/op	       0 B/op	       0 allocs/op
PASS