You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When reading messages from kafka binding cache, the latest message in the Kafka partition might be higher than the most recently received message in the cache, for example if the remaining messages are still in flight.
Therefore, we know that reading historical messages that have already occurred will be completed when we receive that initially observed latest message in the cache.
However, due to filtering, a kafka binding cache client might not receive any messages at all, and be waiting for a new message to arrive.
In this case, it is useful for the kafka binding cache to detect it is up-to-date with all historical messages, but awaiting newer live messages.
This can be communicated per partition by sending a FLUSH reply frame with a KafkaFetchFlushEx indicating the updated progress offset at the initial latest offset.
For kafka binding merged clients, we can collect the FLUSH reply frames for each partition, and when all partitions have been accounted for, then we can send a FLUSH frame with KafkaMergedFlushEx indicating the progress offsets for all partitions, matching the initial latest offsets for all partitions received with the BEGIN reply frame and associated KafkaMergedBeginEx.
Receiving this FLUSH frame before any DATA frames will notify the kafka binding merged client of the transition from historical to live messages, while continuing to receive future messages.
The text was updated successfully, but these errors were encountered:
When reading messages from
kafka
binding cache, the latest message in the Kafka partition might be higher than the most recently received message in the cache, for example if the remaining messages are still in flight.Therefore, we know that reading historical messages that have already occurred will be completed when we receive that initially observed latest message in the cache.
However, due to filtering, a
kafka
binding cache client might not receive any messages at all, and be waiting for a new message to arrive.In this case, it is useful for the
kafka
binding cache to detect it is up-to-date with all historical messages, but awaiting newer live messages.This can be communicated per partition by sending a
FLUSH
reply frame with aKafkaFetchFlushEx
indicating the updated progress offset at the initial latest offset.For
kafka
binding merged clients, we can collect theFLUSH
reply frames for each partition, and when all partitions have been accounted for, then we can send aFLUSH
frame withKafkaMergedFlushEx
indicating the progress offsets for all partitions, matching the initial latest offsets for all partitions received with theBEGIN
reply frame and associatedKafkaMergedBeginEx
.Receiving this
FLUSH
frame before anyDATA
frames will notify thekafka
binding merged client of the transition from historical to live messages, while continuing to receive future messages.The text was updated successfully, but these errors were encountered: