8000 Potential mis-ordering when parsing messages from a client and producing them on kafka? · Issue #261 · sbezverk/gobmp · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
Potential mis-ordering when parsing messages from a client and producing them on kafka? #261
Open
@reshad-equinix

Description

@reshad-equinix

The parser's Parser() function runs in a goroutine as shown below:

// Starting parser per client with dedicated work queue
go parser.Parser(parserQueue, producerQueue, parsStop)

And messages read from the client are sent to the parser via the parserQueue channel:

parserQueue <- fullMsg

And for every message received, the parser does its work in a goroutine:

func Parser(queue chan []byte, producerQueue chan bmp.Message, stop chan struct{}) {
for {
select {
case msg := <-queue:
go parsingWorker(msg, producerQueue)

At the end of its work the parser sends the parsed BMP message to the producer via the producerQueue:

gobmp/pkg/parser/parser.go

Lines 105 to 107 in 9fbd0c3

if producerQueue != nil && bmpMsg.Payload != nil {
producerQueue <- bmpMsg
}

So it seems possible that 2 back-to-back messages from a client device would be handled in parallel because of go parsingWorker(msg, producerQueue) in the Parser() function. And theoretically add/del messages could get handled in an incorrect order by the producer. Unless I've misunderstood this part of the code :-)

It seems overkill to have multiple goroutines to process messages from the same client. I'd make a synchronous call to parsingWorker() instead. But I don't have the history...
Update: I just clued in that it's to handle the stop msg/chan. But I think it's an issue as mentioned above. Use WaitGroup?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      0