8000 dapr gprc connection lost when pod rolling update in kubernetes · Issue #8602 · dapr/dapr · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
dapr gprc connection lost when pod rolling update in kubernetes #8602
Open
@cataglyphis

Description

@cataglyphis

dapr version: 1.13.5

Below picture is the topology of our services in Kubernetes. When we update Pod B, Pod A has lots of error message like below, seems like Pod A still use grpc connection to Pod B.

Pod B dapr sidecar has annotation graceful-shutdown-seconds, and container b has preStop probe.

dapr.io/graceful-shutdown-seconds: "25"

...

lifecycle:
  preStop:
    exec:
      command:
      - sh
      - -c
      - sleep 16

...

terminationGracePeriodSeconds: 30
rpc client unary retry. Err:rpc error: code = Unavailable desc = last connection error: connection error: desc = "transport: Error while dialing: dial tcp 10.110.0.2:50002: connect: connection refused" retry:2

Our question is how to avoid such problem when deployment rolling update?

Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      0