Help with resolving an unhealthy workspace agent #11448
-
|
I have coder deployed on a k8s cluster, I'm using an istio ingress gateway and have a domain pointing to the external IP and a virtual service to direct it to the coder pod matching port 443 to 80 essentially acting as a reverse proxy. The access url for coder is the same as the virtual service host. During the deployment I pass in the certs needed to validate requests When I go to https://custom-coder-domain/health everything is working (url is reachable, database, websockets and workspace proxy are working) When I create a workspace, the kubernetes pod is created. However, after a few minutes of running the agent becomes unhealthy. None of the metrics (cpu usage, RAM usage, etc.) are visible. Also when I shell into either the coder pods or the workspace pod and try to curl https://custom-coder-domain Not sure if this is an issue with coder trying to access itself via the virtual service. Would appreciate any help in resolving this issue |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 9 replies
-
|
cc: @ericpaulsen if you have seen this before. |
Beta Was this translation helpful? Give feedback.
-
|
After a bit more digging I've found that in the logs for the created workspace the error is happening when the pods does: looked at the pods ssl folder and its doesn't have the needed certs to be able to verify the request. I assume the solution would be to make sure those certs have been added to the pod at its creation. Is there a way to do this? |
Beta Was this translation helpful? Give feedback.
-
|
Yes, you need to have the certs available in the pods image. |
Beta Was this translation helpful? Give feedback.
-
|
I'm using the codercom/enterprise-base:ubuntu image, is there a way to either edit the init script or add a script to run "sudo update ca-certificates" as i've mounted the certs into the pod but want to run update ca-certs automatically |
Beta Was this translation helpful? Give feedback.
-
|
You can update the entrypoint in your template. |
Beta Was this translation helpful? Give feedback.
-
|
are there any docs on this? |
Beta Was this translation helpful? Give feedback.
-
|
This depends on the template and how it is running the Coder agent coder/examples/templates/kubernetes/main.tf Line 273 in 5337a70 to command: ["sh", "-c", "sudo update-ca-certificates && ${coder_agent.main.init_script}"] |
Beta Was this translation helpful? Give feedback.
Yes, you need to have the certs available in the pods image.