We have upgraded our Kubernates Service cluster on Azure to latest version 1.12.4. After that we suddenly recognize that pods and nodes cannot communicate between anymore by private ip :

kubectl get pods -o wide -n kube-system -l component=kube-proxy
NAME               READY     STATUS    RESTARTS   AGE       IP           NODE
kube-proxy-bfhbw   1/1       Running   2          16h     aks-agentpool-16086733-1
kube-proxy-d7fj9   1/1       Running   2          16h    aks-agentpool-16086733-0
kube-proxy-j24th   1/1       Running   2          16h    aks-agentpool-16086733-3
kube-proxy-x7ffx   1/1       Running   2          16h   aks-agentpool-16086733-4

As you see the node aks-agentpool-16086733-0 has private IP . When we try to check logs on pods which are on this node we got such error: Get https://aks-agentpool-16086733-0:10250/containerLogs/emw-sit/nginx-sit-deploy-864b7d7588-bw966/nginx-sit?tailLines=5000&timestamps=true: dial tcp i/o timeout

We got the Tiller ( Helm) on this node as well, and if try to connect to tiller we got such error from Client PC:

shmits-imac:~ andris.shmits01$ helm version
Client: &version.Version{SemVer:"v2.12.3", GitCommit:"eecf22f77df5f65c823aacd2dbd30ae6c65f186e", GitTreeState:"clean"}
Error: forwarding ports: error upgrading connection: error dialing backend: dial tcp i/o timeout

Does anybody have any idea why the pods and nodes lost connectivity by private IP ?

  • 1
    Crossposting is not allowed on the Stack Exchange network. Please delete one of the posts. – Sven Feb 8 at 10:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.