After migrating all our Kubernetes workloads to private clusters, we've found that the SD metrics explorer is no longer receiving any data. Although we have cloud NAT setup to work with all subnets, pods in kube-system are unable to connect to the Google API server. In fact, they are unable to make outbound TCP connections to anything on the internet. However, they can ping internet addresses. I've been unable to find a firewall rule that would negate the default allow for egress traffic. Looking for any solution or ideas for additional troubleshooting.

  • Can you show us the iptables set in the pods in that namespace? – Maciek May 9 '19 at 15:36

From this another serverfault question, you can see that by default Unable to access internet on pod in private GKE cluster private GKE cluster do not have external IP addresses.

  • 1
    That was correct when that question was answered. Google has since provided the cloud NAT feature that provides outbound traffic capability for private addresses. For namespaces other than kube-system, outbound traffic is working. For kube-system, outbound ICMP is working, but not TCP. – Chad Campbell Apr 24 '19 at 13:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.