(Followup on GKE pod connecting via VPN?)

I am trying to connect a GKE cluster to a remote network using a GCE VPN to a Cisco ASA 5510. Ping from GKE pod -> remote node arrives at and the ASA says that the echo reply goes back through the tunnel to GKE. However, tcpdump on shows no replies coming in.

Firewall and routing as reported by Google Cloud Console:

Name    Source tag / IP range   Allowed protocols / ports   Target tags

default-allow-icmp   icmp    Apply to all targets
default-allow-internal   tcp:1-65535; udp:1-65535; icmp  Apply to all targets
default-allow-ssh   tcp:22  Apply to all targets
gke-zecluster-d6cc7a55-all   tcp; udp; icmp;     Apply to all targets
gke-zecluster-d6cc7a55-ssh  <public_ip>/32  tcp:22  gke-zecluster-d6cc7a55-node
gke-zecluster-d6cc7a55-vms   tcp:1-65535; udp:1-65535; icmp  gke-zecluster-d6cc7a55-node
k8s-fw-a1a92183fb18e11e5be3442010af0001   tcp:80,443  gke-zecluster-d6cc7a55-node
k8s-fw-a1aa3fe95b18e11e5be3442010af0001   tcp:2003    gke-zecluster-d6cc7a55-node

Name    Destination IP ranges   Priority    Instance tags   Next hop

default-route-3eed071cad0670e8   1000    None    Default internet gateway
default-route-7a9ddc4457c714a0   1000    None    Virtual network
gke-zecluster-d6cc7a55-7b61213c-b187-11e5-be34-42010af00015   1000    None    gke-zecluster-d6cc7a55-node-j4jx (Zone ze-zone-1)
gke-zecluster-d6cc7a55-7ec5f7a9-b187-11e5-be34-42010af00015   1000    None    gke-zecluster-d6cc7a55-node-rluf (Zone ze-zone-1)
vpn-1-tunnel-1-route-1    1000    None    

Is there some logging I can turn on to see what goes on? As far as I can see, the VPN says nothing pertinent about this traffic, only:

15:24:51.058 sending DPD request
15:24:51.058 generating INFORMATIONAL_V1 request 3069408857 [ HASH N(DPD) ]
15:24:51.058 sending packet: from <gce-vpn-ip>[500] to <asa-ip>[500] (92 bytes)
15:24:51.092 received packet: from <asa-ip>[500] to <gce-vpn-ip>[500] (92 bytes)
15:24:51.092 parsed INFORMATIONAL_V1 request 146600869 [ HASH N(DPD_ACK) ]

If I modify the VPN tunnel (GCE VPN, ASA) to have the default net at the GCE end traffic passes correctly in both directions.

I assume this is a routing issue, but what? Should not the route send the traffic back to the GKE node? Or do I have to somehow declare the GKE network as a network?

  • Can you update your post by adding the Network column to the list of firewall and routing rules? – Kamran Feb 5 '16 at 2:55
  • They all list "default" which is – Bittrance Feb 5 '16 at 6:24
  • So those firewall rules will be applied to incoming traffic to your default network ( There is no firewall rule to allow icmp traffic to network. – Kamran Feb 5 '16 at 23:05

If IP address belongs to a GKE node, then for doing ping between the GKE node and your remote node you will need to add a firewall rule on network to allow incoming traffic to GKE node or all targets in that network from your remote source.

  • Note the default-allow-icmp. Should allow ping back in, no? – Bittrance Feb 16 '16 at 5:45

In the end, I had to pick a different option. Setting the spec.hostNetwork option pushed the pod into the node address space, for which the VPN worked fine.

As far as I can tell, when you create a GKE cluster, there is some "magic" networking set up for the pod address space, which appears not to have correct routing as regards VPNs. It is possible that Karman is correct, but I can find no way to declare an explicit virtual network for the pods to stick the firewall rules on. Simply sticking them on the default network does not seem to help.

Creating a new non-legacy network does not help as GKE refuses to create a cluster with pod address in an existing virtual network and GCE SDN refuses to create virtual subnetworks for an address space that GKE has already claimed.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.