Playing with provisioning of Kubernetes cluster on vSphere using Kubespray and Terraform to automate deployment process (no problems with that).

My goal is to have ability to connect Pods to different VLANs (or Port Groups), so then each app will run in it's own network like my VMs do.

I think there is 2 approaches: 1. Create 'All VLANs Port Group (0-4095)' and manage VLANs inside K8S node OS 2. Attach multiply NICs to K8S node VMs each of them will be connected to different VLAN

After googling for weeks I'm still can't find solution for similar requirements.

So my problem is that I'm stuck trying to figure out how to build K8S cluster this way and in the same time I'm not completely sure that I'm on the right way at all.

Please help me before I'm go mad!

  • 1
    you are probably on the wrong track. in general, it is not recommended to have multiple interfaces on a single pod. you can create multiple networks inside the cluster (that work as overlay networks across several compute nodes, too!). you can also decide how to expose certain services through ingress controllers. messing with the internal networking inside the cluster outside the standard functionality of k8s is going to end in tears. – rmalchow Jul 22 at 13:30
  • another things that isn't clear ist: what exactly are you tying to achieve? as in, what would be the purpose? – rmalchow Jul 22 at 13:31
  • @rmalchow >> you are probably on the wrong track - thanks for clarification! – tiv Jul 22 at 15:06
  • @rmalchow >> what exactly are you tying to achieve - I was trying integrate Kubernetes cluster into existing VMs \ Network infrastructure. now I have many VLANs for envs, infrastructure, etc. as example: I have network named "app1" here lives all VMs running app1, if I want to migrate part of app1 to K8S I want to preserve this approach for some reasons like firewall rules. so in one word - I want to minimize infrastructure changes during K8S integration. – tiv Jul 22 at 15:12
  • 1
    the separation you did with the VMs, you probably did that not just for the fun of it, but to achieve a certain goal. this goal can probably be achieved in k8s as well, albeit in a completely different way. – rmalchow Jul 23 at 8:09

This can be achieved with multus-cni plugin. It creates NetworkAttachmentDefinition custom resource, where you can specify network interface, i.e.

apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
  name: macvlan-conf
  config: '{
      "cniVersion": "0.3.0",
      "type": "macvlan",
      "master": "eth0",
      "mode": "bridge",
      "ipam": {
        "type": "host-local",
        "subnet": "",
        "rangeStart": "",
        "rangeEnd": "",
        "routes": [
          { "dst": "" }
        "gateway": ""

And then, you attach this configuration to the pod:

apiVersion: v1
kind: Pod
  name: samplepod
    k8s.v1.cni.cncf.io/networks: macvlan-conf

Furthermore, you can add more interfaces to a pod by creating more custom resources and then referring to them in pod’s annotation

More details

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.