Does Azure support multiple Agent Profiles for Kubernetes with Container Service?

Question

I have deployed a kubernetes cluster on Azure using the following terraform configuration (sensitive info redacted):

resource "azurerm_container_service" "kubernetes" {
  name                   = "myacsname"
  location               = "West Europe"
  resource_group_name    = "someresourcegroup"
  orchestration_platform = "Kubernetes"

  master_profile {
    count      = 1
    dns_prefix = "xxxmaster"
  }

  linux_profile {
    admin_username = "xxxx"

    ssh_key {
      key_data = "${file("authorized_keys")}"
    }
  }

  agent_pool_profile {
    name       = "default"
    count      = 2
    dns_prefix = "xxxagent"
    vm_size    = "Standard_F1s"
  }

  service_principal {
    client_id     = "xxx"
    client_secret = "xxx"
  }

  diagnostics_profile {
    enabled = false
  }
}

The terraform documentation suggests that I can specify multiple agent_pool_profile blocks which makes a lot of sense since I'd like to bind certain pods to more or less powerful nodes. Since the default container service already sets the beta.kubernetes.io/instance-type annotation on the relevant nodes I see no issue there.

How ever terraform does not allow multiple agent_pool_profile blocks for me:

azurerm_container_service.kubernetes: agent_pool_profile: attribute supports 1 item maximum, config has 2 declared

I cannot seem to add more profiles through the Azure portal either. Is the lack of support for different agent pool profiles documented somewhere?

Answer 1

Jason Poon's answer is correct. The Azure Container Service itself does not yet support multiple agent pools, hence the error message.

The API is setup to allow them in the future which is why Terraform models it as supporting multiple agent pools. My guess is they allow you to specify more so that whenever ACS supports it, Terraform will immediately support it as well.

Answer 2

Deploying multiple agent pools is supported if you use acs-engine. Here's a basic Kubernetes template that you would feed acs-engine, but if you wanted to deploy multiple agent pool types you can modify the file such that it looks like:

"masterProfile": { "count": 1, "dnsPrefix": "kubernetes-master", "vmSize": "Standard_D2_v2" }, "agentPoolProfiles": [ { "name": "agentpool2", "count": 3, "vmSize": "Standard_D2_v2", "availabilityProfile": "AvailabilitySet" }, { "name": "agentpool1", "count": 2, "vmSize": "Standard_DS2", "availabilityProfile": "AvailabilitySet" }

Unfortunately, I'm not familiar with terraform to know if it's do-able through that path.