27

I am trying to use a nested loop in terraform. I have two list variables list_of_allowed_accounts and list_of_images, and looking to iterate over list list_of_images and then iterate over list list_of_allowed_accounts.

Here is my terraform code.

variable "list_of_allowed_accounts" {
  type    = "list"
  default = ["111111111", "2222222"]
}

variable "list_of_images" {
  type    = "list"
  default = ["alpine", "java", "jenkins"]
}

data "template_file" "ecr_policy_allowed_accounts" {
  template = "${file("${path.module}/ecr_policy.tpl")}"

  vars {
    count = "${length(var.list_of_allowed_accounts)}"
    account_id = "${element(var.list_of_allowed_accounts, count.index)}"
  }
}

resource "aws_ecr_repository_policy" "repo_policy_allowed_accounts" {
  count = "${length(var.list_of_images)}"
  repository = "${element(aws_ecr_repository.images.*.id, count.index)}"
  count = "${length(var.list_of_allowed_accounts)}"
  policy = "${data.template_file.ecr_policy_allowed_accounts.rendered}"
}

This is a bash equivalent of what I am trying to do.

for image in alpine java jenkins
do 
  for account_id in 111111111 2222222
  do 
    // call template here using variable 'account_id' and 'image'
  done
done

6 Answers 6

42

Terraform doesn't have direct support for this sort of nested iteration, but we can fake it with some arithmetic.

variable "list_of_allowed_accounts" {
  type = "list"
  default = ["1111", "2222"]
}

variable "list_of_images" {
  type = "list"
  default = ["alpine", "java", "jenkins"]
}

data "template_file" "ecr_policy_allowed_accounts" {
  count = "${length(var.list_of_allowed_accounts) * length(var.list_of_images)}"

  template = "${file("${path.module}/ecr_policy.tpl")}"

  vars {
    account_id = "${var.list_of_allowed_accounts[count.index / length(var.list_of_images)]}"
    image      = "${var.list_of_images[count.index % length(var.list_of_images)]}"
  }
}

resource "aws_ecr_repository_policy" "repo_policy_allowed_accounts" {
  count = "${data.template_file.ecr_policy_allowed_accounts.count}"

  repository = "${var.list_of_images[count.index % length(var.list_of_images)]}"
  policy = "${data.template_file.ecr_policy_allowed_accounts.*.rendered[count.index]}"
}

Since we want to create a policy template for every combination of account and image, the count on the template_file data block is the two multiplied together. We can then use the division and modulo operations to get back from count.index to the separate indices into each list.

Since I didn't have a copy of your policy template I just used a placeholder one; this configuration thus gave the following plan:

+ aws_ecr_respository_policy.repo_policy_allowed_accounts.0
    policy:     "policy allowing 1111 to access alpine"
    repository: "alpine"

+ aws_ecr_respository_policy.repo_policy_allowed_accounts.1
    policy:     "policy allowing 1111 to access java"
    repository: "java"

+ aws_ecr_respository_policy.repo_policy_allowed_accounts.2
    policy:     "policy allowing 1111 to access jenkins"
    repository: "jenkins"

+ aws_ecr_respository_policy.repo_policy_allowed_accounts.3
    policy:     "policy allowing 2222 to access alpine"
    repository: "alpine"

+ aws_ecr_respository_policy.repo_policy_allowed_accounts.4
    policy:     "policy allowing 2222 to access java"
    repository: "java"

+ aws_ecr_respository_policy.repo_policy_allowed_accounts.5
    policy:     "policy allowing 2222 to access jenkins"
    repository: "jenkins"

Each policy instance applies to a different pair of account id and image, covering all combinations.

2
  • 3
    Will Make you trouble if you want to extend the configuration, like add a new account or/and an image, than your resources will map to different indexes, however if deleting and recreating them is not a problem this works fine.
    – balazs
    May 23, 2019 at 7:35
  • 3
    @justin-grote has a point in his answer: in terraform 0.12 you will need to use the floor function anywhere you do divide, or else you will get an error about partial indexes. account_id = var.list_of_allowed_accounts[floor(count.index / length(var.list_of_images))]
    – chriscatfr
    Aug 18, 2019 at 19:39
12

The answers here do work (I used them initially), but I think I have a better solution using Terraform's setproduct function. I haven't seen many examples of it used around the interwebs, but setproduct takes two sets (or more importantly, two lists) and produces a list of sets with every permutation of the inputs. In my case I am creating SSM parameters:

variable "list1" {
  type    = "list"
  default = ["outer1", "outer2"]
}

variable "list2" {
  type    = "list"
  default = ["inner1", "inner2", "inner3"]
}

locals {
  product = "${setproduct(var.list1, var.list2)}"
}

resource "aws_ssm_parameter" "params" {
  count     = "${length(var.list1) * length(var.list2)}"
  name      = "/${element(local.product, count.index)[0]}/${element(local.product, count.index)[1]}"
  type      = "String"
  value     = "somevalue"
  overwrite = false
  lifecycle { ignore_changes = ["value"] }
}

This creates SSM parameters named:

/outer1/inner1
/outer1/inner2
/outer1/inner3
/outer2/inner1
/outer2/inner2
/outer2/inner3

My wimpy little brain can parse this a little easier than the modulo magic in the other answers!

3
  • I'll try your solution. I agree it seems far better. But why do you use ${length(var.list1) * length(var.list2)} instead of ${length(local.product)} for the count?
    – chriscatfr
    Aug 18, 2019 at 20:21
  • I'll have to wait until my customer starts using v0.12 :( no wonder why you didn't find many sources.
    – chriscatfr
    Aug 18, 2019 at 20:28
  • No reason, ${length(local.product)} probably makes more since. Also, I'm fairly certain setproduct() exists pre-0.12, (the message at the top of the linked page is just a generic warning for all of their 0.11 docs, I think?)
    – user529562
    Sep 2, 2019 at 17:39
7

FYI if anyone comes here from Google, if you are using terraform 0.12, you will need to use the floor function anywhere you do divide, or else you will get an error about partial indexes.

account_id = var.list_of_allowed_accounts[floor(count.index / length(var.list_of_images))]

4
  • I wish i'd read all the way down the SO page to discover this gem before i tried the math approach. This is how I got it to work with floor(count.index / 8). Thanks for posting.
    – bytejunkie
    Jul 2, 2019 at 8:29
  • with 0.12 setproduct() from @kyle 's solution seems easier.
    – chriscatfr
    Aug 18, 2019 at 21:06
  • If you're on Terraform 0.12, then why not use the newly added for, for_each and/or dynamic nested blocks language constructs to implement something a little less confusing?
    – TrinitronX
    Aug 28, 2019 at 18:01
  • I too tried this solution. “floor” works fine but when you increment count by adding more resources, the indexes get messed up & destroys few unintended resources. Highly not recommended.
    – harshavmb
    Feb 13, 2020 at 21:49
1

While solutions with setproduct and using modulo work in some cases, there is a more elegant solution: using terraform modules (not to be confused with the modulo operator). By using a terraform module an extra opportunity arises to use the count meta-argument.

This solution is especially useful if not all combinations of inputs apply. For example, I used this solution to handle a list of projects, where each project has a list of unique roles.

For example, on the top-level you can use this:

variable "account_images" {
  type = map(list(string))
  default = {
    "1111" = ["alpine", "java"],
    "2222" = ["jenkins"]
  }
}

module "account" {
  source = "../accounts"
  count = length(var.account_images)
  account = keys(var.account_images)[count.index]
  images = var.account_images[keys(var.account_images)[count.index]]
}

And then have an additional terraform module in the ../accounts directory with this content.

variable "account" {
  type = string
}

variable "images" {
  type = list(string)
}


data "template_file" "ecr_policy_allowed_accounts" {
  count = length(var.images)}

  template = "${file("${path.module}/ecr_policy.tpl")}"

  vars {
    account_id = var.account
    image      = var.images[count.index]
  }
}

I find this approach easier to understand.

0

Basically the problem is in the data "template_file", the account_id can not be set the way you think it will since the count in your case is just another var that never gets incremented/changed. Just saying since I miss to see what is your question exactly.

0

I don't have enough reputation points to add a comment to the answer provided by @Martin Atkins, so I am posting his answer with a slight modification, which works around Terraform issue 20567

variable "list_of_allowed_accounts" {
  type = "list"
  default = ["1111", "2222"]
}

variable "list_of_images" {
  type = "list"
  default = ["alpine", "java", "jenkins"]
}

# workaround for TF issue https://github.com/hashicorp/terraform/issues/20567
locals {
  policy_count = "${length(var.list_of_allowed_accounts) * length(var.list_of_images)}"
}

data "template_file" "ecr_policy_allowed_accounts" {
  count = "${local.policy_count}"

  template = "${file("${path.module}/ecr_policy.tpl")}"

  vars {
    account_id = "${var.list_of_allowed_accounts[count.index / length(var.list_of_images)]}"
    image      = "${var.list_of_images[count.index % length(var.list_of_images)]}"
  }
}

resource "aws_ecr_repository_policy" "repo_policy_allowed_accounts" {
  count = "${local.policy_count}"

  repository = "${var.list_of_images[count.index % length(var.list_of_images)]}"
  policy = "${data.template_file.ecr_policy_allowed_accounts.*.rendered[count.index]}"
} 

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .