Terraform & GCP: How to get inventory of current disks?

Question

My goal is to utilize terraform's "google_compute_snapshot" feature (https://www.terraform.io/docs/providers/google/r/compute_snapshot.html) to schedule daily snapshots for all disks in a project.

We will be making the assumption that all VM's will be created using terraform throughout the lifecycle of a project. How would I go about generating a list of available disks so I can pass this list to the "source_disk" argument?

Thanks!

Answer 1

When this post was created there wasn’t a feature in GCP to Scheduled snapshots, this feature was implemented in August of 2019 as GA.

Nowadays, it is very easy to schedule a snapshot by the UI or by gcloud.

For instance, in the UI you can follow the next steps:

1. In the Google Cloud Console, go to the Snapshots page.
2. At the top of the page, click Create Snapshot Schedule.
3. Enter a schedule Name.
4. Select a Region for the snapshot schedule. Your persistent disks must reside in zones that are within the region that contains the snapshot schedule.
5. Optionally, select your Snapshot location or accept the default selection.
6. Select your Schedule frequency.
7. Enter the number of days to Autodelete snapshots after.
8. Select your Deletion rule to determine what happens to snapshots if the schedule's source disk is deleted.
9. Choose the number of days to retain snapshots generated from this schedule.
10. If you are using Windows, you can Enable VSS.
11. Optionally, specify any relevant labels you want to apply to these snapshots.
12. Click Create to create the snapshot schedule.

I’ve followed the next documentation

Also you can use the compute resource-policies create snapshot-schedule gcloud command.

  gcloud compute resource-policies create snapshot-schedule [SCHEDULE_NAME] \
      --description "[SCHEDULE_DESCRIPTION]" \
      --max-retention-days [MAX_RETENTION_DAYS] \
      --start-time [START_TIME] \
      --hourly-schedule [SNAPSHOT_INTERVAL] \
      --daily-schedule \
      --weekly-schedule [SNAPSHOT_INTERVAL] \
      --weekly-schedule-from-file [FILE_NAME] \
      --on-source-disk-delete [DELETION_OPTION]

In this example, the snapshot schedule starts at 22:00 UTC and occurs every week on Tuesday and Thursday.

gcloud compute resource-policies create snapshot-schedule SCHEDULE_NAME \
    --description "MY WEEKLY SNAPSHOT SCHEDULE" \
    --max-retention-days 10 \
    --start-time 22:00 \
    --weekly-schedule tuesday,thursday \
    --region us-west1 \
    --on-source-disk-delete keep-auto-snapshots \
    --snapshot-labels env=dev,media=images \
    --storage-location US

On the other hand, we have the Resource: resourcePolicies You can use resource policies to schedule actions for some Compute Engine resources.

For example, you can use them to schedule persistent disk snapshots.

JSON representation:

{
  "id": string,
  "creationTimestamp": string,
  "selfLink": string,
  "region": string,
  "description": string,
  "name": string,
  "status": enum,
  "kind": string,

  // Union field policy can be only one of the following:
  "snapshotSchedulePolicy": {
    "schedule": {

      // Union field policy can be only one of the following:
      "hourlySchedule": {
        "hoursInCycle": integer,
        "startTime": string,
        "duration": string
      },
      "dailySchedule": {
        "daysInCycle": integer,
        "startTime": string,
        "duration": string
      },
      "weeklySchedule": {
        "dayOfWeeks": [
          {
            "day": enum,
            "startTime": string,
            "duration": string
          }
        ]
      }
      // End of list of possible types for union field policy.
    },
    "retentionPolicy": {
      "maxRetentionDays": integer,
      "onSourceDiskDelete": enum
    },
    "snapshotProperties": {
      "labels": {
        string: string,
        ...
      },
      "storageLocations": [
        string
      ],
      "guestFlush": boolean,
      "chainName": string
    }
  },
  "groupPlacementPolicy": {
    "vmCount": integer,
    "availabilityDomainCount": integer,
    "collocation": enum
  }
  // End of list of possible types for union field policy.
}

Additionally you can see the following post where someone added an example for GCP Snapshot scheduler

Answer 2

To generate the list of available disks on your project, you can run the gcloud command below:

gcloud compute disks list --format="csv[no-heading](name,zone)"

This command will bring the list as csv format including just the name and the zone without header, if you want it as Json or table it can be done as well. see the documentation

If you want to put the information on a file you need just to add >> [file name] at the end of the gcloud command.

You can see all possible fields by running

gcloud compute disks list --format=flattened