App Engine Flexible Deployment Issue: 403 Resource Error

Question

we have been trying to deploy our nodejs code to app engine flexible but unable to as it shows a Resource Error. We have GSuite Organization account and are using it for deployments.

This is the error:

Updating service [default] (this may take several minutes)...failed.           
ERROR: (gcloud.app.deploy) Error Response: [13] Flex operation projects/ae-test-3-280019/regions/asia-east2/operations/d727f9f7-3ee6-4b14-a7a4-9f04ef0aec68 error [INTERNAL]: An internal error occurred while processing task /appengine-flex-v1/insert_flex_deployment/flex_create_resources>2020-06-22T17:10:19.801Z36182.jc.6: Deployment Manager operation ae-test-3-280019/operation-1592845820141-5a8af514a9dd2-70733111-e1927105 errors: [code: "RESOURCE_ERROR"
location: "/deployments/aef-default-20200622t223948/resources/aef-default-20200622t223948"
message: "{\"ResourceType\":\"compute.beta.regionAutoscaler\",\"ResourceErrorCode\":\"403\",\"ResourceErrorMessage\":{\"code\":403,\"message\":\"The caller does not have permission\",\"status\":\"PERMISSION_DENIED\",\"statusMessage\":\"Forbidden\",\"requestPath\":\"https://compute.googleapis.com/compute/beta/projects/ae-test-3-280019/regions/asia-east2/autoscalers\",\"httpMethod\":\"POST\"}}"]

Key points:

• We were initially facing issues in billing and contacted support, they couldn't find a remedy so we created a new billing account and tried on it.
• The error occurs on all of our projects.
• We have all the necessary permissions, even the Organization Administrator as well as owner access.

app.yaml

runtime: nodejs
env: flex
instance_class: F1
automatic_scaling:
  min_num_instances: 1
  max_num_instances: 15
  cool_down_period_sec: 180
  cpu_utilization:
    target_utilization: 0.6
  target_concurrent_requests: 100

app.yaml in console -> App Engine -> Version -> Config

runtime: nodejs
api_version: '1.0'
env: flexible
threadsafe: true
instance_class: F1
automatic_scaling:
  cool_down_period: 180s
  min_num_instances: 1
  max_num_instances: 15
  cpu_utilization:
    target_utilization: 0.6
  target_concurrent_requests: 100
network: {}
liveness_check:
  initial_delay_sec: 300
  check_interval_sec: 30
  timeout_sec: 4
  failure_threshold: 4
  success_threshold: 2
readiness_check:
  check_interval_sec: 5
  timeout_sec: 4
  failure_threshold: 2
  success_threshold: 2
  app_start_timeout_sec: 300

Quota Usage Image

IAM Permission

What are we doing wrong on our end? We tried the simplest nodejs flex sample example in a new project in a new region in a new billing account yet the same issue. We tried multiple variations of app.yaml, but in vain.

Answer 1

I have tried to create an App in GAE Flex in asia-east2, but every time I try with this location, it fails, but this is due to my organization restrictions. I have created the app and successfully deployed in the region asia-northeast1.

Lets start narrowing down our issue: according to the GAE location documentation:

1. Once you have chosen the location for the App it can not be changed.
2. The asia-east2 should be a correct location.

We can discard the location given that your app has been correctly created. (Creating the app is different than deploying the app).

I would like to mention that GAE in the Flex environment does not use the tag instance_class: F1 in the app.yaml. Instead, and according to the app.yaml documentation, you have ti use resources, indicating cpu, memory_gb, disk_size_gb.

It is well worth to make sure your components are in the last version. To ensure this please run 'gcloud components update'.

I have also checked the GCP dashboard status to check if there was an outage with GAE, but it is just OK.

Your permissions seems to be OK. You have the owner role which is enough to deploy an app in GAE, and your Quotas are just OK

In order to discard code-issues, I would suggest:

1. Run the Quickstart NodeJS GAE Flex in a new Project.
2. In the step 1 is required to create an App, with the command gcloud app create --project=[YOUR_PROJECT_ID]. Please, be sure this step finishes OK.
3. Deploy your app.

I would add a last suggestion. Is there a chance for you to make a test without your organization? Just to discard that the issue is there?

If you face the same issue, it will worth to create a Private Issue in the Public Issue Tracker in the GAE section. There, given that is private, you can share more information, so they can prove further help.