0

I used to use NVIDIA GPU T4 in Google Cloud zone europe-central2-b before, but now it is not available since about 4 days ago. Is that normal for such a long time, or does it happen that google removes GPU's? I have already transferred my disk from other region where NVIDIA became unavailable long term and moving data to other region costs money. I would like to avoid that. So if GPU is not available for 4 day, should I keep trying or should I give up?

5
  • 1
    See cloud.google.com/compute/docs/troubleshooting/… but the generic recommendation is "try in a different zone". I can only assume but it is a safe bet that with the ongoing AI hype supply simply can't keep up with demand and even GCP can be temporarily sold-out.
    – HBruijn
    Feb 12 at 14:30
  • Despite the fact Google sponsors and/or links to ServerFault, it is not Google support. The ServerFault community can only help with on-topic questions but can't resolve capacity issues, billing issues, outages and similar problems. When you do have a paid support plan maybe ask your questions there: cloud.google.com/support/docs/manage-cases and they can give you an insider perspective.
    – HBruijn
    Feb 12 at 14:35
  • When you are using the GPU, what is the error message that you are having?
    – Yvan G.
    Feb 12 at 15:07
  • Thanks for responses. I was not asking for the solution, rather about the experience, how long I should wait before giving up. The error I am getting is standard capacity issue: "The zone 'projects/.../zones/europe-central2-b' does not have enough resources available to fulfill the request. Try a different zone, or try again later." But I am not going to migrate to other zone every few days, its expensive. Does anyone know is AWS has similar capacity issues? I have read that never,
    – Jan
    Feb 12 at 19:42
  • In GCP I have seen this a number of times, in different zones/regions, and with regards to different instance types. I have never seen anything like that with AWS, with which I worked many years. I don't know if there is any max. duration this may last. Probably the most "GCP way" to go about it is to be prepared and switch to another region when this happens. Not an ideal situation, and probably a bit of a surprise, in particular if one comes from AWS or Azure background.
    – ipastusi
    Feb 13 at 17:56

1 Answer 1

0

If you have a designated person that processes your account, you may contact them for faster reservation. The other way is you can create reservations on the console to reserve resources that are needed for your workloads. In this way, you can mitigate transferring to another zone/region.

To verify your issue, you can check your logging with error message ZONE\_RESOURCE\_POOL\_EXHAUSTED that indicates the resources you request is currently not available on the zone or region.

Additionally, you may read this thread regarding resources that are not available on a certain region/zone.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .