We have kubernetes running on a cluster in google cloud. Something happened to the cluster, and all of the nodes had the status
node.kubernetes.io/unreachable:NoSchedule. We attempted to delete the cluster, but it is not working.
The instance groups all have the status
This instance group is transforming.
The instances all have the status
This instance is being deleted
The kubernetes clusters all have the status
This cluster is being deleted
but when I click on a cluster it also says either
timed out waiting for Google Compute Engine operation: operation-1554925944268-58632688f6552-473a295c-305f8d99
Googleapi: Error 400: There is a peering operation in progress on the local or peer network. Try again later., badRequest.
It took a long time for the VM instances themselves to get deleted, but eventually they did.
However, the groups and clusters are not being deleted.
If I manually try to delete an instance group it says
Google Compute Engine is not ready for use yet in the project. It may take several minutes if Google Compute Engine has just been enabled, or if this is the first time you use Google Compute Engine in the project.
which makes no sense whatsoever.
Any help debugging this situation would be greatly appreciated.