Google App Engine Instance is abruptly shutting down on Standard Environment

Question

We are using Google App Engine in Standard environment to ingest large amount of data to Google Cloud Firestore with below configuration:

instance_class: B4

basic_scaling: 
 instances: 1

The overall data ingestion of 20GB takes around 1.5 hours. But we have noticed that some time after an hour, instance is abruptly shutting down with below error:

Container terminated on signal 9.

As per this How Instances are Managed documentation, basic scaling can serve the request up to 24 hours.

We can not see any more details in the logs as well. Also checked the memory usage, B4 has 1024 MB and the app is only utilizing up to 350 MB.

If anyone has faced this kind of error, your input would be valuable!

Answer 1

I think this question is more suitable for Stackoverflow as per StackExchange > Help Center > Asking documentation. Please post questions in the proper forum in the future as it will help in better and faster resolution of the issue.

However, I believe that you should take a look at the How Instances are Managed > Instance uptime documentation to configure your App Engine in a way it will be able to cope with such long processes.

Based on your app.yaml configuration file, I can see that you have setup the basic_scaling to 1 and in the documentation it is stated that

"App Engine attempts to keep manual and basic scaling instances running indefinitely."

however it also states that

"Hardware and software failures that cause early termination or frequent restarts can occur without prior warning and can take considerable time to resolve; thus, you should construct your application in a way that tolerates these failures."

Which means that you should be prepared for such behavior in anyway and you can follow the strategies mentioned in the same paragraph for how to deal with those occasions.