I have a long running process in Go on an App Engine flex instance, deployed via docker image.
Most of the time when I deploy to the live version it sends a SIGTERM to the app. I can catch this and do a graceful shutdown. It's great.
Other times, the process just seems to disappear and a new instance is created. I don't get any log output; no indication of what happened. This definitely seems to happen if I change the number of instances (via manual_scaling) but sometimes it happens on a normal deploy.
Is there a way to get a SIGTERM consistently? Are there other strategies I can use to know when the instance is being killed/restarted?
Update: I tried a few test cases:
- "Delete" instance in App Engine UI. The instance cleanly shut down - sending signals - and rebooted since it's configured to have one instance.
- Deploy, changing from 1 to 2 instances. Existing instance rebooted cleanly with signals. New instance came up.
- Deploy, changing from 2 to 1 instances. One existing instance rebooted cleanly with signals. The other one went poof for lack of a better description. Viewing 'All logs' showing STDERR from my app, then nothing. No output in vm.events, vm.syslog, vm.shutdown logs which report lots of interesting stuff during reboot. I also know that signals weren't received by my app because the database is left in a dirty state.
It's this last case that I'd love some more insight into, thanks! Please also let me know if there's a better place or way to ask this question.