I have a Linux server setup (CentOS7) and have a handful of cronjobs running on there. Nothing too heavy duty, but have run into an issue twice in the past week.
It's a ruby process (Rake task, rather) then runs every 15 minutes on the hour. This tasks runs a query from a remote SQL server and generates text files per record. Every 15mins there could be anywhere from 5 to 20 text files generated. These text files (very small in size) are then sent to a Windows shared server where they are processed by a separate application.
What happened earlier and I am beside myself as to why this would not only spawn 4 new Ruby processes (based on the (2) Rake tasks that should be run, not just completely random ruby processes), but absolutely destroy all memory on the Linux server and we had to do a hard reboot. I did a small
git commit to the CentOS server (literally changed one line of code), and it spawned 4 new processes, all Ruby, and killed the memory on the server.
I'm not some bonafide server admin so any insight on how to avoid this in the future would be appreciated. From what I'm seeing, this spawn/hangup issue is only happen when a Github commit happens.
Ok, it doesn't look like the
git commit is the root of the issue, but the rake task being run as a cron job.
Here's is the cron job:
0,15,30,45 * * * * /bin/bash -l -c 'cd /var/www/heatfst && RAILS_ENV=production bundle exec rake send_fst_tickets --silent >> /var/www/heatfst/log/cron/standard.log 2>> /var/www/heatfst/log/cron/error.log'
When it runs and I check cpu% with
top, it spikes up to ~40% cpu memory and just hangs out there for a bit, never going back to normal levels (1-2%).