and thank you beforehand for reading this. I'm pretty new to linux and have just migrated from a shared hosting to a vps for my shop. I set up nginx with fpm and varnish, and some cron jobs to import a product feed from my providers. Up to this point, all is well.
But I enabled the cron jobs too close to each other and now when they ran two nights ago, seems like they're importing 15,000 products non stop. The load has gone past 30 and I only managed to get it down by limiting the child processes in the fpm config. Now the load is down, but it uses up all the RAM and the SWAP (with errors due to lack of RAM and the VPS rebooting, getting me nowhere fast), and I have tried several things: kill pid, killall php-fpm, kill -9 pid, kill -15 pid but none work. As soon as I restart the php-fpm service, the child processes reappear and collapse everything.
I've rebooted, killed them for hours, but nothing. I cannot use the VPS because on every reboot they respawn and pop up again, rendering my shop useless.
Researched and tried all the kill examples I could find, but nothing works. I know tis because of my lack of expertise, but I really need help here diagnosing and fixing this, its been over 48 hours, and I need to kill these processes and not have them reappear.
Again, thanks in advance. Cheers
Output of ps auxf:
php-fpm: master process (/usr/local/php/etc/php-fpm.conf) www 8970 3.0 28.8 1435348 550676 ? R 22:58 0:05 \_ php-fpm: pool www www 8976 2.3 18.2 845740 347228 ? S 22:59 0:02 \_ php-fpm: pool www www 8978 2.9 18.2 860492 348092 ? S 22:59 0:02 \_ php-fpm: pool www www 8983 2.4 16.6 494556 317888 ? D 23:00 0:01 \_ php-fpm: pool www www 8986 3.2 8.9 276684 170424 ? R 23:00 0:00 \_ php-fpm: pool www
And initial cron jobs (all disabled at my first chance):
# 0 23 * * * wget -q -O /dev/null "http://regalosoutletonline.com/wp-cron.php?import_key=REGALOS&import_id=92&action=trigger" #PerfumeriaStockFEED Trigger once every 24h # 1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39,41,43,45,47,49,51,53,55,57,59 * * * * wget -q -O /dev/null "http://regalosoutletonline.com/wp-cron.php?import_key=REGALOS&import_id=93&action=processing" #OciostockFEED Processing once every 2min
I still didn't manage to find the culprit process, even after using pstree -p, which gave me the process tree and I could find the master:
But even killing the master did not fix the issue, as they reappeared.
Then I checked that all crons were deleted, not only disabled, and deleted the plugin and cleaned the db that still didn't help. Then edited my hosts file to block all calls to my provider's feed urls (downloading images), and that finally solved the present issue and got the site back up. I then singled out the one url that was the problem and things are back online now.
But the problem remains because as soon as I enable the url in the hosts file, even with the crons deleted and the plugin not present, the child processes shoot up again. Seems like there is some kind of registry for the php-fpm processes in the machine that makes them continue after a reboot or shutdown. So this is temporarily solved, but I cannot import from this one provider, even though everything is deleted related to it.