Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Having managed to kill sshd on a remote machine (by running a script which used all available memory in the machine, oops...) which I have no access to other than visiting the hosting for[1], I was considering ways to ensure that sshd is always kept running.

Other than a hacky cron job to restart sshd every n minutes or hours, using inittab to get init to keep sshd running seems like a good idea.

Are there any drawbacks from this approach? It would seem like something which it would be sensible for Linux distros to do by default, since sshd is often the only available method of access for a machine..

Additionally, are there any other daemons which I should be using this approach for? Perhaps a monitoring agent such as nrpe for nagios?

[1] Yes, management cards or a network power switch would be a good idea, but they were deemed "unnecessary" at the time...

share|improve this question
    
Ah, it seems that there is the terrible drawback that it won't work at all, due to sshd backgrounding itself (thanks katriel!). Disabling this is only possible via the -d (debug) option, which sets it to only one connection as well as putting it in the foreground. I'll have to go with an alternative init such as Upstart, runit or daemontools as suggested by a few people. At least until sshd gets an option to run in every way as a daemon but not to set itself into the background... –  David Gardner Jul 24 '09 at 20:25
1  
Good grief, the -D option (immediately above -d in the man page) does exactly what you want. –  womble Jul 24 '09 at 23:25
    
Doh! How did I miss that?... I'll certainly try this :) –  David Gardner Jul 25 '09 at 9:21

8 Answers 8

up vote 3 down vote accepted

There's a few implementations of this idea. Upstart is used by Ubuntu and can restart services if they die, Solaris 10 has the Service Management Facility, runit is cross-platform and there's daemontools as already mentioned.

share|improve this answer
    
To be more precise: Fedora uses upstart as well –  wzzrd Jul 24 '09 at 12:34

I can think of no other reason not to do the inittab thing, than that restarting sshd after an upgrade is a little more inconvenient.

Other than that: interesting idea.

share|improve this answer
    
Must be some reason why it's not done... security issues? user levels? can you run apps as anything other than root? –  sybreon Jul 24 '09 at 13:19
    
@sybreon, well, SSH does privilege separation, so after login, sshd spawns a process just for you under your account, even though the mother ssh process runs as root. Security shouldn't be an issue any more than it would be for agetty. Not sure what you mean by 'user levels'. And: hi! You here too? :-) –  wzzrd Jul 24 '09 at 13:39
    
@sybreon, my guess is it is much less convenient to restart sshd in this setup, so after an update, you are stuck with the old version (rpms tend to restart this service on update, debs too probably). Only way to restart it would be to kill it. On a sidenote I have never noticed an sshd crash. –  wzzrd Jul 24 '09 at 13:42

you can tell linux OOM killer not to kill sshd, google for oom_adj for more details, or see i.e. here rhel manual

share|improve this answer

There are benefits to having services that need to be reliable under a scheme that will ensure they're always running. I prefer to use daemontools, myself, for the reasons documented here: http://cr.yp.to/daemontools/faq/create.html

I've not run ssh this way, but I would be happy enough doing it if I was in the situation where I thought that my current SSH management wouldn't work. As far as your "running out of memory" problem, you can deprioritise certain processes like sshd so that they don't get killed by the OOM killer in favour of the program actually causing the problem.

share|improve this answer
1  
Any idea how you de-prioritise processes like sshd? –  Cian Jul 24 '09 at 12:16
2  
@Cian You alter the value in /proc/$PID/oom_adj, lower score -> decrease likelyhood of it being killed, higher-score -> increase likelyhood of it being killed –  wzzrd Jul 24 '09 at 12:33

An interesting idea.

I haven't tried anything like that but I would check at what time in the boot process the things from inittab are started. If it is too early you may not have the network running.

share|improve this answer
    
Depends what runlevel it is set to run at, so this should be OK. –  David Gardner Jul 25 '09 at 9:24

Monit is a monitoring daemon that is designed for just what you want to do here.

share|improve this answer

The only issue that I can foresee is if it were attempt to respawn with a configuration error.

I thought you could rate limit respawning, but I can't seem to find any documentation to support this.

share|improve this answer

Like other have noted before me, using an existing tool like daemontools or monit will probably be the smartest route. You can't use inittab to spawn sshd wince it forks to the background, and init will try to run several sshd's. You'll most likely get "init: re-spawning too fast" messages.

You might want to write a small monitoring script that will run in a loop and make sure the original sshd (the one that accepts connections and forks to handle the sessions) is still running. One it fails, just use the system's init script to re-run it.

Just a note, if your sshd is killed by the kernel's OOM handler, there'e no guarantee your sshd will survive a restart...

share|improve this answer
    
You can tell sshd not to background on start, and daemontools relies on that behaviour to work properly. –  womble Jul 24 '09 at 23:24

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.