Noob question: Should you tell your (outsourced) IT department to auto-update

Question

If you are outsourcing your IT, and you have servers set up with softlayer or server beach or other place, how do you recommend handling updates to Windows as well as software running on those machines?

Do you ask them to automate it and apply all the updates? Do you ask for a manual update where they assess the impact of the updates on your running software and applications (say your .Net based site, or SQL Server)? Most importantly, do you ask them to immediately apply the updates as soon as they are available, so that you're more secure, or do you wait to see if the update caused problems with other servers first? Assume that we don't have any test servers - just a bunch of production servers.

We're having some trouble deciding on a good approach, and can't have a dedicated IT department or even person.

Answer 1

I can't think of a reason where it would ever be okay to auto-update a production server without first testing the impact of the update.

Edited to add:

With the advent of virtualization, it's made it fairly easy for companies to have test environments at a relatively low-cost and ease of setup without having to purchase 2 of every server to duplicate the production environment.

If you absolutely cannot have test servers, then I suggest you test patches to the least important servers in your company. Pick servers that you can afford to be without should they have to go down for a rebuild.

But never, never, never auto-update a production server.

Answer 2

If you outsource, this decision is the responsibility of your service provider, IMHO. You pay them to maintain your systems, hopefully with an SLA. They will have to decide how to fulfill this agreement.

On a more general note, just blindly applying all patches is a disaster waiting to happen, because you can never be sure that a patch doesn't create a more serious problem than the one it should solve. So you have to differentiate: An emergency patch that (as an example) fixes a hole that would anyone allow to break into your IIS server has to applied immediately of course, but if you are not immediately affected, I consider it good practice to at least wait a bit and see if others have problems with a given patch if you can't test it yourself (which of course is the best way). In a few words: It depends...

Answer 3

Go into it aware of the risks. Good apps have been hurt by Windows Update (see Skype 2007). Earlier this year we saw

Botched McAfee update shutting down corporate XP machines worldwide

http://www.engadget.com/2010/04/21/mcafee-update--shutting-down-xp-machines/

The funny thing is that anti-virus signatures are probably something everyone, even those companies that have official "no-auto-update" policies, are auto-updating.

But you described "no test servers", so I'm guessing you're not at an org that would be doing testing anyway, even given the opportunity. And the outsourced IT department isn't in the best position to determine if an update would have negative impact on your application.

So perhaps the best approach, given your apparent resources, is to ask for regular software / operating system inventory from them with, with details of what revision and patch level things are at. When things break, you'll stand a chance of what updates broke it. Auto-updates will likely not take you to major version upgrades, so you can still coordinate those.

And get yourself some test servers! :)