3

I've found lots of information about the right way to approach patching windows servers (e.g. To update or to not update?), but I'm in a situation where most of that information is just not practical at the moment.

What I'm looking for is the least bad option in a resource-constrained scenario. Specifically, assuming these are my only options: should I just blindly apply "important" (as opposed to "optional") Windows Updates to production servers, or am I better off not applying them at all?

Here's a little context: I'm a DBA in a small shop with about 15 servers hosting SQL Server instances; the SQL boxes are the only ones I'm concerned with (i.e. I'm not worried about Exchange boxes, domain controllers, etc.). We're live 24 hours a day, but I don't have regularly scheduled maintenance windows (i.e. I need to explicitly schedule one with our ops/support group if I need to reboot, etc.).

I'm comfortable enough with managing a server to know my limits and not break anything, but I lack deep knowledge. I've talked to the network/server guys about this, but they essentially told me that we don't have a process or policy for this and that I should just use my best judgement. So they know it's an issue, but it's just one that's not going to be addressed anytime soon. I don't have the skills or time to implement a good solution by myself, I'm really just trying to minimize my risk as much as I can with the resources I have today. Philosophically, I'm inclined to apply the patches, but I'll admit that I likely don't fully understand the risks of doing so.

To sum up in the form of an answerable question: is it safer to patch blindly, or not patch at all?

I'm open to other options, but they pretty much have to be as easy and flexible as manually applying windows updates in a more-or-less ad hoc fashion. (that sucks, I know, but it's my reality at the moment and for the near future)

3
  • 1
    This question is more of an opinion piece instead of asking a real question with a definitive yes or now. The answer is of course it depends...on your environment, your backups, your tolerance for risk....
    – mdpc
    Mar 5, 2013 at 23:04
  • @mdpc - I agree, it's a borderline question at best. I tried to narrow the scope to reflect just my particular circumstances and options, but I'd understand if it gets closed.
    – user163107
    Mar 5, 2013 at 23:39
  • 2
    @mdpc - the tl;dr version of the question is "I don't have the resources to test WU in a staging environment; should I patch production without testing, or not?" - which is a good enough question I think Mar 6, 2013 at 1:23

3 Answers 3

8

or not patch at all?

This is simply not an option these days. Your system will almost certainly get compromised at some point if you do not apply the required security patches. The only real question is how long you can wait after a patch release until you must install it.

patch blindly

Ideally you would test, but if you can't test, then at least mitigate the potential damage by making sure you have a extremely well tested backup and restore system. So when an update does break things you can recover from it. Then you must be able to tolerate some down time for the updates to get applied.

3
  • I'm confident in my SQL backups, but I don't really have access to the server level backups. These are VMs though, and I do have a standard SQL build (script and documentation), so the worst case scenario for a bad patch is that I have to rebuild the server and restore data from backups. That's probably a decent estimate of an upper bound on potential down time, so that helps. Thanks.
    – user163107
    Mar 6, 2013 at 0:03
  • @user163107 - fwiw the only patch I blindly approved without testing first that caused me major major problems was an AU to install .NET 2 onto a terminal server, and it destroyed a major piece of software that not in support contract. That was because of an exceptionally shitty piece of software that had no reason to break because of a separate .NET runtime but did anyway. Mar 6, 2013 at 1:26
  • @user163107 The worst problem I've seen with a Windows security update in the last year was one that caused fonts to render strangely in PowerPoint. And that wasn't an Office update. MS pushed a fixed update 10 days later. Mar 6, 2013 at 4:35
1

In general, I usually apply the Windows update patches directly. Although I have excellent backups, and also I do have a restore point that I could fall back on. As there are generally significant security patches, I would not wait very long.

As for testing, well I have not found that I can generally well test anything with an array of differering items like Windows without a significant amount of time. And again, with a high security profile and excellent backups I am unable to wait. I have not generally had a problem. It is probably advisable to do your development servers first and work up to the production servers, just to be prudent.

One note, patches may open up a different irregularity then that you are experiencing.

Thus you have your answer .... it depends!

1
  • When you say you've "not generally had a problem", do you mean that applying security patches have not caused problems with with your servers or the apps they host? That makes me wonder if maybe my real question is one of understanding how risky it is to apply patches... i.e. i know "it depends" is a the right answer (I'm a DBA, efter all, ;-) I just don't have enough experience with windows server to evaluate the risk.
    – user163107
    Mar 5, 2013 at 23:52
0

From the comments you already have a good backup / restore process available via the snapshot capability of your hypervisor. With care these are indespensible for applying untested updates.

Zoredache's answer really covers your bases pre update. It doesn't cover the immediate aftermath. What you really need in addition is a ready way to put your systems through its paces once complete. It is one thing to have an update untested before it is installed. Leaving it untested once you have the capability to do something about it is quite another.

My suggestion on SQL is write up a mock application (assuming you have a front end) or a ready way to fire your common stored procedures and verify the results are as you expect. My experience with SQL related updates has always (so far anyways) centered on not haviglng the service start or one of the core functions of our applications having a problem. Having a mock has saved my bacon several times in the last 10 years.

Not the answer you're looking for? Browse other questions tagged .