I'm new to the IT industry and I'm definitely not a sysadmin (I'm a road construction technician by trade). So please correct me if I mix up terminology, sysadmin concepts, etc.
My organization's IT department is very small. So small, that we seem to be barely treading water when it comes to keeping our IT systems "up".
The thing that seems to cause us the most problems is Windows Server updates. I'm told that the updates aren't totally automatic (they don't install the minute Microsoft releases them), but they are scheduled to happen automatically on a monthly basis -- on weekdays after-hours.
We come across issues like the following:
The virtual application server that contains the WebSphere JVMs for our work order management system was updated last night (via the 1-month schedule).
Today, when users started using the work order management system, we got a bunch of integration/java errors from the JVM.
- This issue was clearly caused by the updates. We've never had that issue before, and it happened right after the updates were applied.
Our sysadmins restarted the server which seemed to solve the issue. We dealt with the failed integration messages, etc. and life goes on...at least until the next update happens.
That's just one example. We actually had a handful of systems that had similar issues after the update last night.
I know that my organization isn't the only one that has problems with windows updates. It seems to be pretty widespread. But my question is:
Is there some sort of basic technique for handling updates that my organization might have overlooked?
For example, it occurred to me that we could:
- Manually apply the updates on Saturday mornings
- Test the crap out of ALL our systems
- Have ALL weekend to deal with issues and restart servers if we need to (instead of doing it live during business hours).
- We could also consider restarting ALL servers each weekend, and maybe even do updates at that time, so that fewer updates were applied each time, which might be less risky.
The major drawback to the ideas above is manpower (money). We don't have it. It would take a significant shift in organizational thinking & budgeting to do those things, so I just wanted to ask: what's the right way to handle these updates? Could the above ideas solve the majority of our issues?