up vote 1 down vote favorite
share [g+] share [fb]

We are currently hosting two terminal servers for our users to connect to via Citrix Metaframe. One of the applications we host there is an inhouse application that was programmed a long time ago, specific to the warehouse operations of our company. Sometimes the users will run queries in this application that crashes the app, leaving it to consume 95-100% CPU. The user might then open a second entity of the app, only to crash that one aswell later that day.

I am monitoring the servers CPU usage via SNMP polling. So everytime an alarm tells me the server is under heavy load, I am manually logging on as admin to kill the crashed process. This happens roughly 2 times every day. The terminal server becomes crippled until the proccess is killed (even takes 2-3 minutes to log on), affecting all other users badly.

Is there any tool or procedure I can implement to help me with this? I need it to check for high CPU usage in specific processes. If high CPU is found for the application and is observed for X amount of time, KILL IT! (can't have it killing the app if CPU usage temporarily spikes).

Have tried several "Auto kill" tools, but have found none that can target specific .exe files for CPU over time.

link|improve this question
Thinking about this, what's the server hardware and what is the process doing to lock it up? If it's strictly CPU on a modern server with multiple multicore hyperthreaded processors, something seems wrong - I wouldn't expect an old and likely single-threaded app to be able to bring such a server to its knees unless there's another bottleneck somewhere, whether it be RAM consumption, disk queue (time to rearrange) or something else I'm not thinking of offhand. – fencepost Apr 22 '11 at 16:06
feedback

2 Answers

up vote 1 down vote

I don't have a specific tool recommendation for what you want, but at least as a quick-and-dirty you can probably cobble something together with a couple of the Sysinternals tools. You might be able to run this non-interactively (scheduled), but it definitely wouldn't be a service.

  1. Use Procdump to monitor CPU usage of processes with the appropriate process name (based on the executable). When a process hits the configured levels, generate a dump. Possibly start this from a looping .cmd file with a slight wait so each instance of procdump will only dump a single time, or possibly use the -n option to specify a very large number of dumps before it exits.
  2. Monitor your dump output directory for new files. Parse out PIDs as required, then kill the files or not - it doesn't sound like you're going to be debugging and fixing the app.
  3. Use PsKill to kill the relevant process.

As with anything cobbled together, this may or may not work well or be stable in the long run, but if the preferred option (fix the problem app) isn't viable then you're reduced to the second string anyway.

The Sysinternals tools are now available from Microsoft at http://technet.microsoft.com/en-us/sysinternals/default

link|improve this answer
feedback
up vote 1 down vote

Windows System Resource Manager (WSRM) is supposed to help you with this: http://technet.microsoft.com/en-us/library/cc732553.aspx (terminal service configuration here)

I've actually never had to use it, but it claims to be able to create policies to control memory and cpu resources for applications and services. Here's an article aimed at srv 2003

link|improve this answer
Might not be applicable to the asker's environment - for 2003, it only works on Enterprise and Datacenter editions. The download for Windows Server 2003 is at microsoft.com/downloads/en/… – fencepost Apr 22 '11 at 16:13
Indeed, thanks for pointing that out! – l0c0b0x Apr 22 '11 at 17:49
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.