Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I've noticed that my AWS server occasionally starts using a bunch of CPU for no particular reason, looking something like this:

enter image description here

Observe that it does not occur at specific times, but has a very definite pattern to it. It lasts just under an hour.

Remoting to the machine during this occurrence would invariably make it stop happening. Leaving the account permanently logged on allowed me to capture a more fine-grained CPU usage trace. It looked like this:

enter image description here

That's right; the processes that actually consume that CPU are not in the list. Instead, they appear and disappear all the time. ProcMon was obviously the tool for the job, so I captured a trace. This is what I found:

enter image description here

There's also Postgres involved:

enter image description here

However all the CPU usage is by the Winlogon/LogonUI/etc:

enter image description here

Here's a short excerpt of process start and stop events during this occurrence:

enter image description here

Note that postgres is not interleaved with each start/stop of smss/winlogon/etc, but only some of them.

Any ideas why this happens, and how to prevent it?

share|improve this question
1  
you should check your event logs for time-correlated events –  the-wabbit Feb 25 '12 at 20:47

3 Answers 3

For the postgres part this is because postgres creates a process - not a thread - for each session. This is quite costly on windows ( but rather efficient on unix systems ).

Winlogon / LogonUi part this is rather strange. Is the server remotely accessible? Could there be a network scanner on the network which would try to open port 3389 on the server and thus span a rdp session, which would explain the smss / winlogon / logonui sequence? I think of a network scanner because the session is closed immediately.

So my guess for the bounty: you have a nmap process or some "network discovery" tool which scan ports on your network, or your server is open to the internet without firewall on port 3389 ( and maybe 5432 ).

share|improve this answer
    
What's a "session" in this context - an open connection to the database? –  romkyns Feb 25 '12 at 17:20
    
In the case of postgres a session is a connection to the database. Each remote client creates a process. –  Olivier S Feb 25 '12 at 19:00
up vote 1 down vote accepted

The problem was that someone was brute-forcing my RDP login. A secondary issue was that network level authentication was disabled, making each login attempt relatively CPU-expensive.

The solution was to change the RDP port away from 3389 to stop the brute force attacks, and to enable network level authentication to reduce the CPU cost of a logon attempt.

Tip #1, from syneticon-dj: check the event logs. These spikes were correlated with lots of logon failures, trying usernames like "john", "admin", "test", etc, each one with about 3-5 different passwords. They arrived 3-4 seconds apart.

Tip #2, from Olivier S: this server, being an Amazon EC2 instance, requires RDP. The real problem was that by default, EC2 machines have Network Level Authentication disabled, for some reason. This means that every time someone wants to attempt a password, an entire logon UI is spun up, just to present them with a pretty remote desktop session. This is what caused all the CPU usage.

share|improve this answer

I found the answer to this was 15 people trying to brute force my RDP on port 3389.

Open up a command prompt and type netstat -n look for your IP:3389 if there is more than 1 connection that's not yourself then somebody is trying to get in.

Solution to stop near 100% CPU was change the default 3389 to something else.

You can google a solution for this, the port is stored in the registry

You might also need to mod your firewall rules accordingly

This cured my problem and I have my CPU back!

share|improve this answer
    
Yes, that's the same issue I had and the same fix too. I could have explained it a bit better. Another way to confirm it is to check the event log, it will have an entry for every failed login. –  romkyns Jul 21 at 11:33

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.