Suddenly, many users of this organization's systems began reporting that they had been locked out of their Active Directory accounts.
The entire network is physical and local. There is no direct internet access. There is a firewall that connects the local network via VPN to a broader corperate network. Here are the components of the local network:
NY: Active Directory Domain.
server1: Windows Server 2008 R2; Active Directory Domain Controller for
NY; Remote Desktop Licensing Server; HyperV VM Host; only administrators are allowed to connect.
server2: Windows Server 2008 R2; Remote Desktop Session Host; member of
NY; users connect to this machine regularly to work on documents.
NL: Windows XP; VM running on HyperV hosted on
server1; member of
NY; used by only a select few users (maybe 3) at regular times each week.
FINXFER: Windows XP; VM running on HyperV hosted on
server1; member of
NY; runs a propriety piece of software that transfers data over the network at regular intervals.
- Various other Windows XP VMs running on
server1that mostly run background services and aren't usually connected to using Remote Desktop.
- Various HP thin client machines used for Remote Desktop connection to the above.
I started by looking at the event log on
server1 (the domain controller). I filtered for event 4740 "A user account was locked out" and found that there was an occurrence of this event once every 2 to 3 minutes:
Each occurrence of the event looks like the following:
A user account was locked out. Subject: Security ID: SYSTEM Account Name: SERVER1$ Account Domain: NY Logon ID: 0x3e7 Account That Was Locked Out: Security ID: NY\JoeSmith Account Name: JoeSmith Additional Information: Caller Computer Name: NL
Each occurrence has a different Active Directory username, but the rest is the same in every case.
This is an immediate red flag for me, as the frequency and repetition of the lockouts seem to suggest that someone or something is running down a list of usernames and trying to guess passwords until those users are locked out.
I notice that each of the events contains the line
Caller Computer Name: NL, the Microsoft documentation for 4740 say that it contains:
the name of computer account from which logon attempt was received and after which target account was locked out.
As far as I can tell, this means that someone or something is trying to log in to
NY credentials, or that something on the
NL machine itself is trying to authenticate using
To try and find the source, I enabled the following audit policies under "Local Security Policies" on
As a temporary fix, we started keeping the
NL VM off when not in use, and the lockouts do stop while the machine is offline. This has been going on for several weeks now.
Recently I left the
NL VM online overnight to let the logs build up, and what I found didn't look good. I found the following two events occurring on
NL multiple times a second, throughout the entire night:
Event Type: Failure Audit Event Source: Security Event Category: Account Logon Event ID: 680 Date: 1/20/2020 Time: 8:31:24 PM User: NT AUTHORITY\SYSTEM Computer: NL Description: Logon attempt by: MICROSOFT_AUTHENTICATION_PACKAGE_V1_0 Logon account: Finance Source Workstation: FINXFER Error Code: 0xC000006A
Event Type: Failure Audit Event Source: Security Event Category: Logon/Logoff Event ID: 529 Date: 1/20/2020 Time: 8:31:24 PM User: NT AUTHORITY\SYSTEM Computer: NL Description: Logon Failure: Reason: Unknown user name or bad password User Name: Finance Domain: FINXFER Logon Type: 3 Logon Process: NtLmSsp Authentication Package: NETWARE_AUTHENTICATION_PACKAGE_V1_0 Workstation Name: FINXFER
After these repeated a number of times, I would finally get this event:
Event Type: Success Audit Event Source: Security Event Category: Account Management Event ID: 644 Date: 1/20/2020 Time: 8:31:29 PM User: NT AUTHORITY\SYSTEM Computer: NL Description: User Account Locked Out: Target Account Name: Administrator Target Account ID: NL\Administrator Caller Machine Name: FINXFER Caller User Name: NL$ Caller Domain: NY Caller Logon ID: (0x0,0x3E7)
This is slightly different than I expected, because it's only trying the "administrator" login instead of random users, which is what I was seeing before. Of course, the administrator account can't actually get locked out, so if this is an attacker it makes sense that that's the user it would try to guess.
FINXFER VM does transfer information over the network, but it should not be doing anything with the
NL machine and definitely not multiple times a second.
What other tools can I use, or what other logs can I search through/enable, in order to hunt down the source of these login attempts that are resulting in account lockouts? How can I find out what program on
FNIXFER is initiating the login attempts?