5

There is a strange and confusing (to me and to users) issue plaguing authentication. I do not know how long it has been occurring, but I believe it to be quite a while. Only recently, with the use of the Account Lockout tool have I realized that these authentication issues are sometimes caused by a glitch in the system rather than user error.

What happens is that a user authenticates correctly, but the system rejects their password. I wish to repeat: they log in with the correct username, password, and domain. This is not fat-fingering; it is not a client issue; it is not user error; it is not an expired password; it is not specific to any service.

The behavior when a user correctly authenticates is that the DC resets the ‘failed login’ count back to 0. When they fail, it increments it and sets the ‘last fail’ time. But when this glitch occurs, neither happens; the authentication attempt is rejected, but the count does not go up by 1, nor does it reset, and the last fail time does not change.

The issue occurs across multiple devices and services. Today I had a student fail to log in on multiple computers, as well as webmail. I compared the event logs from the computer and the DC; I con see no difference between the events when the user was wrongly rejected (and the failure count did not go up) and when she was correctly rejected because I had her mistype the password on purpose.

I have done this myself, attempting to log in to a student’s freshly created account (using a known password scheme). I have had it happen to users on many of the services that authenticate through AD. It has happened to staff, faculty and students. As far as I can tell, this is an authentication issue directly on the DCs; something wonky with the account, but not one of the typical culprits of expired password, disabled, etc.

Resetting the password fixes the issue. The problem just goes away. But the frequency of the issue (about 8-10 cases just this week, out of at most 100 network password resets) leads me to believe it is a serious problem.

I do not know how long this issue has been occurring. Without using the Account Lockout tool, I would never have seen that the error count was failing to increment, and thus assumed that the user was wrong about knowing the password. I have had many occurrences where users swore they knew their login, and it ‘worked yesterday’. I do not know how many of these times it was true, if ever. Even after getting the tool, it has taken multiple occurrences and several months of the issue occurring before I believed it was a real problem. Not till I actually had it happen to me, typing the student’s initial password, and seeing the failure count fail to rise, did I really believe it.

Our AD environment is mostly on Windows server 2008. Some DCs are still Server 2003. The environment is a single domain. If there are any other relevant technical details necessary for troubleshooting, please let me know.

Edit: As the accepted answer shows, it really was user error. The event that 'proved' to me that it was a real problem was when I logged in to a newly created account and it failed without incrementing the bad password count. We have standards for new accounts and what to reset passwords to. Likely another admin ignored the standard and reset this user's password to something else. When I attempted to log in to the new user, and the bad password failed to increment (yet I was rejected), I thought it was proof of an issue. Much Googling failed to find a page describing the situations under which the Bad Password Count fails to rise... hopefully this answer will help someone else in the future.

5
  • You probably know the sequence of events, but just in case...how do you know that the auth attempts are even reaching the DCs? What error(s) are the clients getting? What do the security logs in the DCs show when they are attempting a login? Anything? If you run Wireshark on a mirrored port of their computer are you able to determine that traffic is going back and forth to the DC for the login attempts?
    – TheCleaner
    May 15, 2013 at 19:04
  • I have not done any packet inspection. However, my current diagnosis method is to have a user attempt to log in with a deliberately bad password; this attempt reaches the server, increments the fail count. I then have the user try the password they think it is. They observe identical results (rejection by email/webmail/computer/sharepoint/whatever). On my end, I observe the failure count FAIL to increase. I can see authentication failure events in the log of the DC. But the count didn't rise, because that was the correct password. 3/4 login problem calls currently are caused by this issue. May 16, 2013 at 0:19
  • can you post the DC security log failure events here as well as the error message/screen that clients get when they observe these results? Without those, it sounds like your best recourse would be to open a ticket with MS support and have them work through the issue with you.
    – TheCleaner
    May 16, 2013 at 1:56
  • The user error doesn't really matter, because this is system agnostic. They get rejected by email, computer, IIS, the VPN... any system authenticating via AD can raise this, and each system communicates a password failure differently. The DCs are reporting an authentication failure to whatever service attempts the login; how the service shows that to the user varies. The DC error logs are identical (to the best of my ability to compare) between a legit and a glitched authentication failure, but I'll try to post an example. May 16, 2013 at 2:03
  • I'm with @TheCleaner on this, post one of the failed logon events from the DC security log. It will give us an idea of why the DC might have rejected the attempt May 16, 2013 at 15:46

3 Answers 3

9
+300

Do you have password history enabled? If the password entered matches either of the two last passwords for the account, the auth will be rejected but badPwdCount will not be incremented. I'm trying to wrap my head around the rest of your description, but that would at least explain the "missing" bad password increment.

EDIT

Rereading your question, it sounds like administratively resetting passwords always has positive results, correct? Also wondering what OS your PDCe is on (2003, 2008). Are there any firewalls potentially blocking access to the PDCe (or any other DCs for that matter)? Keep in mind that while end-user password changes communicated from the client to the local DC via the kpasswd protocol (TCP/464), PDCe notification of password changes are via an RPC call. The destination ports will have changed from 2003 to 2008.

4
  • We do have password history enabled. I had no idea that badPwdCount did not increment in this case! This means there is no problem at all... just users forgetting that they changed their password! Thank you! Testing to see if this explains symptoms now... May 16, 2013 at 17:40
  • YES! Woohoo! Months of confusion solved! I love you Charles! May 16, 2013 at 17:45
  • 1
    That's great to hear. Was it really as simple as users attempting to use previous passwords? I thought you mentioned failures on brand new accounts. May 17, 2013 at 1:07
  • I did mention failures on brand new accounts. In fact, the one on a new account was my proof that it was 'real'. But what I suspect is this: Account created with initial password. User emails IT, someone resets to a different than standard password. User calls IT, talk to me, I try the default password... and the failure does not increment (because it is in the password history). So it was new, but not untouched, and I didn't realize. May 17, 2013 at 2:50
4

This smells like a problem with either your replication or the DC that has the PDC Emulator role.

Can you run netdom query fsmo on each DC and compare the results from each? Make sure they all think the same server holds the PDC Emulator role. Next, take a look at the output of dcdiag and see what it has to say. Also, verify replication with repadmin /showrepl against each DC.

If I had to take a completely blind guess, I'd say that there's either an inconsistency in who the DCs think hold the PDC Emulator role, or the server that once held it was improperly decommissioned and the role was never moved.

9
  • Should all these commands work on a 2003 DC? I am not sure if the path is just wrong, or if these are 2008 only commands... Jan 11, 2013 at 23:49
  • I'm not 100% sure of netdom, but the others should. Alternately you can run them all from a W7 workstation with RSAT and the AD command line tools enabled
    – MDMarra
    Jan 12, 2013 at 0:13
  • I never updated this question, but unfortunately this did not turn out to help; all of our DCs are accurately configured with the correct PDC. Any other recommendations you can provide? Apr 3, 2013 at 20:01
  • What about the outputs of the dcdiag and repadmin commands that I specified above?
    – MDMarra
    May 10, 2013 at 18:54
  • repadmin reports no replication issues (we also set a monitor to auto-recheck, and none ever pop up), and dcdiag reports no issues either. But accounts are still getting messed up... and this was not a single incident in the past, it is a persistent thing that glitches new users. May 14, 2013 at 18:37
0

This is not an issue, this is a feature: Enhancements were introduced for domains at Windows Server 2003 functional level and above. When the bad password matches either of the two most recent entries in password history, the badPwdCount attribute is not incremented and the badPasswordTime attribute is not updated. This means that normal users can make more attempts before they are locked out. Their bad password attempts are more likely to be passwords they recently used.

The meaning of the lockoutObservationWindow attribute is the same. But because badPasswordTime is not updated for every bad password attempt, it affects the number of attempts users are allowed in some cases. The badPwdCount is more likely to reset when a user attempts with an old password.

This new feature is sometimes called password history n-2. The most recent previous password is referred as n-1. The next most recent are n-2. Not all authentication types will take advantage of this new feature. Kerberos and NTLM authentication protocols support password history n-2. These protocols are used when either a password or smart card is used for interactive login. Other protocols, such as RADIUS and PEAP, may or may not increment badPwdCount when a bad password is attempted. Some protocols do not forward bad password attempts to the PDC Emulator. That might explain why phone users can get locked out if the phone attempts repeatedly to authenticate with a bad password.

In addition, the system can only know about the previous 2 passwords in history if pwdHistoryLength is at least 3. With that setting, the user can rotate through 3 passwords, so the previous 2 are retained in password history. If pwdHistoryLength is 2, the user can alternate between two passwords. The only password attempt that will not increment badPwdCount, in that case, is the previous one. The system does not retain the second most recent password.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .