While migrating my Kerberos servers from CentOS 6 to CentOS 7 I seem to have broken something on the current KDC and, despite several hours of trial-and-error and searching documentation, I cannot figure out what happened.
When a user logs in using his Kerberos password the profile runs a short program to print the password expiration date. This process uses remctl and, until a day ago, worked well. Now it reports an error that makes no sense to me:
remctl: GSS-API error initializing context: Unspecified GSS failure. Minor code may provide more information, Generic error (see e-text)
If I run the remctl command with env KRB5_TRACE=/dev/stdout I get the following:
 1568213225.255331: ccselect can't find appropriate cache for server principal host/scakerb01.lereta.net@  1568213225.255416: Retrieving stephen@TOTALFLOOD.COM -> krb5_ccache_conf_data/proxy_impersonator@X-CACHECONF: from FILE:/tmp/krb5cc_501 with result: -1765328243/Matching credential not found remctl: GSS-API error initializing context: Unspecified GSS failure. Minor code may provide more information, Generic error (see e-text)
klist reports the TGT is there and logins work as expected.
Any idea what I broke? I checked the clock skew and it is less than one second.
The Kerberos version on CentOS 6 is 1.10.3 which I know is old.