9

I'm having an issue on CentOS 6.7 where SSH logins are 1s more than any non-6.7 machines on this network (e.g. 7.2, 5.11). Running debugging on the client side showed the hang at "Entering interactive session".

The command I'm using to base this test on is time ssh <host> true from my laptop, using SSH Keys.

The two things I've already checked/modified are UseDNS and GSSAPIAuthentication, and both are disabled.

I started a separate daemon on a different port with debugging on, and have found where the short hang occurs:

debug1: SELinux support enabled
debug3: ssh_selinux_setup_exec_context: setting execution context

{1s hang}

debug3: ssh_selinux_setup_exec_context: done

SELinux is set to 'permissive'. I'm not sure why it would even bother with "setting contexts". Is there any way to have these two get along better without disabling SELinux altogether? I realize 1s isn't much, but I use this particular machine as an SSH gateway to hosts with IP whitelists (this is a static IP machine), and it adds up throughout the day.

After running an strace, the hang is a little more granular:

22:16:05.445032 open("/selinux/user", O_RDWR|O_LARGEFILE) = 4 <0.000090>
22:16:05.445235 write(4, "unconfined_u:system_r:sshd_t:s0-s0:c0.c1023 unconfined_u", 56) = 56 <0.334742>
22:16:05.780128 read(4, "18\0unconfined_u:system_r:prelink_mask_t:s0-s0:c0.c1023\0unconfined_u:system_r:unconfined_mount_t:s0-s0:c0.c1023\0unconfined_u:system_r:abrt_helper_t:s0-s0:c0.c1023\0unconfined_u:system_r:oddjob_mkhomedir_t:s0-s0:c0.c1023\0unconfined_u:system_r:unconfined_notrans_t:s0-s0:c0.c1023\0unconfined_u:system_r:unconfined_execmem_t:s0-s0:c0.c1023\0unconfined_u:system_r:unconfined_java_t:s0-s0:c0.c1023\0unconfined_u:system_r:unconfined_mono_t:s0-s0:c0.c1023\0unconfined_u:system_r:chkpwd_t:s0-s0:c0.c1023\0unconfined_u:system_r:passwd_t:s0-s0:c0.c1023\0unconfined_u:system_r:updpwd_t:s0-s0:c0.c1023\0unconfined_u:system_r:mount_t:s0-s0:c0.c1023\0unconfined_u:system_r:rssh_t:s0-s0:c0.c1023\0unconfined_u:system_r:xauth_t:s0-s0:c0.c1023\0unconfined_u:system_r:unconfined_t:s0-s0:c0.c1023\0unconfined_u:system_r:openshift_t:s0-s0:c0.c1023\0unconfined_u:unconfined_r:oddjob_mkhomedir_t:s0-s0:c0.c1023\0unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023\0", 4095) = 929 <0.000079>

Writing to /selinux/user takes 350ms alone.

Update 1 - Things I've since tried:

  • Updating. Many of the boxes required updates. This hasn't had any material effect on login times.
  • semodule -d unconfined - This has had a positive effect, reducing the login times by roughly 500ms. However, my C7 and C5.11 boxes (which I've since (re)learned have SELinux disabled) average ~525ms
  • Comparing my C6.7 machines - Turns out the 64bit boxes are faster than my 32-bit boxes. Maybe this is a 'duh' moment for some, but since none of these boxes are particularly taxed, I wasn't expecting a 100-300ms variance. I was able to break the 1s mark (850ms) on one of the 64bit hosts. The lowest on 32bit has been 1.085s

Am I asking too much? Is 10-1200ms an acceptable login time for selinux-permissive machines? Curious what other people's benchmarks are.

  • Check your system logs. – Michael Hampton Jan 17 '16 at 15:20
  • I have... Unfortunately, nothing noteworthy in messages or secure – Morgon Jan 17 '16 at 15:22
  • @Morgon: Have you ckecked if you're running something like "fail2ban" - fail2ban.org - in that machine and/or if you have set up "iptables" in some way that may be doing some kind of "rate limiting" to incoming SSH connections depending on source IP address(es) ? – ricmarques Jan 17 '16 at 16:25
  • 4
    If logs aren't helping, you should probably use strace and see what call it is hanging on. – Andrew B Jan 17 '16 at 18:27
  • 1
    Thanks, @AndrewB. I've posted it here: pastebin.com/raw/3c08tcMd. if you search for "/selinux/user", you'll see the 1s hangtime while trying to write to that file. -- Going down that path, I found a marginally-related post about disabling 'unconfined' domains (bugzilla.redhat.com/show_bug.cgi?id=811656#c9). This does shave ~.5 seconds from my logins, but still .5 seconds slower than Cent7/FC4. Still looking to eke out that last bit of performance, if possible. – Morgon Jan 17 '16 at 20:30
1

This may not be related, but selinux related slowness has been report for IPA-based accounts here: https://access.redhat.com/discussions/3499951

The solution for that case is:

I looked into SSSD and IPA user forums and since my systems do not use SELINUX, I went ahead and added this to the domain section in the sssd configuration:

selinux_provider=none

After this the login process is immediate, as expected.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.