I'm having an issue on CentOS 6.7 where SSH logins are 1s more than any non-6.7 machines on this network (e.g. 7.2, 5.11). Running debugging on the client side showed the hang at "Entering interactive session".

The command I'm using to base this test on is time ssh <host> true from my laptop, using SSH Keys.

The two things I've already checked/modified are UseDNS and GSSAPIAuthentication, and both are disabled.

I started a separate daemon on a different port with debugging on, and have found where the short hang occurs:

debug1: SELinux support enabled
debug3: ssh_selinux_setup_exec_context: setting execution context

{1s hang}

debug3: ssh_selinux_setup_exec_context: done

SELinux is set to 'permissive'. I'm not sure why it would even bother with "setting contexts". Is there any way to have these two get along better without disabling SELinux altogether? I realize 1s isn't much, but I use this particular machine as an SSH gateway to hosts with IP whitelists (this is a static IP machine), and it adds up throughout the day.

After running an strace, the hang is a little more granular:

22:16:05.445032 open("/selinux/user", O_RDWR|O_LARGEFILE) = 4 <0.000090>
22:16:05.445235 write(4, "unconfined_u:system_r:sshd_t:s0-s0:c0.c1023 unconfined_u", 56) = 56 <0.334742>
22:16:05.780128 read(4, "18\0unconfined_u:system_r:prelink_mask_t:s0-s0:c0.c1023\0unconfined_u:system_r:unconfined_mount_t:s0-s0:c0.c1023\0unconfined_u:system_r:abrt_helper_t:s0-s0:c0.c1023\0unconfined_u:system_r:oddjob_mkhomedir_t:s0-s0:c0.c1023\0unconfined_u:system_r:unconfined_notrans_t:s0-s0:c0.c1023\0unconfined_u:system_r:unconfined_execmem_t:s0-s0:c0.c1023\0unconfined_u:system_r:unconfined_java_t:s0-s0:c0.c1023\0unconfined_u:system_r:unconfined_mono_t:s0-s0:c0.c1023\0unconfined_u:system_r:chkpwd_t:s0-s0:c0.c1023\0unconfined_u:system_r:passwd_t:s0-s0:c0.c1023\0unconfined_u:system_r:updpwd_t:s0-s0:c0.c1023\0unconfined_u:system_r:mount_t:s0-s0:c0.c1023\0unconfined_u:system_r:rssh_t:s0-s0:c0.c1023\0unconfined_u:system_r:xauth_t:s0-s0:c0.c1023\0unconfined_u:system_r:unconfined_t:s0-s0:c0.c1023\0unconfined_u:system_r:openshift_t:s0-s0:c0.c1023\0unconfined_u:unconfined_r:oddjob_mkhomedir_t:s0-s0:c0.c1023\0unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023\0", 4095) = 929 <0.000079>

Writing to /selinux/user takes 350ms alone.

Update 1 - Things I've since tried:

  • Updating. Many of the boxes required updates. This hasn't had any material effect on login times.
  • semodule -d unconfined - This has had a positive effect, reducing the login times by roughly 500ms. However, my C7 and C5.11 boxes (which I've since (re)learned have SELinux disabled) average ~525ms
  • Comparing my C6.7 machines - Turns out the 64bit boxes are faster than my 32-bit boxes. Maybe this is a 'duh' moment for some, but since none of these boxes are particularly taxed, I wasn't expecting a 100-300ms variance. I was able to break the 1s mark (850ms) on one of the 64bit hosts. The lowest on 32bit has been 1.085s

Am I asking too much? Is 10-1200ms an acceptable login time for selinux-permissive machines? Curious what other people's benchmarks are.

  • Check your system logs. – Michael Hampton Jan 17 '16 at 15:20
  • I have... Unfortunately, nothing noteworthy in messages or secure – Morgon Jan 17 '16 at 15:22
  • @Morgon: Have you ckecked if you're running something like "fail2ban" - fail2ban.org - in that machine and/or if you have set up "iptables" in some way that may be doing some kind of "rate limiting" to incoming SSH connections depending on source IP address(es) ? – ricmarques Jan 17 '16 at 16:25
  • 2
    If logs aren't helping, you should probably use strace and see what call it is hanging on. – Andrew B Jan 17 '16 at 18:27
  • 1
    Thanks, @AndrewB. I've posted it here: pastebin.com/raw/3c08tcMd. if you search for "/selinux/user", you'll see the 1s hangtime while trying to write to that file. -- Going down that path, I found a marginally-related post about disabling 'unconfined' domains (bugzilla.redhat.com/show_bug.cgi?id=811656#c9). This does shave ~.5 seconds from my logins, but still .5 seconds slower than Cent7/FC4. Still looking to eke out that last bit of performance, if possible. – Morgon Jan 17 '16 at 20:30

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.