0

I'm trying to run a container on a RHEL 6.5 but I keep hitting this problem:

sudo docker run -u postgres -it registry/postgres /bin/bash
/bin/bash: error while loading shared libraries: libtinfo.so.5: cannot open shared object file: Permission denied

When run as user 'root', the container starts fine but the problem appears again when trying to switch to another user:

$ sudo docker run -u root -it registry/database /bin/bash
[root@8a20410eaa5e /]# su postgres
su: /bin/bash: Permission denied

This is a specific container built by us, based on CentOS 6.5 an that runs Postgres. The Dockerfile to build it has "USER postgres" in it, and it works fine elsewhere except these servers. I can reproduce the same behaviour with a busybox container:

$ sudo docker run -u nobody -it 10.188.13.136:8080/busybox
/ $ ls
/bin/sh: ls: Permission denied

The RHEL 6.5 host has SELinux enabled. We have other other hosts where SELinux and this container works fine there. The audit log for this host looks clean, no error messages that I can see when trying to run the container.

This is what we've tried so far:

  • update the SELinux policies in RHEL ("sudo yum upgrade selinux-policy"), as they were not the latest versions
  • get SELinux into permissive mode (setenforce 0); not tried to switch it off completely and reboot
  • start the Docker daemon with "--selinux-enabled=true"
  • start the container with --privileged
  • start the container with --security-opt=:label:disable
  • we're running the latest RHEL 6.5 kernel: 2.6.32-504.16.2.el6.x86_64

Also run a strace session for the 'su ' command within the container but could not see much beyond these:

 17    setgid(10000)                     = 0
 17    setuid(10000)                     = 0
 17    munmap(0x7f07a3540000, 2101304)   = 0
 17    munmap(0x7f07a311c000, 2113776)   = 0
 17    munmap(0x7f07a2f03000, 2196352)   = 0
 17    munmap(0x7f07a2cea000, 2198192)   = 0
 17    munmap(0x7f07a2ae8000, 2101272)   = 0
 17    munmap(0x7f07a28e4000, 2109624)   = 0
 17    munmap(0x7f07a26e0000, 2109672)   = 0
 17    munmap(0x7f07a24d3000, 2148896)   = 0
 17    munmap(0x7f07a22d0000, 2105488)   = 0
 17    munmap(0x7f07a20cb000, 2113848)   = 0
 17    munmap(0x7f07a1ec5000, 2118168)   = 0
 17    munmap(0x7f07a3321000, 2221912)   = 0
 17    execve("/bin/bash", ["bash"], [/* 15 vars */]) = -1 EACCES (Permission denied)
 17    write(2, "su: ", 4)               = 4
 17    write(2, "/bin/bash", 9)          = 9

The full strace dump is here in case it's needed: http://pastebin.com/42C2B8LP.

We're not sure what to look for next, any ideas?

  • 1
    have you tried to check you auditd logs? /var/log/audit/audit.log, put selinux in permissive mode, in this way you can see if the problem is selinux. – c4f4t0r May 15 '15 at 9:56
  • We did, both. Setting SElinux to permissive (setenforce 0) and then running the container did not seem to help. I cannot see anything unusual in the audit logs either when running the container: pastebin.com/F6LSUn2J (this is from an unsuccessful run with "sudo docker run -u postgres -it registry/postgres /bin/bash", which returns "/bin/bash: error while loading shared libraries: libtinfo.so.5: cannot open shared object file: Permission denied") – Oscar Renalias May 15 '15 at 10:38
  • I also forgot to state that this is happening on Docker 1.6 from EPEL-testing. We did downgrade to 1.3.2 to verify if it was an issue with Docker but the behaviour was still reproducible. – Oscar Renalias May 15 '15 at 11:55
  • How did you set up your storage? – Michael Hampton May 15 '15 at 15:09
  • @MichaelHampton it's under /var/lib/docker, default location. /var is an LVM volume group: /dev/mapper/vg_main-lv_var on /var type ext4 (rw). It's the same in every host. Or did you mean something else? – Oscar Renalias May 15 '15 at 21:36
0

I was finally able to solve this problem. Which means that it seems that I found a solution, but I'm not still sure what the problem is:

1) pull container from registry 2.0 + run with docker 1.6 -> fail

2) pull container from registry 0.9.x (either Docker's own or the Artifactory server that we run internally) + run with docker 1.6 -> works

3) pull container from registry 2.0 + run with docker 1.5 or older -> fail

4) pull container form registry 0.9.x + run with docker 1.5 or newer -> works

I don't really think that Registry 2.0 is at fault but I have no better answer. The new registry is much faster than the old one but guess we'll stick with the older registry for the time being.

  • Did you find out the root cause? The same thing happened to me RHEL 6.8, docker 1.7.1. How did you force docker to pull from v1? – Nam Ngo Apr 7 '16 at 1:04
  • No, we did not find the root cause. The way we "forced" a v1 registry was to run the old registry legacy image and not the newer v2 one (use hub.docker.com/_/registry and use a label lower than "2"), and it's been fine since then. – Oscar Renalias Apr 10 '16 at 18:56

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.