1

I'm trying to run Docker on Elastic MapReduce streaming but am having trouble with a permissions issue. In my bootstrap script, I need the "hadoop" user to be part of the "docker" group (as described on the AWS Docker Basics page):

sudo usermod -a -G docker hadoop

but I cannot log out or spawn a subshell (with newgrp). Is there a way to force the group setting to take effect in my current shell session?

2 Answers 2

0

No - unless the hadoop process is running as root, then you could do some gdb trickery to change the EUID, but not the group. But I guess that won't help in your case.

What could help is to temporarily allow the hadoop user access to the files that are meant for the docker group to be accessed.

Example: the hadoop user needs write-access to the /foo directory, but only the docker group can currently write it:

$ ls -ld /foo/
drwxrwx--- 2 root docker 4096 Mar 10 00:42 /foo/

Most file systems support ACLs nowadays:

$ sudo setfacl -m g:docker:rwx /foo

Now the hadoop user will be able to write to /foo.

I know, that's not what you asked, but since there's no possibility to change the EGID of a running process, maybe it works the other way around with the help of ACLs.

1
  • Thanks @ckujau - but yeah, it's not just a file permissions thing. For some reason (that I don't really understand), on Amazon Linux AMI the "docker" command can only be run by members of the "docker" group, otherwise there's a TCP issue or something as the docker client attempts to communicate with the docker engine.
    – Max
    Mar 10, 2016 at 13:59
-1

It took me a while to figure out how to run Docker on Hadoop Streaming on EMR. It turns out that the user that runs the job is yarn and not hadoop. So the bootstrap script needs to have the following lines:

sudo useradd -r yarn # create 'yarn' since it does not exist yet
sudo usermod -a -G docker yarn

Note, that I manually create yarn user because when the bootstrap script runs Hadoop is not installed yet

2
  • 2
    This doesn't even address the question Aug 8, 2017 at 10:28
  • 1
    It addresses the original problem @Max had. The OP was asking a wrong question...
    – regata
    Aug 8, 2017 at 16:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .