2

I am trying to bootstrap a Centos7 EC2 instance (ami-02eac2c0129f6376b) with bash user-data. Because it runs as root and I need to create a lot of things as the centos user, I use sudo -Hu centos <cmd> many times. Each call introduces a 25 second delay. I have hundreds. What can I do to speed this up?

This is a known issue, but I cannot find any solutions.

I have tried:

  • Add and /etc/hosts entry for my hostname
printf "\n127.0.0.1   %s %s\n" "$(hostname -f)" "$(hostname -s)" | tee -a /etc/hosts
  • Remove myhostname from /etc/nsswitch.conf
sed -Ei 's/\s*myhostname//' /etc/nsswitch.conf

These delays are EXTREMELY painful because I am currently in the Trial and Error phase of building out knew user-data scripts.

What can I do?

  • Do you not have DNS lookups and hostnames enabled for your VPC? – Michael - sqlbot Aug 2 at 15:22
  • @Michael-sqlbot I curl several resources in my user-data. They have no problem doing lookups. But for some reason, every sudo takes 25 seconds to make the call to the command. – Bruno Bronosky Aug 2 at 20:39
  • Hmmm... does dig $(hostname) work (return the instance's internal IP) and dig -x z.z.z.z (z.z.z.z = instance's private IP) work (return the hostname) or do they time out or return an error? IIRC sudo tries to look up the machine's IP... 25 sec seems like a very long time if this is the problem, but this is the only idea I have. – Michael - sqlbot Aug 2 at 23:01
2

While it's better to figure out why sudo is slow and resolve that, you can group your commands together to only call sudo once. Inside your script, you could do something like this:

sudo -Hu centos bash <<EOF
somecommand
somecommand2
morecommands
EOF
2

A bug in the SELinux policy included with RHEL and CentOS 7 caused sudo to wait for exactly 25 seconds on each command. This bug was fixed with the selinux-policy-3.13.1-229.el7_6.10 package. You should create a new AMI with this package (and preferably the whole system) updated.

  • I am using - Name: selinux-policy Arch: noarch Version: 3.13.1 Release: 229.el7_6.15 – Bruno Bronosky Aug 4 at 1:53
  • @BrunoBronosky Is that already in your AMI or does your cloud-init update to it on first boot? If it's already in your AMI then the problem may be elsewhere, and it's time to start looking at how your AMI was put together. But that doesn't appear to be the latest CentOS 7 AMI, so you probably should just go ahead and get the latest AMI. – Michael Hampton Aug 4 at 1:57
  • My yum update -y updates selinux-policy.noarch 0:3.13.1-229.el7_6.6 to 0:3.13.1-229.el7_6.15 and selinux-policy-targeted.noarch 0:3.13.1-229.el7_6.6 to 0:3.13.1-229.el7_6.15 Do I need to do something after the yum update to get it to take effect? – Bruno Bronosky Aug 4 at 2:01
  • @BrunoBronosky The update needs to be installed before the first run of cloud-init, so you need a new AMI. You should be on ami-089ccd342f0be98ab, or something later, not ami-02eac2c0129f6376b. If you aren't getting that AMI, see this reddit thread. – Michael Hampton Aug 4 at 2:02
  • I cannot find that image. I have tried aws ec2 describe-images --image-ids ami-089ccd342f0be98ab and even aws ec2 describe-images --owners aws-marketplace --filters '[ {"Name": "product-code", "Values": ["aw0evgkw8e5c1q413zgy5pjce"]} ]' --query 'sort_by(Images, &CreationDate)' the image I am using is the latest I can find. Here is how I am actually selecting my image. stackoverflow.com/a/56401124/117471 – Bruno Bronosky Aug 4 at 2:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.