1

I am diagnosing a weird problem with some of my Ubuntu 14 LTS / Apache 2.4 machines with Certbot renewals. Yes, I know Ubuntu 14 LTS is EOL soon; these are some of the straggler machines on the rebuild list.

Essentially, Certbot has been working well for a while on these machines, with renewals handled via a cron job. Sometime between October and now, renewals have been failing with a 403 during the HTTP-01 challenge:

Detail: Invalid response from
http://example.com/.well-known/acme-challenge/asdkjhfaklsjdfhalksdjfhkljasdfh:
"<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML
2.0//EN\">\n<html><head>\n<title>403
Forbidden</title>\n</head><body>\n<h1>Forbidden</h1>\n<p"

To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address.

I have diagnosed the problem down to the .well-known directory being denied during the challenges because of Apache config (a Require all denied on /).

As a stop-gap, I am renewing the certs by hand by a temporary Require all allowed on /, but want to fix this properly.

So, where does Certbot actually serve up challenges from? I tried a

<Directory /var/lib/letsencrypt/http_challenges>
    Options Indexes FollowSymLinks
    AllowOverride None
    Require all granted
</Directory>

but that didn't seem to work. Because of the ratelimiting, guess-and-checks are slow.

0

Requests are served up from /var/lib/letsencrypt/http_challenges. I think this is related to certbot 0.28.0 changes and EOL for TLS-SNI-01

On my Ubuntu 14.04 / Apache 2.4.7 machines two things were needed.

I had to remove the root lockdown. I edited /etc/apache2/conf-enabled/security.conf and also added in where the challenges get served from at the same time.

<Directory />
   AllowOverride None
   Require all granted
</Directory>

<Directory /var/lib/letsencrypt/http_challenges>
   AllowOverride All
   Require all granted
</Directory>

Restart Apache and do a certbot dry run

$ sudo service apache2 restart
$ sudo certbot renew --dry-run

Then, put back the lockdown:

<Directory />
   AllowOverride None
   Require all denied
</Directory>

<Directory /var/lib/letsencrypt/http_challenges>
   AllowOverride All
   Require all granted
</Directory>

Repeat the Apache restart certbot dry run

$ sudo service apache2 restart
$ sudo certbot renew --dry-run

At this point HTTP-01 challenges showed success. In my case, I forced the issue of the TLS-SNI-01 shutdown, and force renewed my certs and made sure they used HTTP-01 challenges.

I really wish I knew why this sequence was needed, and not just a happy accident. I don't think it was a fluke, though, since I needed to do the exact same steps on ~10 machines. None of my newer Ubuntu servers needed this.

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.