1

I am trying to configure CertBot and it only works when I serve my site over http. Usually I have an https redirect and I don't want to have to change the site config each time I need to use certbot. I tried to serve only /.well-known/ over http but it is still failing any ideas how to resolve this?

I am trying to copy this idea but not working --> NGINX redirect everything except letsencrypt to https

Eg: This Works:

server {
    listen 80;
    listen [::]:80;
    server_name example.com www.example.com;


location / {

        proxy_pass              http://localhost:8575/;
        include                 /etc/nginx/conf.d/proxy.conf;
    }
}

This does not: (Note that the current configured SSL Certs are not correct, but needed for NGinX to start)

server {

   listen 80;
   listen [::]:80;
   server_name www.example.com example.com;

    location /.well-known/acme-challenge/ {
        proxy_pass              http://localhost:8575/;
        include                 /etc/nginx/conf.d/proxy.conf;
    }

location / {
       return 301 https://$server_name$request_uri;
    }

}

server {
        listen 443 ssl;
        listen        [::]:443;
        server_name www.example.com example.com;

#        ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
#        ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
        ssl_certificate /etc/ssl/crt/crt.crt;
        ssl_certificate_key /etc/ssl/crt/key.key;

location / {

        proxy_pass              http://localhost:8575/;
        include                 /etc/nginx/conf.d/proxy.conf;
    }
}

Error Log:

certbot    | Saving debug log to /var/log/letsencrypt/letsencrypt.log
certbot    | Plugins selected: Authenticator webroot, Installer None
certbot    | Registering without email!
certbot    | Obtaining a new certificate
certbot    | Performing the following challenges:
certbot    | http-01 challenge for www.example.com
certbot    | http-01 challenge for example.com
certbot    | Using the webroot path /var/www/html for all unmatched domains.
certbot    | Waiting for verification...
certbot    | Challenge failed for domain www.example.com
certbot    | Challenge failed for domain example.com
certbot    | http-01 challenge for www.example.com
certbot    | http-01 challenge for example.com
certbot    | Cleaning up challenges
certbot    | IMPORTANT NOTES:
certbot    |  - The following errors were reported by the server:
certbot    |
certbot    |    Domain: www.example.com
certbot    |    Type:   unauthorized
certbot    |    Detail: Invalid response from
certbot    |    http://www.example.com/.well-known/acme-challenge/WyVEA5g6BWVDPpYUhEJ0bG5iH6daF1rZpFd0vuTXOa0
certbot    |    [50.117.156.123]: "        <!DOCTYPE html><html lang=\"en-US\">\r\n
certbot    |    \t<head>\n\n\t\t        <meta charset=\"UTF-8\">\r\n        <meta
certbot    |    name=\"viewport\" con"
certbot    |
certbot    |    Domain: example.com
certbot    |    Type:   unauthorized
certbot    |    Detail: Invalid response from
certbot    |    https://www.example.com/x61_h9wxFY2Ye8-16GllyMq_dfsXbsEB1lYOjeq4LjU
certbot    |    [50.117.156.123]: "        <!DOCTYPE html><html lang=\"en-US\">\r\n
certbot    |    \t<head>\n\n\t\t        <meta charset=\"UTF-8\">\r\n        <meta
certbot    |    name=\"viewport\" con"
certbot    |
certbot    |    To fix these errors, please make sure that your domain name was
certbot    |    entered correctly and the DNS A/AAAA record(s) for that domain
certbot    |    contain(s) the right IP address.
certbot    |  - Your account credentials have been saved in your Certbot
certbot    |    configuration directory at /etc/letsencrypt. You should make a
certbot    |    secure backup of this folder now. This configuration directory will
certbot    |    also contain certificates and private keys obtained by Certbot so
certbot    |    making regular backups of this folder is ideal.
certbot    | Some challenges have failed.
certbot exited with code 1
1

In your config, proxy_pass is not necessary for certbot if nginx has access to the root directory. You can just allow access for .well-known directory and declare server_name and root for the site.

I had written on this for a blog post (https://dev.slickalpha.blog/2019/11/installing-lemp-stack-on-debian-buster.html).

server {
    listen 80;
    listen [::]:80;
    server_name example.com www.example.com;
    root /var/www/example.com/site;
    location ~ /.well-known/acme-challenge/ {
       allow all;
    }
    location / { 
       return 301 https://$server_name$request_uri;
    }
}
2
  • @FreeSoftwareServers Okay, I didn't know that. I've applied your edit to the answer. Thanks. – Ajay Singh May 22 '20 at 6:37
  • I also wouldn't recommend that ~ as it gave errors during my launch due to proxy_pass and I don't think it's needed. But I have reconsidered that nginx doesn't need write access to the real webroot of the webapp, just any old folder/location should do, perhaps even tmp? I'm not sure if this acme-challenge contains sensitive information or anything I need to have be persistent TBH. – FreeSoftwareServers May 22 '20 at 8:44
1

With HTTP-01 challenge it's OK to redirect to HTTPS first and serve the challenge over TLS. However, the challenge always starts with a plain HTTP connection using port 80, and you can only redirect to HTTPS on port 443.

Our implementation of the HTTP-01 challenge follows redirects, up to 10 redirects deep. It only accepts redirects to “http:” or “https:”, and only to ports 80 or 443. It does not accept redirects to IP addresses. When redirected to an HTTPS URL, it does not validate certificates (since this challenge is intended to bootstrap valid certificates, it may encounter self-signed or expired certificates along the way).

The HTTP-01 challenge can only be done on port 80. Allowing clients to specify arbitrary ports would make the challenge less secure, and so it is not allowed by the ACME standard.

Therefore, this kind of Nginx configuration should work, as well:

server {
    listen 80;
    server_name www.example.com example.com;

    location / {
        return 301 https://$server_name$request_uri;
    }
}

server {
    listen 443 ssl;
    server_name www.example.com example.com;

    location /.well-known/acme-challenge/ {
        # HTTP-01 challenge
    }

    location / {
        # Your web application
    }
}

In your case this means the following could be either in the HTTP or the HTTPS server block.

location /.well-known/acme-challenge/ {
    proxy_pass   http://localhost:8575/.well-known/acme-challenge/;
    include      /etc/nginx/conf.d/proxy.conf;
}

You were able to replace the /.well-known/acme-challenge/ with $request_uri because:

When variables are used in proxy_pass:

location /name/ {
    proxy_pass http://127.0.0.1$request_uri;
}

In this case, if URI is specified in the directive, it is passed to the server as is, replacing the original request URI.

Also, if the / has the same root, you don't need a separate location at all.

11
  • Just don't understand why it needs its own server location when root is forwarded normally perhaps if I tried adding requesturi with just my normal https root redirect it wouldn't affect anything and allow this to work with my configurations as they are. Perhaps it should be added to all my configurations – FreeSoftwareServers May 22 '20 at 8:14
  • If it's served from the same root or using the same proxy, it should work without the separate location block. This is a more generalized answer for serving it differently, if e.g. the web application was reverse proxied and /acme-challenge/ served as static files. – Esa Jokinen May 22 '20 at 8:17
  • OK, but this should simplify my setup too the real trick for me was requestURI inside the proxy_pass directive, I'll go remove the location block tomorrow and retest. – FreeSoftwareServers May 22 '20 at 8:19
  • 1
    Think that you had a web application on http://localhost:8080/ (not in http://localhost:8080/app/) and you wanted to serve it on https://example.com/app/, because the root was used for another website. That wouldn't be possible, if the proxy automatically added the location path to the request. – Esa Jokinen May 22 '20 at 8:35
  • 1
    The webroot for ACME challenge can be same for every domain. With Apache it's even possible to add a global alias, which makes it a lot easier to add new domains even before they are configured. In Nginx you could have this served from the default server. – Esa Jokinen May 22 '20 at 9:13
1

Update: My main issue was not using $request_uri w/ proxy_pass directive (which also doesn't allow ~ BTW). But, there was nothing wrong with using this in the HTTPS block. Further more after looking at both https://serverfault.com/a/1018199/312793 and

https://serverfault.com/a/1017720/312793 I realized that I don't need to actually pass my "real" root directory of my webapp, just somewhere that nginx can serve to certbot to read/write files. As well, you can have one directory serve multiple sites so I decided it would be most proficient to add the location inside the default nginx server block I have setup to re-route incorrectly formatted requests to include certbot so I can now add domains without adjust any configs 100%. In fact, the web app doesn't even need to be running, just nginx.

Here is my new default server block. Note: I created a folder acme inside my nginx "real" webroot and serve that directory for the location /.well-known/acme-challenge/

server {
        listen 80 default_server;
        listen [::]:80 default_server;
        root /var/www/html/;
        index index.html;
}
server {
        listen 443 default_server;
        listen [::]:443 default_server;
        ssl_certificate /etc/ssl/fake/fake.crt;
        ssl_certificate_key /etc/ssl/fake/fake.key;

        location /.well-known/acme-challenge/ {
        root /var/www/html/acme;
         allow all;
        }

        root /var/www/html/;
        index index.html;
}

Just like when doing setup, you need to have something for SSL certs or nginx won't start correctly. Very happy with this setup/resolution!


Needed to add the following: $request_uri

location /.well-known/acme-challenge/ {
    proxy_pass              http://localhost:8575/$request_uri;
    include                 /etc/nginx/conf.d/proxy.conf;
}
1
  • Great that you were able to solve it. Please accept your answer, otherwise the question will not be marked as "solved" in the system and will pop up again and again in the future. It is perfectly fine to accept your own answers. – Gerald Schneider May 19 '20 at 6:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.