0

I have an nginx server that is the reverse proxy for numerous internal services. While I don’t know when the issue started; currently, the rewrite statement is no longer working to redirect all http traffic to https.

The end goal for the rewrite statements are for all web traffic (including the sub folders) to example.com to be rewritten to www.example.com. Additionally, all web traffic for http:// www.example.com to be redirected to https:// www.example.com.

Currently, if you navigate to http:// example.com/service1, you’re presented a 404 not found by nginx. /var/log/nginx/error.log log file data below:

2017/08/24 18:36:55 [error] 31476#31476: *919529 open() "/etc/nginx/html/service1" failed (2: No such file or directory), client: 1.2.3.5, server: , request: "GET /chub HTTP/1.1", host: "example.com"

One item that I noticed was that the log file that shows the attempt to access the web page isn’t the one defined in the config file. This leads me to believe that there is an issue with the config file and that nginx is not using the directives within it to route the traffic.

If you navigate to https:// example.com/service1, you are redirected to the correct URL. Also, it does not matter whether you use www. or not, the output is the same. The only thing that proves successful, is https:// www.example.com. I’ve tried numerous different listen/server_name/return/rewrite directives and none of them have been successful.

Below is the nginx site config. Note that some of the texts wraps around due to window size. Also note, i had to add spaces in some of the URLs since i don't have the 10 rep points needed to add more than 2 URLs.

log_format awstats '$remote_addr - $remote_user [$time_local] $status 
'
                   '"$request" $body_bytes_sent "$http_referer" '
                   '"$http_user_agent" "http_x_forwarded_for"';

server {
  server_name example.com www.example.com;
  access_log /var/log/nginx/www.access.log timed_combined;
  error_log  /var/log/nginx/www.error.log;
  return 301 https://www.example.com$request_uri;
}

server {
  listen 1.2.3.4:443 ssl http2;
  server_name example.com www.example.com;

  ssl on;
  ssl_certificate /etc/pki/tls/certs/example.com.chained.crt;
  ssl_certificate_key /etc/pki/tls/private/example.com.key;

  add_header Alternate-Protocol 443:npn-spdy/3,443:npn-spdy/2;

  include /etc/nginx/proxy.conf;

  access_log /var/log/nginx/www.access.log timed_combined;
  error_log  /var/log/nginx/www.error.log;
  error_log syslog:server=127.0.0.1:514;

  client_max_body_size 512M;

# This section uses a non-standard nginx http status code to silently 
# drop requests for invalid objects.
# All of the objects below are malicious attempts to probe the 
# server.

  # Generic location no-no's
  location ^~ /cgi-bin/ { return 444; }
  location ^~ /scripts/ { return 444; }
  location ^~ /admin/ { return 444; }
  location ^~ /administrator/ { return 444; }
  location ^~ /sdk/ { return 444; }
  location ^~ /file/ { return 444; }
  location ^~ /level/ { return 444; }
  location ^~ /w00tw00t.at.blackhats.romanian.anti-sec:) { return 444; }

  # Third-party software no-no's
  location ^~ /manager/ { return 444; }
  location ^~ /ispmgr/ { return 444; }
  location ^~ /MyAdmin/ { return 444; }
  location ^~ /myadmin/ { return 444; }
  location ^~ /phpMyAdmin/ { return 444; }
  location ^~ /phpmyadmin/ { return 444; }
  location ^~ /pma/ { return 444; }

  # Microsoft specific no-no's
  location ^~ /IISADMPWD/ { return 444; }
  location ^~ /msadci/ { return 444; }
  location ^~ /_vti_bin/ { return 444; }
  location ^~ /_vti_pvt/ { return 444; }
  location ~* ./(\.php)$ { return 444; }

  # These items aren't invalid perse, however there are "issues" 
  # downstream that makes it easier to cause them to just die
  # peacefully rather than tax resources.
  location ^~ /service1/++resource++quickupload_static/ { return 444; 
}
  location ^~ /dav/ { return 444; }
  location ^~ /mnt/ { return 444; }

  # Rather than service this one, we'll just drop the request 
  # altogether.
  location ~* ./(Thumbs\.db)$ { return 444; } # Originates from 
Microsoft Windows
  location ~* ./(\.DS_Store)$ { return 444; } # Originates from MacOS 
X
  location ~* ./(browserconfig\.xml)$ { return 444; } # Originates 
specifically from Internet Explorer 11 (used for browser pinning)

  # ===============================================
  # This section governs access to the Service1.
  # ===============================================

  # This section is the main url rewriting for the Plone Virtual 
  # Hosting Monster.
  # It also returns the custom error pages for this application.

  location ^p_$ { return 301 https://www.example.com/^p_/; }
  location ^misc_$ { return 301 https://www.example.com/^misc_/; }
  location /manage_page_style.css { return 301 
https://www.example.com/service1/manage_page_style.css; }
  location /++resource++plone.dexterity.item.gif { return 301 
   https://www.example.com/service1/++resource++plone.dexterity.item.gif; 
}

location /service1/ {
proxy_intercept_errors on;
proxy_pass http://localhost:6081/VirtualHostBase/https/www.example.com:443/service1/service1/VirtualHostRoot/_vh_service1/;
proxy_redirect default;
error_page 400 /400.html;
error_page 401 /401.html;
error_page 403 /403.html;
error_page 404 /404.html;
error_page 405 /405.html;
error_page 408 /408.html;
error_page 411 /411.html;
error_page 500 /500.html;
error_page 502 /502.html;
error_page 503 /503.html;
error_page 504 /504.html;
  }

  # This manages the ability for search engines to crawl 
  # www.example.com.
  location /robots.txt {
    proxy_pass http://1.2.3.6/robots.txt;
  }

  # The following three sections perform special url rewriting for 
  # plone resources that are normally expected to exist at the site 
  # root level, which is otherwise inaccessible using the proxy.
  location /misc_/ {
    proxy_pass 
http://localhost:6081/VirtualHostBase/https/www.example.com:443/
VirtualHostRoot/misc_/;
    proxy_redirect default;
  }
  location /p_/ {
    proxy_pass 
http://localhost:6081/VirtualHostBase/https/www.example.com:443/
VirtualHostRoot/p_/;
    proxy_redirect default;
  }

  # This section governs access to the various HAProxy status pages.
  location = /haproxy-status { return 301 
https://www.example.com/haproxy-status/; }
  location /haproxy-status/ {
    proxy_pass http://backendserver:3380/haproxy-status/;
  }

# This is the default 'catch-all' statement. If a request doesn't 
# match any of the above rules, it gets sent to the application web 
# server, where it will probably die.
location / { proxy_pass http://localhost:3380/; }

  location = /400.html { root /var/www/status-codes/service1; }
  location = /401.html { root /var/www/status-codes/service1; }
  location = /403.html { root /var/www/status-codes/service1; }
  location = /404.html { root /var/www/status-codes/service1; }
  location = /405.html { root /var/www/status-codes/service1; }
  location = /408.html { root /var/www/status-codes/service1; }
  location = /411.html { root /var/www/status-codes/service1; }
  location = /500.html { root /var/www/status-codes/service1; }
  location = /502.html { root /var/www/status-codes/service1; }
  location = /503.html { root /var/www/status-codes/service1; }
  location = /504.html { root /var/www/status-codes/service1; }
}    

For additional help with my troubleshooting steps, below are some curl outputs that I've tried.

curl -I -k http://www.example.com
HTTP/1.1 200 OK
Server: nginx/1.13.4
Date: Thu, 24 Aug 2017 18:47:31 GMT
Content-Type: text/html
Content-Length: 1094
Last-Modified: Thu, 17 Aug 2017 18:46:39 GMT
Connection: keep-alive
Vary: Accept-Encoding
ETag: "5995e48f-446"
Accept-Ranges: bytes

curl -I -k http://www.example.com/service1
HTTP/1.1 404 Not Found
Server: nginx/1.13.4
Date: Thu, 24 Aug 2017 18:48:14 GMT
Content-Type: text/html
Content-Length: 169
Connection: keep-alive
Vary: Accept-Encoding

curl -I -k https://www.example.com
HTTP/1.1 200 OK
Server: nginx/1.13.4
Date: Thu, 24 Aug 2017 18:48:52 GMT
Content-Type: text/html
Content-Length: 1094
Last-Modified: Thu, 17 Aug 2017 18:46:39 GMT
Connection: keep-alive
Vary: Accept-Encoding
ETag: "5995e48f-446"
Alternate-Protocol: 443:npn-spdy/3,443:npn-spdy/2
Accept-Ranges: bytes

curl -I -k https://www.example.com/service1
HTTP/1.1 301 Moved Permanently
Server: nginx/1.13.4
Date: Thu, 24 Aug 2017 18:49:22 GMT
Content-Type: text/html
Content-Length: 185
Location: https://www.example.com/service1/
Connection: keep-alive
Alternate-Protocol: 443:npn-spdy/3,443:npn-spdy/2

Any help or info that you may be able to provide to resolve this issue would be greatly appreciated. Also, if there are some log output or other command outputs that you will need to better isolate the issue, please let me know as well.

Thank you!

EDIT 1:

@Tim I've tried to use a rewrite statement instead of the redirect but that didn't seem to make a difference. Below is one example that i tried.

rewrite ^ https://$host$request_uri? permanent;

Do you have a suggestion for me to try that may be better?

Since i do not have 10 rep points yet, this site wouldn't let me include more than 2 URLs. So to get past that, i had to add those spaces. I should have noted that ahead of time.

From what i read on the nxinx documentation, if there are no listen statements, it defaults to listen on all interfaces on port 80. I have also tried adding the listen statement as well with no change in outcome.

Below is the entire nginx.conf:

user  nginx;

worker_processes  auto;
worker_rlimit_nofile  20000;
pid  /var/run/nginx.pid;

error_log  /var/log/nginx/error.log debug;

events {
  worker_connections  4096;
  multi_accept on;
  use epoll;
}

http {
  include  /etc/nginx/mime.types;
  default_type  application/octet-stream;

  log_format  main  '$remote_addr - $remote_user [$time_local] 
"$request" '
                    '$status $body_bytes_sent "$http_referer" '
                    '"$http_user_agent" "$http_x_forwarded_for"';

  log_format  timed_combined    '$remote_addr - $remote_user 
[$time_local]  '
                                '"$request" $status $body_bytes_sent 
'
                                '"$http_referer" "$http_user_agent" '
                                '$request_time 
$upstream_response_time $pipe';

  log_format  graylog2_format   '$remote_addr - $remote_user 
[$time_local] '
                                '"$request" $status $body_bytes_sent 
'
                                '"$http_referer" "$http_user_agent" '
                                '"$http_x_forwarded_for" 
<msec=$msec|connection=$connection|connection_requests=$con$

  access_log  /var/log/nginx/access.log  main;
  access_log syslog:server=127.0.0.1:514 graylog2_format;

  sendfile on;
  tcp_nopush on;
  tcp_nodelay on;
  keepalive_timeout 65;
  types_hash_max_size 2048;
  send_timeout 600;

  gzip on;
  gzip_disable "MSIE [1-6]\.";
  gzip_static on;
  gzip_vary on;
  gzip_proxied any;
  gzip_comp_level 4;
  gzip_http_version 1.1;
  gzip_types text/plain
             text/css
             text/xml
             text/javascript
             image/png
             image/gif
             image/jpg
             image/jpeg
             application/json
             application/javascript
             application/x-javascript
             application/xml
             application/xml+rss;

  ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
  ssl_buffer_size 8k;
  ssl_session_cache shared:SSL:10m;
  ssl_session_timeout 10m;

  include /etc/nginx/conf.d/*.conf;
}
  • You don't have any "rewrite" commands, you have redirects set up. There are spaces in most of your curl commands, that's an invalid URL. There's no "listen 80" type directive in your forwarding server, but I assume it defaults to 80/http. Please do the curls again, show your complete Nginx server / site config, editing the question rather than replying in comments. – Tim Aug 24 '17 at 19:19
  • @Tim I've added the details that you requested. – D. Hair Aug 29 '17 at 12:41
  • Your http server block looks ok to me. My best guess is you've done something a bit odd that means the http server block you defined isn't being hit. Do you have a "default server" server block set up? If so make sure it logs to its own file and check that log file. Have you looked in the access / error logs to see what they say? – Tim Aug 29 '17 at 19:49
  • @Tim Yes, there is a default server config but I've gone as far as to take that out of the equation by moving the config file out of the directory. I have looked into the access/error log and this is the entry when i attempt to his the site: 2017/08/24 18:36:55 [error] 31476#31476: *919529 open() "/etc/nginx/html/service1" failed (2: No such file or directory), client: 1.2.3.5, server: , request: "GET /chub HTTP/1.1", host: "example.com" – D. Hair Aug 30 '17 at 13:17
  • That looks like a clue. I can't spot anything else wrong sorry. Maybe someone else can help, or maybe you'll need to hire someone to come in and take a look directly. – Tim Aug 30 '17 at 19:43

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.