2

Later edit After a LOT of troubleshooting the actual issue turned out to be a missing semicolon after the server_name directive. nginx -t -c /etc/nginx/nginx.conf wasn't catching it. Double-check for typos if you ever run into a case similar to this.

Original question follows:

I'm in the process of setting up a new server built on ubuntu 16.04 using nginx 1.10.0.

The specific issue is that while my new configuration essentially matches my old nginx configuration from an ubuntu 13.10 server using nginx 1.4.4, nginx 1.10.0 is only creating either ipv4 or ipv6 workers, but not both. This behavior is not present on the old server. Not sure what else to try at this point.

I've verified that my nginx installation was built with ipv6.

nginx version: nginx/1.10.0 (Ubuntu)
built with OpenSSL 1.0.2g-fips  1 Mar 2016
TLS SNI support enabled
configure arguments: --with-cc-opt='-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_addition_module --with-http_dav_module --with-http_geoip_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_v2_module --with-http_sub_module --with-http_xslt_module --with-stream --with-stream_ssl_module --with-mail --with-mail_ssl_module --with-threads

Below are my current configurations for the new server:

# /etc/nginx/nginx.conf
user www-data;
worker_rlimit_nofile 30000;
worker_processes 8;
pid /run/nginx.pid;

events {
  worker_connections 500000;
}

http {
  sendfile on;
  tcp_nopush on;
  tcp_nodelay on;
  keepalive_timeout 65;
  types_hash_max_size 2048;

  include /etc/nginx/mime.types;
  default_type application/octet-stream;

  ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
  ssl_prefer_server_ciphers on;

  access_log /var/log/nginx/access.log;
  error_log /var/log/nginx/error.log;

  gzip on;
  gzip_disable "msie6";
  gzip_vary on;
  gzip_proxied any;
  gzip_comp_level 6;
  gzip_buffers 16 8k;
  gzip_http_version 1.1;
  gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

  include /etc/nginx/conf.d/*.conf;
  include /etc/nginx/sites-enabled/*;
}

And I have a single site enabled at the moment for testing. I will have multiple vhosts configured eventually.

# /etc/nginx/sites-enabled/blog
server {
  server_name test.bloggyblog.com

  listen 80;
  listen [::]:80; 

  root /usr/local/apps/blog;
  index index.php;

  location / {
    try_files $uri $uri/ =404;
  }

  location ~ \.php$ {
    try_files $uri =404;
    fastcgi_split_path_info ^(.+\.php)(/.+)$;
    fastcgi_pass unix:/var/run/php/php7.0-fpm.sock;
    fastcgi_index index.php;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    include fastcgi_params;
  }
}

Lastly, the weird thing is whether the workers get bound to ipv4 or ipv6 entirely depends on the order in which the listen directives are placed. In the following data, I've switched the order and tried different configurations multiple times. After each change to /etc/nginx/sites-enabled/blog I did sudo service nginx stop; sudo service nginx start; sudo lsof -i; to get the data.

Also note that I changed the workers count to 8 after performing these steps. However while the number of workers increased, the same behavior was seen where all workers were either ipv4 or ipv6.

listen [::]:80; 
listen 80;
nginx    27675     root    6u  IPv4 204423      0t0  TCP *:http (LISTEN)
nginx    27676 www-data    6u  IPv4 204423      0t0  TCP *:http (LISTEN)

listen 80;
listen [::]:80;
nginx    27747     root    6u  IPv6 205134      0t0  TCP *:http (LISTEN)
nginx    27748 www-data    6u  IPv6 205134      0t0  TCP *:http (LISTEN)

listen 80;
listen [::]:80 default ipv6only=on;
nginx    27819     root    6u  IPv6 205849      0t0  TCP *:http (LISTEN)
nginx    27820 www-data    6u  IPv6 205849      0t0  TCP *:http (LISTEN)

listen 80;
listen [::]:80 default ipv6only=off;
nginx    27885     root    6u  IPv6 206495      0t0  TCP *:http (LISTEN)
nginx    27886 www-data    6u  IPv6 206495      0t0  TCP *:http (LISTEN)

listen 80;
listen [::]:80 default;
nginx    27953     root    6u  IPv6 207184      0t0  TCP *:http (LISTEN)
nginx    27954 www-data    6u  IPv6 207184      0t0  TCP *:http (LISTEN)

2 Answers 2

3

It looks like your default setting for ipv6only is different. On most operating systems you can create IPv6 sockets that also accept IPv4 connections so that you only need one socket (one listen directive).

It seems that on your old server it used ipv6only=on so you created both an IPv4 and an IPv6 socket. On your new server the default is ipv6only=off, which makes the IPv6 socket listen on IPv4 as well. And that creates a conflict with the separate IPv4 socket. If you remove the IPv4 listen line it will probably just work with both protocols.

To make things predictable it's probably best to explicitly set the ipv6only flag, and use one of these:

listen 80;
listen [::]:80 ipv6only=on;

or

listen [::]:80 ipv6only=off;
4
  • Yeah, my old server config has ipv6only=on. However, I've tried the combinations you mentioned in my original question. I only get either ipv4 or ipv6 connections, never both. The workers are always one or the other. I've been testing the connections from another server that has ipv4 and ipv6 connectivity, and when the workers are set to one state, the other can't connect.
    – Geuis
    Commented Jul 26, 2016 at 19:40
  • I don't see the listen [::]:80 ipv6only=off; without the listen 80; in your test results. Did you try that one? Commented Jul 26, 2016 at 22:06
  • yah I was getting the same result.
    – Geuis
    Commented Jul 26, 2016 at 22:12
  • 2
    Turns out it was a freaking typo that "nginx -t -c /etc/nginx/nginx.conf" wasn't catching. I didn't have a trailing ";" after the server_name directive. Everything is now working exactly as expected with net.ipv6.bindv6only = 0. Ughhhh. Thanks for your help!
    – Geuis
    Commented Jul 26, 2016 at 22:31
2

So I was finally able to resolve this, but not in any way I've seen described anywhere else.

I set "net.ipv6.bindv6only = 1" in /etc/sysctl.conf and reloaded via "sudo sysctl -p". Then with the configuration in the server directive "listen 80; listen [::]:80;" I now get my workers split evenly between ipv4 and ipv6 and now I can connect from either ip source.

Add to /etc/sysctl.conf and reload with "sudo sysctl -p":

net.ipv6.bindv6only = 1

Sample nginx config:

server {
  server_name test.bloggyblog.com

  listen 80;
  listen [::]:80;

  root /usr/local/apps/blog;
  index index.php;

  location / {
    try_files $uri $uri/ =404;
  }

  location ~ \.php$ {
    try_files $uri =404;
    fastcgi_split_path_info ^(.+\.php)(/.+)$;
    fastcgi_pass unix:/var/run/php/php7.0-fpm.sock;
    fastcgi_index index.php;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    include fastcgi_params;
  }
}

Reload nginx

sudo service nginx reload

Then "sudo lsof -i"

nginx 2096 www-data 6u IPv4 286321 0t0 TCP *:http (LISTEN) 
nginx 2096 www-data 7u IPv6 286322 0t0 TCP *:http (LISTEN) 
nginx 2097 www-data 6u IPv4 286321 0t0 TCP *:http (LISTEN) 
nginx 2097 www-data 7u IPv6 286322 0t0 TCP *:http (LISTEN) 
nginx 2098 www-data 6u IPv4 286321 0t0 TCP *:http (LISTEN) 
nginx 2098 www-data 7u IPv6 286322 0t0 TCP *:http (LISTEN) 
nginx 2099 www-data 6u IPv4 286321 0t0 TCP *:http (LISTEN) 
nginx 2099 www-data 7u IPv6 286322 0t0 TCP *:http (LISTEN) 
nginx 2100 www-data 6u IPv4 286321 0t0 TCP *:http (LISTEN) 
nginx 2100 www-data 7u IPv6 286322 0t0 TCP *:http (LISTEN) 
nginx 2101 www-data 6u IPv4 286321 0t0 TCP *:http (LISTEN) 
nginx 2101 www-data 7u IPv6 286322 0t0 TCP *:http (LISTEN) 
nginx 2102 www-data 6u IPv4 286321 0t0 TCP *:http (LISTEN) 
nginx 2102 www-data 7u IPv6 286322 0t0 TCP *:http (LISTEN) 
nginx 2103 www-data 6u IPv4 286321 0t0 TCP *:http (LISTEN) 
nginx 2103 www-data 7u IPv6 286322 0t0 TCP *:http (LISTEN)

Confirm with "netstat -tlp"

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name              
tcp        0      0 *:http                  *:*                     LISTEN      -               
tcp6       0      0 [::]:http               [::]:*                  LISTEN      -
3
  • Watch out for side effects with other services that might be relying on the behavior you've disabled with that sysctl. Commented Jul 26, 2016 at 21:30
  • @MichaelHampton Yeah you're not wrong. But so far I'm not finding any other way to get this to behave like everyone says it should.
    – Geuis
    Commented Jul 26, 2016 at 21:40
  • 3
    @MichaelHampton Turns out it was a freaking typo that "nginx -t -c /etc/nginx/nginx.conf" wasn't catching. I didn't have a trailing ";" after the server_name directive. Everything is now working exactly as expected with net.ipv6.bindv6only = 0. Ughhhh.
    – Geuis
    Commented Jul 26, 2016 at 22:30

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .