220

I use nginx as a reverse proxy. Whenever I update the config for it using

sudo "cp -r #{nginx_config_path}* /etc/nginx/sites-enabled/"
sudo "kill -s HUP `cat /var/run/nginx.pid`"

I face a brief downtime. How can I avoid that?

3
  • 1
    Are those meant to be command line commands? I've never seen anyone wrap an entire sudo command in quotes like that, it may not be necessary. Apr 12, 2016 at 12:59
  • 5
    Just a general comment: I think the standard/recommended practice is create a soft/symbolic link for your site configuration under sites-enabled, not copy it. Not related to your particular issue, but you may want to look into that. Apr 12, 2016 at 13:01
  • 2
    You should not be facing a downtime. kill HUP is the way to do a graceful reload in nginx. May 6, 2016 at 16:40

7 Answers 7

293

Run service nginx reload, /etc/init.d/nginx reload or /usr/sbin/nginx reload

It will do a hot reload of the configuration without downtime. If you have pending requests, then there will be lingering nginx processes that will handle those connections before it dies, so it's an extremely graceful way to reload configs.

Sometimes you may want to prepend with sudo

9
  • 16
    Both of those should do exactly what the question states: send SIGHUP to the nginx master process. There should not be a difference. nginx.org/en/docs/control.html
    – Gnarfoz
    Jul 27, 2012 at 12:36
  • When I issue the command on CentOS it keeps saying "Usage /etc/init.d/nginx (start..stop...restart..reload)" .. and that's exactly how I used it. Within the file /init.d/nginx I found kill -HUP cat $PIDFILE || echo -n " can't reload"
    – mashup
    Apr 13, 2015 at 18:21
  • 2
    do you know what the difference is between service nginx reload and nginx -s reload? If I run the former, I get this output: Reloading nginx configuration: nginx., but my changes aren't updated. If I run the latter, I get no output, but my changes are reflected.
    – Ryan Quinn
    Apr 21, 2017 at 3:46
  • I just tried this after adding a log_not_found directive but found that I had to actually do a restart to get it to work. I guess reloading does not work for all directives? Jun 22, 2017 at 12:24
  • I get downtime when I do this. For example I get blips like 502 bad gateway which is killing some requests. I'm running an old version of nginx - not sure if that's the reason
    – Mark
    Aug 5, 2020 at 14:48
148

Run /usr/sbin/nginx -s reload

See http://wiki.nginx.org/CommandLine for more command line options.

3
  • 1
    Finally, a command that does work in Debian Jessie. Jun 30, 2017 at 11:49
  • 2
    This is a better way. Because your server doesn't down if your configurations have errors (just shows errors in this case). Sep 19, 2018 at 14:51
  • 2
    if nginx default pid not in the default location , need '-p'. ie: ` /opt/gitlab/embedded/sbin/nginx -s reload -p /var/opt/gitlab/nginx`
    – qxo
    Jan 9, 2019 at 1:59
15

Nginx and Signals

The kill approach you used (kill -s HUP $(cat /var/run/nginx.pid) is correct. Init scripts for RH or Debian distributions are in the end also implemented using kill command. You can check Init example from nginx website or contents of Ubuntu Nginx package.

There are multiple signals, that nginx can listen to (mentioned in wiki):

  • TERM, INT - Quick shutdown.
  • QUIT - Graceful shutdown.
  • KILL - Halts a stubborn process.
  • HUP - Configuration reload. Start the new worker processes with a new configuration. Gracefully shutdown the old worker processes.
  • USR1 - Reopen the log files.
  • USR2 - Upgrade Executable on the fly.
  • WINCH - Gracefully shutdown the worker processes.

Nginx Reload

Nginx reload (HUP signal) is more specifically implemented as several steps [1,2]:

  • The master process checks the syntax validity.
  • Applies new configuration, that is, to open log files and new listen sockets.
  • If this fails, it rolls back changes and continues to work with old configuration.
  • If this succeeds, it starts new worker processes, and sends messages to old worker processes requesting them to shut down gracefully.
  • Old worker processes close listen sockets and continue to service old clients.
  • After all clients are serviced, old worker processes are shut down.

Only one issue I can think of why you had downtime (based on the reload process) is that you were using only one worker process (worker_processes directive), which by design was serving old clients, but had closed listen socket, therefore you couldn't open new connection.

I can also recommend you to always use /usr/sbin/nginx -t to validate configuration files before applying new config.

Nginx Reload in Depth

Reconfigure signal is handled in file ngx_process_cycle.c and we can see it starts new worker processes in function ngx_start_worker_processes(...) and at the end it stops old worker processes in function ngx_signal_worker_processes(...), which iterates over them with NGX_SHUTDOWN_SIGNAL signal.

Resources:

4
  • So that means that "nginx -s reload" does not restart the nginx service (i.e. resulting in a new pid) but just instructs the nginx service to reload its configuration without a restart? If I have a docker container in which I am attached to the nginx process with "nginx -g 'daemon off;' " making a reload would not kill my container then?
    – Crine
    Jul 10, 2020 at 18:51
  • 1
    @Crine No, it will not kill your container.
    – Lirt
    Jul 15, 2020 at 7:15
  • Interestingly, I'm doing this and it doesn't seem to be picking up changes made to the nginx.conf file. I've added a few location blocks to test and indeed hitting localhost/test didn't exist even after sending the nanny process a HUP (and verifying new worker process pids). Is there any insight available into the steps you outlined here (ie, applies new config, rolls back, etc)? Interested in debugging this.
    – a p
    Sep 20, 2021 at 21:42
  • @ap can you confirm that the configuration file you use is actually working and the issue is only in reload process?
    – Lirt
    Sep 21, 2021 at 9:17
12

No, you are incorrect, you aren't supposed to be facing any downtime with the procedure you describe. (Nginx can do not only configuration reload on the fly without any downtime, but even the upgrade of the executable on the fly, still without any downtime.)

As per http://nginx.org/docs/control.html#reconfiguration, sending the HUP signal to nginx makes sure that it performs a graceful restart, and, if the configuration files are incorrect, the whole procedure is abandoned, and you're left with the nginx as before sending the HUP signal. At no point should any downtime be possible.

In order for nginx to re-read the configuration file, a HUP signal should be sent to the master process. The master process first checks the syntax validity, then tries to apply new configuration, that is, to open log files and new listen sockets. If this fails, it rolls back changes and continues to work with old configuration.

12

For completeness, the systemd way of doing it:

systemctl reload nginx
2

Usually, reloading configuration file of a service should not affect the running service. However, this depends on how the SIGHUP signal is processed.

If a specific service is experiencing a downtime during reload, this can be circumvented by running the same service on multiple servers preferably using a load balancer. In this case, you can take out one server at a time and reload/restart it. Then, it can be re-added after confirming it is OK.

2
  • While this does not directly answer the question, this is definitely a best-practice scenario that the OP would be smart to follow for avoiding downtime in general.
    – Andrew M.
    Apr 11, 2012 at 17:29
  • 1
    Details on how nginx handles different signals: nginx.org/en/docs/control.html
    – Gnarfoz
    Jul 27, 2012 at 12:37
0

First, you sure true the config. You should check the config in nginx.conf. You can use command below for the check.

nginx -t

After that, if config is true, you can nginx reload. False is first of all must edit your config and than again check use nginx -t and nginx reload.

You can use the below command.

systemctl reload nginx

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .