Tag Info

Hot answers tagged

11

Okay, I have found the answer finally. The optimal number of Unicorn workers is not directly connected to the number of CPU cores, it depends on your load and internal app structure/responsiveness. Basically we use sampling profiler to determine workers' state, we try to keep workers 70% idle and 30% doing the actual work. So, 70% of the samples should be ...


9

It sounds like the bottleneck is the app powering the socket rather than it being Nginx itself. We see this a lot with PHP when used with sockets versus a TCP/IP connection. In our case, PHP bottlenecks much earlier than Nginx ever would though. Have you checked over the sysctl.conf connection tracking limit, socket backlog limit net.core.somaxconn ...


8

I would start with Ruby On Rails Security Guide. I am looking to secure them against exploits which give remote shell. Pay attention to the File Uploads section. Make sure file uploads don't overwrite important files Validate file name Use plugins: attachment_fu or paperclip Never to allow users to upload: .php, .cgi,... Set apropriate Centent-Type ...


7

This is normal. You don't need HAProxy, though HAProxy may provide you with "fairer" load balancing between the app servers than the nginx load balancing. So, on your app servers, set Unicorn to listen a network interface. This should preferably be a private IP address. For this example, let's say your app servers are 192.168.1.100, 192.168.1.101 and ...


7

Heroku (for example) puts nginx in front of varnish probably for the reasons you mentioned. So, nginx -> varnish -> app servers. At my work, we use Varnish in front of everything (varnish -> nginx -> apache/wsgi) and it works pretty well. The thinking there is that if there's a cache hit, it is served from the highest point in the stack. There are a few ...


7

I picked little a different solution from SpamapS's.. I'm also running an app with preload_app = true, managed by Upstart. When I was looking to solve this problem myself, I'd been using Upstart's "exec" to start my app ("exec bundle exec unicorn_rails blah blah"). Then I found your question, and it made me realize that instead of using Upstart's "exec" to ...


6

As of upstart v1.10.0 you can define a "reload signal". https://launchpad.net/upstart/1.x/1.10.0 This version is available on ubuntu as of v13.10 ... I used these instructions to upgrade my ec2 instance: http://askubuntu.com/questions/302762/upgrading-13-04-to-13-10 I use rails+puma and do graceful reloads with this directive in my /etc/init/app-1.conf ...


5

That's not a configuration option, it's a Ruby code snippet that tells it to set copy_on_write_friendly if the GC object has that method. For example, in ruby mainline 1.9.2p290: 1.9.2p290 :003 > GC.copy_on_write_friendly NoMethodError: undefined method `copy_on_write_friendly' for GC:Module from (irb):3 from ...


5

You're right about N+1 for CPU-bound jobs. On the other hand, unicorn does not use threads, so every IO op. blocks the process and another process may kick in and parse HTTP headers, concatenate strings and do every CPU-intensive tasks it needs to serve the user (doing it earlier to reduce request latency). And you may want to have more threads/processes ...


5

When an app gets slammed there's usually only one thing to do in the short term. Throw money at the problem. Also known as keep adding servers. In the medium term tune and tweak the settings. In the long term, fix all that crappy code that's there that didn't need to be fixed before. It's time to pay off that technical debt.


4

If I understand you well, you are trying to start a service without sudo under normal user. This will not work and will give your errors like the ones you received Operation not permitted. This is true in most (if not all) cases because any service will require one or more of the following to do its job: Binding/listening on ports < 1024. Reading ...


4

You should add option forwardfor to Haproxy config and configure nginx realip module: http://nginx.org/en/docs/http/ngx_http_realip_module.html


4

Change the server_name directive in your example.com container: server { listen example.com:80; client_max_body_size 1G; server_name example.com www.example.com; keepalive_timeout 5; root /home/example/application/current/public; try_files $uri/index.html $uri.html $uri @example_application; location @example_application { ...


4

I found the solution to this problem. It turns out that there were messages that I missed in my logfile that were indicating that the startup process of the new child failed. The reason for this failure is that I did not declare unicorn in my Gemfile. For some reason, it seems that it is possible to start unicorn from the command line without unicorn ...


3

You don't need multiple nginx instances, you could specify multiple proxy_pass with different unix sockets or defined multiple upstream sections.


3

As far as I know, Nginx automatically tries every available server - and if all return an error, it will return the error code of the last response. proxy_next_upstream just seems to offer some finer control as to what errors will be processed and which will be ignored (but the default is already any 'error' in connecting, sending, or receiving - with, I ...


3

Indeed, a limitation of upstart is that it cannot track daemons that do what unicorn is doing.. that being fork/exec and exit their main process. Believe it or not, sshd does the same thing on SIGHUP, and if you look, /etc/init/ssh.conf makes sure sshd runs in the foreground. This is also one reason apache2 still uses an init.d script. It sounds like ...


3

Ubuntu 10.04 comes with upstart version 0.6.6 which has the kill signal and kill timeout parameters which would configure relevant signal and timeout values. The "reload" signal is hardcoded to SIGHUP. Newer versions of upstart introduce the reload signal parameter - see other answers to this question for further detail.


3

Unix sockets only work within that system. A socket across the network is called a TCP connection. The way to do this is have Unicorn listen on a TCP socket and configure nginx to use them as upstreams (as you mentioned). Unicorn doesn't have distributed load balancing; you need nginx to do the load balancing.


3

Short answer: Nginx really only has one job, and that job is to accept an incoming request and hand it off to a backend server. Given that, your front end server(s) only need to run nginx and your backend server(s) only need to run rails. Make sense? Now, if your backend and frontend are both running on the same server then, yes, of course, it would have ...


3

When working with Rails, nginx can be used to serve several roles, most often as an HTTP load balancer or as an app server front-end. Your setup is a common one that will work, but may impose scaling challenges as you grow to higher levels of traffic. When you're working with a handful of VPS servers as you describe, one nginx instance can server as both ...


3

Sounds like requests queuing in your ELBs due to (maybe) long Keep-Alive headers. ELBs will try to minimize the number of connections to the backend by honoring the Keep-Alive header. Short (or disabled) keep-alive will force ELB to open a single connection per request to your backend, should your backend nodes have free resources to handle that incoming ...


2

I work at Rackspace and we would like to help you resolve this issue. If you could give us a call at 1-800-961-4454 we can check the health of the host that your server is on and move it to a new one if it does appear to be a noisy neighbor issue. I would also be interested in seeing the output of 'vmstat -S M 1 10', 'sar -b' (after some time has passed) and ...


2

The god process should be controlled by upstart scripts thus running it as a sudo user. This is made possible by binstubs. A binstub is a shim to an exectuable installed by a gem. Binstubs are provided by bundler. They allow bin files to be run from any where on the system. It's nessory to add rbenv paths in your upstart scripts to do this. export ...


2

nginx runs as nginxuser as your configuration states. You say your directories are owned by root:root. Ensure that each and every directory from the root to the file attempting to be accessed by nginx has the right execution (and maybe reading?) permission. In your case, the other permission group (chmod o+(r)x <recursive list of dirs>). If that is ...


2

The issue if you want to serve them through proxied servers is that the regex location block has higher priority in nginx location's search when URI matches. So, remove the last location block or write one unique fallback location and use try_files. open_file_cache max=10 inactive=10m; open_file_cache_valid 5m; open_file_cache_min_uses 1; ...


2

To debug this, I would systematically isolate each component to find the bottleneck. At all times I would tail -f any log files, including nginx access/error logs and the rails logs for errors. The first question is, is it slow in production only, or also in development mode? The answer will help you diagnose by focusing on what's different between the ...


2

I finally figured it out. The problem was that the deployer user's primary group was wrong. It should be 'staff' but was 'deployer' instead. This means that unicorn tries to hand over the ownership of new worker processes to the group it's supposed to use, but only root can do that. Just in case anybody else needs to know, I changed the primary group by ...


2

To prevent clients from forging a proxy's headers, both X-Forwarded-For and Client-Ip have to equal to not raise this kind of error. Simply set the Client-Ip header in your nginx config: As for the IP addresses in your log: 192.168.255.5 probably is NodeBalancer's internal IP (examples in Linode Library show 192.168.0.0/16 subnet as usual; that should ...


2

There are a lot different ways to improve the performance of your app. You're already doing one, optimizing. Especially if you start optimizing queries (ie fixing for n+1 queries). There are 3 other areas I would look into as well. Scaling vertically & scaling horizontally (http://en.wikipedia.org/wiki/Scalability#Scale_horizontally_vs._vertically), and ...



Only top voted, non community-wiki answers of a minimum length are eligible