1

I currently have 5 servers. 1 server is used as the main server which holds all the video files, the other 4 are replicated and fetches videos from the main server using rsync. Videos are delivered through HTTP via nginx pseudo streaming.

What I would want to ask is if there are better approach on this? Would it be better if 1 specific server only holds that file? eg: file1.mp4 will not be replicated across all the servers and would remain only on one server? Would this approach lessen the IO load?

Because right now, viewers are randomly forwarded to any of those servers when viewing a video. Since all those servers holds the same files, I assume it takes time to seek because there are nearly 13,000 videos for each of them.

Also, are there tweaks to TCPIP/sysconf to optimize this video service?

Server config:

  • All servers are configured with raid 10 4x2TB 12GB memory.
  • 1GBPS unmetered (100tb.com)
  • CentOS
  • Nginx Pseudo Streaming

2 Answers 2

0

I'd replicate them over your servers, if you have a video that goes viral, you will get a huge load on one server. If you replicate them, you can distribute that load onto your other servers.

If it keeps growing like this, I suggest you look into special storage options like SAN.

0

Above all other things, start gathering metrics. Metrics on everything you can and store them. Find out what files are being hit, how often, which servers, what sort of requests come in, where they come from, bandwidth use, network latency, I/O access. Is your system waiting for the file system? Or for Nginx to process a request? Don't rely on hunches and thinking that the speed of your architecture is being held back because of one aspect. It could just as easily be another. Having said that, here are some little things you could alter.

You may want to switch off atime updates for the file systems if you haven't done so already. As this is some sort of web system you're better off utilising your logs rather than relying on the filesystem's notion of access time. This will avoid extra unnecessary writes to the disk.

The only TCP option I'd be tempted to alter would be to switch on Explicit Congestion Notification. Although it is not widespread, I think it will start to become more prevalent as time goes by. As it spreads, it will aid in reducing congestion on the links between your servers and your users.

One caveat with ECN is some buggy ancient routers ignore packets with ECN. If you're willing to take this risk, then switch it on.

You also may want to investigate buffer bloat if possible. Although the actual streaming isn't too bothered about buffers (as long as the video packets get there before the local buffer runs out the video player doesn't mind too much), the initial GET requests will be. It is just something to be aware of as video streaming is a sure fire way of congesting a network.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .