3

I want to host (co-located) a service at a Pier1, Q9 or some similar hosting facility.
Is it best practice to put my failover hardware in separate cabinets, and in your experience, will the hosting provider allow this?

My concern is that if something happens, say catches fire in the cabinet or a disgruntled rack renter pours his coke into the servers, both my HA stacks will be compromised.

In your experience, if I put them in separate cabinets, can they be on the same subnet, or is it against hosting policy for me to run a cable out of one cabinet and into another?

Or is it the case that these kind of events never occur and I should quit worrying?

Edit: It is a high business value, stateless web-application where seconds of downtime would be very detrimental.

3
  • 2
    More detail is needed. A stateless web application is different than a database app, which is different than an ERP solution or a high-transaction financial application.
    – ewwhite
    Jan 13, 2013 at 1:30
  • edited, (stateless, webapp)
    – Will
    Jan 13, 2013 at 1:54
  • 2
    It depends on how many Nines you're chasing. More nines will require more redundancy, you have to start having multiple of more and more. Basic redundancy starts with a RAID, UPS, then dual NICs.. Active/Passive server clusters, multiple power sources, and multiple Internet connection.. Eventually you get to the point of having multiple datacenters with anycast address scopes. Each step costs progressively more and provides better uptime; you have to evaluate the likelihood of downtime, multiply by the cost and that's about where you should be spending on redundancy.
    – Chris S
    Jan 13, 2013 at 2:59

6 Answers 6

3

It depends on the HA requirements of your service.

If the service is not that important to the business, there probably is not point in separating the cluster nodes between cabinets, especially if the surrounding infrastructure, like core network, load balancers etc. are also divided.

In high business value services, separating the nodes in failover cluster to separate fire chambers is a common practice and I recommend to do so if the business sees it worthwhile and is willing to pay the price.

Yes, the hosting provider will allow you to do it, unless it is one of the smallest and just does not have multiple cabinets.

Normally you do not run your own cables between cabinets. The provider offers the interconnection between the cabinets you as a service and you have your core network bridged with it or the provider may also offer the network as a service.

The nodes in the cluster may use the same VLAN, which still remains as a SPOF in addition to the clustering technology itself and probably also the common storage. To prevent the services from failure in those components, you'd need to build yourself a disaster recovery system, which is commonly located geographically far enough and where the service is mirrored asynchronously to.

2
  • You're assuming too much about his application(s) to make any claims about VLAN requirements, subnets, etc. It could be a simple website that just needs a front-end loadbalancer. It could be something very complex with multiple subnets. We don't know; you shouldn't assume.
    – mfinni
    Jan 12, 2013 at 22:45
  • 1
    Agreed. There is no point of dividing the cluster members between cabinets unless the components around like the Load Balancer also are HA enough. I edited my answer.
    – grassroot
    Jan 12, 2013 at 23:29
6

Single rack power failure is more common than most of the other concerns you've listed: A single PDU strip popping or a circuit getting overloaded... I've had that happen a handful of times, but none of the other issues you listed.

Ideally your 'HA' solution uses two different datacenters. (eg. two zones in a AWS region)

2
  • 1
    +1 Different data centers is the way to go.
    – stoj
    Jan 13, 2013 at 2:22
  • 1
    Any quality data center is going to provide A+B feed power (allowing 80% utilization to prevent your stated scenario) via separate power distribution units. And the internal redundancies of the server (e.g. dual power supplies) are meant to take advantage of that. It's a big jump from that to multiple data centers.
    – ewwhite
    Jan 13, 2013 at 6:08
3

Not necessarily. It depends on what types of situations you realistically want to protect against.

I've had several environments where the HA strategy was to build parallel application stacks in multiple cabinets. That meant all tiers of the application/service; switching, firewalls, etc. were present in two cabinets, such that we could lose one entire rack (or any component within) without affecting application availability.

You don't specify the scope or nature of your application. More detail would help there. But if you're talking about something small like a single webserver, this is not necessary. Geographical separation, DNS failover and making sure that individual servers have internal redundancies (dual power supplies, redundant fans, RAID) will be a good start.

Running cabling between cabinets (cross-connects) within a facility isn't uncommon. Usually, it's the result of not being able to obtain contiguous space. But I would not worry about a technician spilling a liquid beverage on your rack or fire... Although, component fires can occur... but I had an entire failover infrastructure stack to avoid downtime.

1

It depends. Your fears are reasonable, though the issues you describe are rare.

I would say that it is certainly better to have the redundant unit in another rack, if not another building/city. I'm not sure that I would spend extra just to have the redundant server in an adjacent rack. But, if the cost is the same, it can't hurt.

The hosting providers won't allow you to run your own cable between cabinets. But, they will put your two network ports(one in each cabinet) on the same VLAN for you, so you can use the same subnet if that is important to you.

Only you can decide what a failure is worth to you. Once you have that figured out, you'll know if it is worth worrying about or not.

1
  • 1
    I'm allowed to run any cables except for power in my colo facilities; one of them even installed a bridge for me. Jan 12, 2013 at 23:00
1

The risk of fire or sabotage you mention, or other similar environmental problems, is unlikely to be adequately mitigated by placing your tech in racks that are near enough to make running a cable between them viable. Consider the storms last year in the NYC area: whole datacentres were off the grid.

I'd suggest that its not worth worrying too much about HA unless you've got proper seperation between your cluster members, or at least if you're very clear that this is being implemented to deal with the normal run of errors (e.g. servers breaking down) rather than anything more exotic.

0

The key point of resilience is that it can go as deep or as shallow as you wish (or have funds for). The technical equivalent of that is you can unicast or multicast as much as you want (or you have funds for).

If your business has a requirement for resilience, the business will identify how far that resilience should deliver. If you have a network that is required 5 minutes every day then what benefit is resilience? If your network is a hospital and relevant 24x7x365 then you have a completly different objective.

I have worked in an environment where ex-Cisco employees have tried to dictate the requirements and what I must do and yet they have misunderstood by looking at the infrastructyre rather than what the business/client requires.

If you need full resilience, the business will meet the cost. If you dont, dont bother..

It really is someone else's decision.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .