Thoughts on network configuration in new office space

Question

Within a month we're moving to a new office space for our company with ~ 35 people, and me, as the only system administrator, is in charge of setting up the new network for all four floors, and I'd like to have your thoughts and suggestions on how this could be done the best.

All I know, so far, is that three floors will have a 9U cabinet for switches, and one of the floors will have a full rack (the IT dept. floor). On the IT. dept. floor, we will have two incoming connections from separate service providers.

So my questions are ...

1. What equipment would you use for this? On the IT. dept. floor as well as the other three floors.

2. What technology would you use for the LAN itself?

3. Since one of the service providers is solely for fail over purposes, how would you handle an outage of the first service provider, as in, failing over to the second one? (I don't mind manual labour, but if it can be done without it, of course that would be the best way)

As I said, this is a blank paper for anyone to fill in, I already have my own ideas in mind but I'd like to hear some from other points of views as well as from people with experience, and please, be thorough. :-)

EDIT: We also need some sort of VPN functionality on whatever router is in use, for telecommuting purposes.

Answer 1

I would go with the following.

Run two sets of CAT6 to each floor to the cabinet. Terminate the other end at the IT floor. Use managed switches on each floor, suggest 1 GB ports. I suggest managed for future growth. Use one of sets of CAT6 for connecting to each switch. Keep the second in reserve in case of a failure. Keep a spare switch on site if possible.

I would personally go with both a switch and a router on the IT floor, allowing separate subnets for each floor. The router isn't strictly required but it allows more room for future growth. Untangling subnets that have static IPs thrown in when you are growing is not fun.. Since you have a clean slate this would be a good time.

As far as your request for failover / firewall configuration goes there are a lot of options. Pretty much any product will handle a manual failover as long as you have access to the building to make the change from the inside.

Product recommendations are a bit out of scope but I will share a couple additional thoughts - Cisco and Juniper make good networking equipment, both routers and switches. HP and Dell make decent lower cost switches but there can be some compatibility problems in the way they detect network loops. I don't have much love for the Cisco ASA series of firewalls. SonicWalls, Forefront and the various Linux derived firewalls have treated me well.

Answer 2

Some sort of failover that's local to your site (like switching the ethernet cable on your exterior gateway from FOO-ISP to BAR-ISP when FOO-ISP goes dead) only makes sense if all of your connections are outbound. Otherwise, you're going to need your own netblock (set of IP addresses that's assigned to your company) and run BGP on your exterior router to both of your ISPs (question: do you have physical backup on the "last mile", in case someone with a backhoe decides to dig before searching for buried cable?) so that it will automatically switch over in case of an outage from one of them.

I second @Tim Brigham's recommendation to use smart switches, as they have all sorts of features that will save your bacon when users decide to start randomly plugging cables into ports - and that's in addition to keeping your data closets locked with keys limited to only those who need them, right?

For your floor-to-floor wiring, I would run fibre from each of the floor's switches to your central router to allow for easy future speed upgrades - and yes, gigabit ethernet is the minimum spec you should be installing. Someone will be trying to push massive amounts of data across the wire and to do so on mere 100mbit ether is maddening. Any smart switch worth buying has at least a pair of mini-GBIC ports in it these days — you might even luck out and find one with transceivers included.

You'll need to match your GBICs to your fibre (in terms of wavelength). The other thing that I've found is that it's best to maintain a single brand and model number of GBIC in your shop if it's at all possible, as they work better together than a random pair. Worst case, you keep them in matched pairs.

Another thing that I've done is to physically separate the VoIP network from the data network. Yes, it's twice the infrastructure, and a pain when someone decides they want to plug into the wrong port, but it keeps the phones happy, even if someone is abusing the network/switches with heavy data transfers.

And when you allocate your subnets? Don't put them sequentially. E.g. don't use 10.0.1.x for the first floor, 10.0.2.x for the second, etc. You've got a lot of address space - spread it out, as people will chew through addresses like you wouldn't believe, and you don't want to be stuck renumbering/re-netmasking people during the workweek.