As far as the NIC teaming is concerned your coworker is more or less correct.
By default NIC teaming in ESX maps each Virtual NIC in your VM's to a single uplink (the physical NICs) on the vSwitch that it is connected to. The specific NIC load blancing policies are:
- Port ID: all traffic from each
Virtual NIC is mapped to one uplink
based on port number.
- Source Mac hash: all traffic from
each Virtual NIC is mapped to one
uplink based on a hash of the Virtual
NIC's MAC address
- IP hash: a hash of both the source
and destination ip-addresses for
ip-based traffic is used to select
Of these three only IP hashing will give you any aggregation effect. ESX can only control outbound traffic path selection so to get proper distribution of both inbound and outbound traffic your phyisical switches must also be configured appropriately for port aggregation (Etherchannel\LACP).
There is a very useful VMware KB article here about how to configure various switches (Cisco\HP) so that both inbound traffic (where the switch has to decide the path selection) and outbound (where ESX handles the path selection) are distributed.
Note that none of these policies will ever result in traffic being distributed across more than one uplink for traffic between a single source ip-address and a single destination, they only provide aggregation when there is a range of ip-addresses involved.
To answer your second question - that depends on how much data each request involves. A single 100Megabit connection can push through about 8000 packets/sec, possibly more if the payload size is much smaller than 1500 bytes. However (simplifying things massively and ignoring overheads) if a typical request involves 30k of data for example, then each one will need 20 packets to complete so the NIC can theoretically handle about 250 such requests/sec. If your requests involve 1Meg of traffic on average you are down to 7/8 requests per second at best and under normal conditions I'd say you'd be doing well if you got actual numbers that were >50% of those rates.
That is just to get a rough idea if the link can carry the raw data bandwidth. To get closer to what a specific link can handle you also need to factor in all the handshaking involved, end to end latency between the client and the server, how many concurrent connections your architecture can keep in flight and a lot else.