DISCLAIMER: I am not an expert on, nor have I ever used, distributed filesystems or operating systems. Never really felt I needed either of them, and haven't wanted the additional overhead associated with maintaining them. But I've read up on them. So this is my two cents.
I can control 2 servers from the same ssh connection
Technically speaking, there would need to be two connections by definition.
SSH is not a multicast protocol. However, it is possible to do this provided you're willing to switch to a distributed operating system (such as, for example, Inferno). You'd accomplish your goal by SSHing into one of the machines' NICs and allowing the changes made on one machine to propagate to the other machine using the operating system's synchronization protocols.
Now, if that seems like too much (which it well may), you can set up shared directories (using
nfs, for example. I'd recommend against
samba for security reasons). Shared SSH doesn't really make much sense, although you can use a non-standard SSH client which dispatches SSH commands to multiple clients at once of which there are a few, easy-to-google options (including
multissh among many others).
The search string "control multiple machines over ssh at once" got me a bunch of helpful hits on duckduckgo.
Or am I better of having them as it is.
the multi-ssh stuff mentioned above won't incur any performance penalty, and doesn't require any special configuration of the servers, so if you want to start using one of those tools there's no reason not to, but
network filesystems will slow you down. The performance of a filesystem depends heavily on I/O latency, and the highest I/O latencies come from networking. At the moment, when one of those servers wants to retrieve a file, it only needs to access a shared drive over a high-speed connection within the same datacenter (read: within the same building). If you distribute the filesystem across the two machines, then their synchronization will always require packets to travel over unreliable links all the way across the country. Even assuming perfect connections, and assuming there's no processing delay for packets, this will take a few extra milliseconds each time (due to physical limitations on the speed of light). Usually, if you're going to use a file share or distributed filesystem, you want to limit that to machines within the same building. Otherwise, latency will become a seriously annoying issue.
A distributed operating system may or may not be a good idea. For one thing, AFAIK none of them are widely used or widely targeted as supported platforms for software, so you'll need to accept that whatever program you want to run probably won't run on the O/S you've chosen. For scientific computing applications- if you expect to do a lot of number crunching that can be broken up into a bunch of independent, resource-hungry subtasks that don't need to do much communicating- a distributed O/S is a great idea. However, again, you may as well do that with machines in the same datacenter. Or simply use one machine that's twice as beefy.