In our VMWare infrastructure, I have a production network ( subnet) and an isolated virtual lab network (also which is created by Veeam, with the Veeam proxy appliance sitting between them.

I created a NAS VM (FreeNAS 9.10) with two virtual NICs, and I wanted to connect one to the production network, and one to the isolated lab network, for easy transfer of large files between the two. However, this does not work. It seems that whichever network is connected first works (I can ping the VM, pull up the Web GUI, pull up the Windows Share) but when I connect the second network, the VM cannot be accessed by the second network. Even if I set a static IP on both, and use a different IP address on each virtual NIC, only the first network connected seems to work.

In my primitive understanding of networks, I'm thinking maybe the traffic INTO the NAS on that second network is working, but the NAS is trying to reply on and thinks it should do so on the other virtual NIC? Maybe this setup is impossible? Although, that seems unlikely, since somehow the Busy Box virtual appliance/router/proxy that Veeam creates can handle a similar setup.

  • My workaround so far is to just keep virtually "unplugging" one of the VM NICs from one network, and plugging it into the other. So the NAS is only connected to one network or the other. Its annoying to have to do, but it works and is relatively quick. – Adam Nofsinger Aug 16 '16 at 15:42

Having the same network on both NICs in your NAS is your problem. It's confused.

  • No way around this? Or is this sort of a heavy linux networking/router question beyond the scope of a Q&A site? I mean, I know the Veeam Proxy Appliance has two NICs with the same networks on them (the production, and the virtual test lab Granted, I don't know if a web server or anything other than the gateway/proxy services would work if I attempted to access them from both networks. – Adam Nofsinger Aug 16 '16 at 15:38
  • 2
    You don't understand the basics of IP networking - get someone in to help you, you're out of your depth. – Chopper3 Aug 16 '16 at 15:44
  • @Chopper3 - I assure you I at least know "the basics" and I understand what Veeam engineers were able to accomplish with their proxy appliance. I added an answer related to netfilter marked packets, which I think is the key ingredient I was missing. – Adam Nofsinger Aug 23 '16 at 14:15
  • @AdamNofsinger If you ever figure it out, I'd be interested to know. In fact, I think it would be worthwhile posting a question very specifically asking about this and answering it yourself for all to clearly see. This one is getting kind of messy. – Ryan Babchishin Aug 23 '16 at 17:34

A good practise is to use different network ranges for production and lab. For example, you use subnet for production and subnet for lab.

Update : According to your needs, you want the NAS to be seen with the same IP adress in the prod subnet and the lab subnet. What I would do is set up NAT (on a small fw or linux box or linux VM) in front of the NAS. Therefore prod and lab with have same IP adress while still being isolated.

| PROD |
| NAS                  |
| re0     |
| re1 |
| VMFW                 |
| eth0|
| eth1   |
| LAB      |

edit don't forget a few network magic to send all network destinated to interesting ports to

  • The network ranges being the same is sort of a requirement for this setup. The Virtual lab is supposed to be an exact (nearly) replica of production network, so don't want to even change IP addresses of hosts copied over there. And also want it isolated for user - it has a copy of our domain controller, for instance. – Adam Nofsinger Aug 16 '16 at 15:37
  • That is simply not feasible. As Chopper3 has already mentioned in a previous comment, it seems that you don't understand the basics of IP networking (this is a fairly fundamental principle which you are trying to contravene); please get someone to help you or start out by educating yourself on the basics before continuing this line of questioning. – BE77Y Aug 16 '16 at 16:20
  • not feasible? I don't know about freenas, but with freebsd you could set up jails having the same ip adress on 2 differents physical interfaces serving the same filesystems like a nas. The real hard part of the question is how to achive this in a not too dirty way with freenas inside a wm. – bgtvfr Aug 16 '16 at 16:52
  • @bgtvfr - sorry, to be clear my comment was in reply to Adam Nofsinger's comment - not your answer. – BE77Y Aug 17 '16 at 8:21
  • @bgtvfr - I think you may be on to something there. FreeNAS actually does have some capability to setup Jails (I actually had no idea what that means, but I'm thinking it is like an isolated virtual FreeNAS within FreeNAS?) doc.freenas.org/9.10/freenas_jails.html Maybe I'll head down that route and update with an answer if that works. – Adam Nofsinger Aug 17 '16 at 13:03

I think @bgtvfr has a very good, very close answer. I've sort of given up on this whole idea for now, but I know it is possible. The Veeam Virtual Lab proxy appliance does this for us on our network, and works well, and after doing some research and looking at all of the startup scripts on this (BusyBox Linux) device, I believe the key piece of configuration I'm missing is using something like netfilter to mark packets.

E.g. from https://unix.stackexchange.com/a/4421

The following commands create an alternate routing table via eth1 for packets that have the mark 1 (except packets to localhost). The ip command is from the iproute2 suite

ip rule add fwmark 1 table 1 
ip route add table 1 dev lo
ip route add table 1 dev eth1 

The other half of the job is recognizing packets that must get the mark 1; then use iptables -t mangle -A OUTPUT … -j MARK --set-mark 1 on these packets to have them routed through routing table 1. I think the following should do it (replace by the address of the non-default-route interface):

iptables -t mangle -A OUTPUT -m conntrack --ctorigdst -j MARK --set-mark 1 

I'm not sure if that's enough, maybe another rule is needed on the incoming packets to tell the conntrack module to track them.

Here is the clue I found in the configuration / init file for the Veeam Proxy appliance, where eth0 and eth1 are both subnet - eth0 being my production network and eth1 the virtual test lab: Veeam Virtual Appliance iptables commands I'm just too green with netfilter/iptables, but seems like maybe this is marking packets that come in on the virtual lab interface so that they go back out on the same interface rather than the default gateway interface?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.