WARNING about your setup
I would advise you caution using this system in this setup, if you have only one physical adapter and then only one uplink to the universe from the system. I would suggest going and purchasing an extra adapter for the system, and add that to your system - this way you can still have a backup physical connection.
The other headache you're going to be facing with only one uplink is that you will have to move your VMKernel to the pfSense network, and then you won't be able to have that VMKernel web-accessible. You will then need to set up VPN on the pfSense so you can VPN into the network to access the ESXi management network through the pfSense.
Some fairly important notes on my answer, of which you should take note
- I am not going to go through how to configure pfSense. This question here asked about the ESXi network setup, so I'm addressing that part of the answer. If you need help setting up pfSense, make a separate question, or refer to one of the billion tutorials on the Internet.
- The ESXi server I am using for my screenshots has nine (9) physical ethernet ports on it. Of those, only two are not in use, so they are on these images as
vmnic6. While the pictured items on my diagrams for this are not actively in use, this approach still 'works'. I've proof-of-concept tested this with both VMware Workstation (in a different but similar approach), and on my ESXi.
- My ESXi system is ESXi 6.0U2, with the Embedded Web UI turned on. I may be switching between the WebUI and vSphere Client; in my screenshots, but the basics remain the same for configuring things.
- My screenshots will NOT have the VMKernel on them. The VmKernel on my ESXi is on a separate vSwitch, with a separate uplink, on a separate subnet. My diagrams though will show the VMKernel item.
With my obligatory warnings out of the way, this is really not as difficult as it seems.
With physical equipment, and a physical pfSense appliance, you will have a WAN port and a LAN port. Your WAN port will be connected to the uplink to the world. The LAN port will be connected to some type of switch and the rest of your LAN infrastructure on the internal network. The pfSense WAN will be configured for the WAN IP address (static or dynamic), and the pfSense LAN will be configured for the LAN IP address with static settings, and you either turning on DHCP for the LAN or having every single system static IP'd with the default gateway pointing at that LAN.
With everything on the ESXi, we have to emulate this, but it's still the same concept.
ESXi and vSphere
Given you already have VMs, you already likely have a
vSwitch0, which in turn is linked to physical system uplinks, like this, but with VMs in the "VM Network" group, and with a
vmnic that actually is connected to something:
Importantly from this point forward, anything on vSwitch0 will now be considered on the "Uplink", as this has physical NICs on it. We can change this later. The pfSense WAN virtual NIC will be on this switch.
Create the vSwitch1, for the pfSense LAN
Here's where we'll create the LAN. If you've created vSwitches before, you can easily breeze through this part, but effectively, we are adding a vSwitch for VMs, not the VMKernel. The key here though is to not connect a physical adapter to this network, since we're running it through pfSense, which will be a virtual NIC connected into this vSwitch. I named the new port grouping here "pfSense LAN".
Deferred until later: Move the VMs into the pfSense LAN
Because this may be a system where you cannot have downtime, I'm going to have the VMs moving to the pfSense LAN as a later step.
Now, for pfSense. When you create the pfSense VM, you need to give it two virtual NICs - one for the pfSense "WAN", which is the 'outside' that the uplink exists on, and one for the pfSense "LAN", which is the internal network which the pfSense firewall protects and does NAT and routing to the uplink for the machines on the LAN.
When you set up the pfSense, you need to make sure you give the VM two virtual network adapters (
vNICs, from this point forward in this answer). They each need to be on the individual network groups we have, one on
vSwitch0 ("VM Network" as earlier), and one on
vSwitch1 ("pfSense LAN"). I use
E1000 type adapters - they seem to work well with the FreeBSD based pfSense system.
This is optional, but I highly recommend this. Make sure you have a VM on the "pfSense LAN" network where you can access the GUI so you can access the pfSense after you set it up. You'll need this to configure the pfSense Firewall, unless you know what you're doing on the CLI for pfSense (I will not add the pfSense Setup steps here, as that's a different question, basically.)
ONLY move to this section AFTER you've configured pfSense for the LAN and WAN settings, and set up firewall rules so you can start with an "Everything can go out to the world from the LAN" setup, and then start adding whitelist items and firewall rules before locking down.
After you've configure your pfSense, we come back to the part where you may have downtime if you're having to have access from the outside in to the VMs.
Move the VMs to the pfSense LAN
Basically, edit the network adapter settings for each of your VMs that are on the default "VM Network", and set them to be on the "pfSense LAN" network segment. If everything is static IP'd, then you should be all set, just make sure on the VMs to set the "gateway" to be the Static IP you set on the pfSense LAN adapter. (That way, the VMs can route through the pfSense).
Once you're done, and your VMs can connect to anything outside through the pfSense (by setting the gateway correctly on each machine), then you just need to configure the pfSense to permit connections inbound to the machines from the outside where necessary.
And now you're done. The pfSense should be able to reach to the 'net, and so should the VMs. If you really care about "VM Network" being present, add a group to
vSwitch0 called "pfSense WAN" and then edit the pfSense network settings to have the "VM Network" vNIC connected to the new "pfSense WAN" group.
The only thing we haven't adjusted here is where the VMKernel sits. While I am not going to do this here, because it will break my system heavily, you theoretically can add a VMKernel to
vSwitch1's properties, and give it a static IP in the "pfSense LAN" if you want the management to only be accessible form the "pfSense LAN" section of the network. Once you confirm that you can reach in through, there, you can remove the VMKernel on
vSwitch0. If everything starts breaking, though, then you'll have to have physical access to the machine to reconfigure the management network through the machine itself.
Note that if you are already on a LAN or such where the management network resides, leave the management network alone. Unless your ESXi needs to reach out to the Internet, you don't need it to go through a gateway.
Post-configuration pfSense / ESXi Network Topology Diagram
Effectively, with regards to ESXi and the topology with regards to your VMs, pfSense, and the uplink, you will end up with something like this, focusing solely on the ESXi part.