If you configure a "host-only" interface, VirtualBox will use a virtual interface on the host. You can define forwarding to that interface.
To use a host-only interface, start VirtualBox, and from the "File" menu choose "Preferences". Choose "Network" in the window which appear, and press the "add" button. After adding the network, press the "settings" button on the right. There, you can edit the virtual network settings. The "IP Address" will be the host's IP address on that virtual network.
Make sure the DHCP server option is switched off.
If you're done, press the "OK" button. After this, you can select "host-only" in the network preferences of the guest computer. It will default the interface to the one you created in the above steps.
Start the virtual machine. Give it a fixed IP address, and set up the gateway. You should be able to ping the guest computer's IP address from the host and vice versa.
A new adapter will appear on the host operating system, named "vboxnet0". Through this, the guest computer will be reachable, but only from the host computer (of course). You can configure port forwarding on the host computer as usual, and with this, you can access your guest directly.