If I understand correctly, you want to take a physical VGA port and have access to it via IP. If so, specialist hardware will be needed at the server end; that's by no means an uncommon requirement, but it tends to be offered only on multiport KVM devices.
Adderview seem to do a one-port KVM-to-IP converter called the
Adderlink IP. As far as I can tell, it plugs into the KVM ports on the server and presents the console over java and/or VNC interfaces, so you can use either a browser or a VNC client on the remote client system to access the server's physical console.
I haven't used that device myself, though I have an Adder KVM switch on my desk at home, and it's pretty good. I'm sure comparable devices will be available from other manufacturers.