You don't need VNC or even to run X via a service, though X and whatever gnome-visual stuff will get installed along with your visual program. I use aptitude to install to keep an eye on the dependencies.
I use graphical apps via
ssh from home all the time without doing anything special, just use
ssh -X as you would
ssh before. Once logged in, you background the task, example:
$ midori &
and then the window eventually appears just as if it were your local machine.
I do it mostly for getting a browser inside the network to manage something. The last time, I previously used firefox, but my new Debian install is without anything.
One tool I did really like and enjoy was opening xfce-panel on the server from my linux desktop, in which I had a few different useful plugins, including hardware monitoring. Sometimes gedit would come in handy when I knew there'd be many edits and it seemed faster to start a new shell with -X and then background the GUI rather than just use vi.
Edit to add: You reminded me I needed an internal browser, so from home I just installed a bunch of browsers on my work's host &
midori is the best I have tried yet. Netsurf couldn't open webmin, arora was way too slow, & epiphany was slower than normal to show GUI input.