I am logging in to a remote linux box using
gnome-session --display=$DISPLAY. If I start a background process, is there a way to have it stay alive (keep running) when I exit Xming so that it completes or still shows up when I log-in again?
No; when your X session goes away all programs using it lose their connection to the X server, which typically causes them to exit. It is possible to arrange for programs that require neither X11 nor a terminal to be persistent, but if it needs either of those then it cannot survive loss of access to the Display.
If you want a persistent X session, take a look at VNC. I've used X11VNC for this but there are other ways to set up persistent sessions.