You're asking what you can delete to suit your needs for a smaller server. But the way you're working it sounds like you may have a misunderstanding of the distro; Ubuntu is based on Debian, but it isn't Debian with custom packages making it Ubuntu. You can't just cut things out like the way it sounds like you want.
Plus, even if there were a script or packager, you're saying things like you don't need FTP since having discovered SSH. Well, if you were stripping a server to the "core", you may not necessarily need SSH, so automated tools might just strip that out too.
Essentially you need to intervene and figure out what you need and don't need and remove packages you don't use.
I don't know what the conditions are for your VPS, but my first reaction when people are looking to strip out a server to a minimal footprint is to tell them to look for distros that are geared for minimal footprints. Distrowatch may help you there. Ubuntu server is a general purpose OS that is not aimed at being small. If you're looking for a specialized server distro you can find them, but you'll lose some functionality that you may really be used to like the apt-get system or some other tool you've come to rely on.
Not to mention that if you just go through stripping parts out, you may end up hitting a package that you think you don't need but actually do because of some thrice-removed dependency.
End answer, I know of no such easy-automated-stripping tool for a server configuration other than the minimal configuration at install time, and I'm wary of anything that does this because it may not know what I want in the server and may remove things I want. I instead would look for a specialized distro that is geared for the server role I want, or I would use the apt-get tools to see what's installed and remove things that I don't want (since the packager may minimize my whoops-moments).