Swap is beneficial, because it allows inactive memory blocks to be written to disk, freeing up real memory for stuff like file system caching. Swap also (to some extent) protects you against short bursts of memory demand. Linux (by default) over commits real memory, relying on the fact that most applications ask for more than they need, but if they suddenly all need that memory swap can buffer you against applications crashing.
Long term use of swap instead of real memory can kill performance, but in the short term, swap can improve performance and protect against some failure scenarios. I would run without swap in only a few very limited situations.
Without knowing how your VPS is configured, it's not easy to suggest which method to use to add swap, however, my experience of VPS's in general is that your disk space is already carved up, so I would use option 3 (which creates a regular file), it's less efficient than using a logical volume but I don't know if you can create any on your vps. Option 1 is only useful for extending existing swap which you apparently don't have.