This is an extremely generic overview:
CPIO does a better job of duplication a file system, including taking backups. It preserves things like hardlinks, fifos, and other not-a-standard-file features. Most implementations of CPIO do everything TAR does, including reading and writing .tar files. CPIO usually takes a list of files from standard input to archive; this makes it very easy to pipe a list from something else (like
CPIO passthrough is very useful if you have a very long list of files you want to copy from directory A to directory B. (For example, you could use
find to locate all files that have changed in the last 2 years on your system)
TAR does a better job of simply dumping all your standard files to/from a tape (or archive file). It's a bit simpler to use (for most common tasks). It meets most people's simple backup demands easily; and most of it's popularity is from this fact.
And now for the fine print. There's several different versions and implementations of both CPIO and TAR. Each one has different features and some have different command line options. There are things that each can do where the other can not; if you find yourself limited by one, try the other. Everyone has a favorite, and 99% of the time either will accomplish the task.