While installing package I get some additional packages on dependency. But when I remove installed package with yum, the dependency packages are not removed, though I don't need them anymore.
How can I remove them automatically?
package-cleanup in yum-utils will give you a list of packages that are not longer used. It cannot automatically remove them though it would be simple to use the output in a script that did this.
You could also look into a yum plugin called remove-with-leaves. I have never used it so I'm not sure of its compatibility with the version of yum in CentOS 5. Also, people seem to have a lot of problems given the results of a quick google search.