I have web application running in jboss 4.0.2 and it has older java 5 version, is this new update causing possible security flaws in my web application server?
Java 5 is already EOL, this is, no more upgrades unless you have a support contract. If you are using the freely downloadable version of the jdk and not a "java for business" release, if the previous statement is true, then the probable best approach is to upgrade to java6 after upgrading jboss to a newer version. See this documentation from redhat: http://www.jboss.com/products/platforms/application/supportedconfigurations/#JEAP4.2
jboss-eap 4.2 is way newer than the 4.0.x branch and the jdk6 is not a supported configuration.
What I am trying to say is that you may run into issues, and also that the version of jboss you are running is extremely old.
You should evaluate jumping to jboss as 4.2.3 (you can read the release notes here http://sourceforge.net/project/shownotes.php?release_id=614346&group_id=22866).
Bottom line your situation is catch-22.
Just need to bite the bullet and do some testing and then try the upgrades needed.
I'm not quite sure what your problem is. Have you upgraded your server to Java 6 and are now concerned about security holes ? What evidence do you have for this (if any) ?
Here's the Java 6 adoption guide. As well as a bunch of new APIs there'll be bug fixes and (possibly) security holes closed. But I don't know of any particular issues being introduced in Java 6. If you have security holes in your app, they're likely to be there as part of the application itself, and not the underlying Java infrastructure.
You should generally always use a distribution which allows you to automatically upgrade to the latest versions of software.
E.g. under Ubuntu the sun-java6-jdk package does exactly that.
In any case, I would always upgrade to the latest version recommended by Sun.