Usually the culprit when you see a large proportion of out-of-order packets in a stream is badly done load-balanced routing being done by some router somewhere on the path of the packets. IP doesn't guarantee in-order delivery of packets but protocols like TCP and RTP do assume that packets will be in order most of the time and degrade badly when that assumption doesn't hold. Therefore, best practices dictate that routers should not be configured in ways that will cause lots of out or order packets.
When most routers are configure to load balance traffic across multiple links, they use a hashing algorithm (often based on source and destination IP addresses and maybe TCP/UDP ports) which ensures that packets that belong to a single stream all choose the same member of the load-balanced group. This helps ensure that the packets will stay in order. If a router does load balancing without using such a hash, and packets belonging to a single stream end up on different links, they can easily take different amounts of time to transmit (due mainly to buffering, meaning this will happen even if the load balanced links have identical characteristics) and end up out of order at the destination.
If that's what's happening to you, and it is bad enough that the RTP receiver's jitter buffer can't compensate, then I'm afraid that there's nothing you can do about it except get whoever managed the network to fix it.