What could be the general reason for
UDP packet loss
Congestion (too many packets) with lack of QOS (random packets dropped, VOIP not handled with priority) and / or faulty equipment (line quality etc.)
For the first, get QOS capable equipment, for the later check the lines (hardware, switches, whatever) for being bad.
Note that you need for an internet conenction QOS routers on both ends - which you wont have (unless voip is offered by your provide, then he likely has the infrastructurei n place). That said, as your down channel typically is a lot bigger than the up channel, a local router prioritizing only the out channel is normally "good enough", as: it already brings hugh improovements.
Bad line quality is a general hard to handle problems, though.