There will, even under normal operation, be audit failures constantly as services try legacy methods and such. However, if you see consistent remote authentication failures with random usernames or actual usernames, it's a bruteforce attack. The source port is probably not relevant.
If the server doesn't need to be listening to the internet on the service that is being attacked, it shouldn't. Firewall it. In fact, you should generally have a firewall around the server that denies access to everything except what is necessary. What specifically those things are depends very much on your environment and what the server is for (there is no One True Firewall Ruleset).
If the login attempts are on a service that you do need to be listening on, you should consider deploying an IPS of some kind to help limit the bruteforce. Keep in mind also that with strong passwords, the odds of an attack like this succeeding are comparable to the odds of winning a lottery.
If you notice that the failed attempts seem to be coming from your internal network and aren't related to someone fat-fingering a password or a service of some kind automatically trying to log in with the wrong password (a certain number of authentication failures are normal in all environments), investigate the possibility of a compromised host.