There are four types of security you need to consider when setting up a PBX:
- Asterisk Configuration
- Hacking Detection (intrusion detection)
- Fraud Detection
This type of security is usually applied at a firewall (not at the PBX). Most firewalls simply route SIP/IAX/RTP/etc packets to the PBX. So they really dont do any protecting (but they can at least block SSH/telnet/etc connections). The first (accepted) answer references on-host (PBX) rules using iptables. That is not recommended - don't let attackers past the perimeter protection. Know attackers/hackers/fraudsters should be blocked at the perimeter.
2. Asterisk Configuration
This is common sense today and most configuration generators already take care of this for you. But if you are working with Asterisk directly then use complex device names and complex secrets. Disable guest access (allowguest) and don't allow meaningful failure responses to attackers (alwaysauthreject). As well, be careful with your dial command (in Asterisk) as the wrong parameter can allow a caller to flash hook and dial any outside line. Change all passwords from default, don't let Asterisk run under root, and if you do choose to use a config generator change the default passwords there too.
3. Hacking Detection (intrusion detection)
Tools like fail2ban are trivial (in fact Digium warns users NOT to use fail2ban as a firewall/security device). That said, if you really don't have the skills to setup anything more, then fail2ban is better than nothing. What most people don't realize is that fail2ban depends completely on Asterisk to detect and reject a dial/register attempt before an IP can be banned. So if the attack does not cause these Asterisk errors then fail2ban does nothing. (As well, with fail2ban you are blocking attackers at the PBX, not at the firewall).
Now for the real hacking. How do you know if an attacker is using malformed SIP packets? In that case look into "snort" or other SIP packet analyzers. What if the attacker is rotating through an large subnet if IP's or VPN IP's? Make sure your detection tool can track that. What about block IP's based on geofencing? Advanced firewalls can do this, as can some open source firewalls like "pfsense" and proprietary Asterisk security systems like "SecAst".
Configuration generators (eg: FreePBX) have a poor security history, particularly surrounding exploits to the end-user GUI. If you choose to expose the HTTP GUI to the internet then you better install host based hacking detection such as "tripwire".
4. Fraud Detection
Serious hackers intercept packets, and even hack the phones themselves, to steal valid credentials. How do you stop hackers with valid credentials? There are some open source tools to track SIP channels which you could use to detect number of channels by source (write your own detection scripts), or commercial products like "SecAst" which track rate of dialing, number of call setups per second (since fraudsters race to exploit credentials for toll fraud before extensions/IP's/trunks are shut down).
What about detecting a suspicious number of calls to/from particular DID's, as is often the case in toll fraud. Tools like "SecAst" can track those, and even compare to phone numbers known to be used for fraud (even if the attacker keeps his total channel volume low).
Similar to #3, if your users operate in well defined geographic areas then geofence extensions even if they have valid credentials. Advanced firewalls can do this, as can some open source firewalls like "pfsense" and Cisco and proprietary Asterisk security systems like "SecAst".
So, in summary there is a LOT you can and should do to secure your PBX. I've listed a bunch of open source products above that allow you to achieve a reasonable level of security (if your skills are sufficient). I also mentioned some proprietary tools that usually handle all of the above for you, but you will have to spend some money to use them. However, after your first $400,000 phone bill you may find that cheap now means more expensive later (Google $400,000 phone bill Astricon and you will see what I mean)