I would start with the Center for Internet Security's "benchmark" checklists. These are consensus based checklists compiled by security professionals for a variety of platforms and software packages. Some tools mentioned by the checklists, or otherwise commonly recommended that will aid in your hunt for security issues:
- Nessus/OpenVAS (vulnerability scanner)
- Nmap (port scanner)
- TCPdump/Wireshark (libpcap packet capture)
- SNORT (intrusion detection system)
(tcpdump is installed on many linux systems by default, or may be easily installed from a package repository, and has a comprehensive man page)
If this is for the company you work for, make sure that security analysis is authorized by management, and that the scans aren't going to cause any outage or application silliness. Yes a simple portscan can cause problems - portscan older HP Laserjet printers and they'll spit out heaps of paper.