Is there any reason why I would want to have
iptables -A INPUT -j REJECT
iptables -A INPUT -j DROP
As a general rule, use REJECT when you want the other end to know the port is unreachable' use DROP for connections to hosts you don't want people to see.
Usually, all rules for connections inside your LAN should use REJECT. For the Internet, With the exception of ident on certain servers, connections from the Internet are usually DROPPED.
Using DROP makes the connection appear to be to an unoccupied IP address. Scanners may choose not to continue scanning addresses which appear unoccupied. Given that NAT can be used to redirect a connection on the firewall, the existence of a well known service does not necessarily indicate the existence of a server on an address.
Ident should be passed or rejected on any address providing SMTP service. However, use of Ident look-ups by SMTP serves has fallen out of use. There are chat protocols which also rely on a working ident service.
The difference is that the REJECT target sends a reject response to the source, while the DROP target sends nothing.
This can be useful e.g. for the ident service. If you use REJECT then the clients doesn't need to wait for timeout.
More about this: http://www.linuxtopia.org/Linux_Firewall_iptables/x4550.html
Usually, you want to ignore probes from attackers to certain ports, by which I mean you do not want to send back 'connection refused'. 'Connection refused' means: 'there is a server here', and possibly gives away more information, whereas dropping a packet doesn't give away clues about software versions, possible vulnerabilities or even the fact that a server is listening at you IP.
The above is one of the main reasons to use DROP instead of REJECT.