In OSX server, the firewall is usually opened per Applicaation, not per port. However, I have a ton of self-compiled servers, and I find it more convenient to open their ports, rather than select their binaries from somewhere. (It's often cumbersome to spot the "binary" in a server, between millions of scripts and binaries.)
How do I open individual ports on the OSX Server Firewall?
Which version of OS X Server? Lion?
Are you sure it's "Server" & not just an OS X client you're using as a server?
If it is indeed OS X Server, make sure you're using Server Admin to manage the Firewall, not System Preferences -> Security & Privacy. The latter controls the OS X Application Firewall, which it sounds like you were using. The former controls the lower level BSD PF firewall, which is much better & actually acclaimed by most as the best firewall out there. You can also use pfctl from the command line to manage the PF firewall if you want.
Edit
Sorry, some of that info above is wrong. Apparently Server Admin still only manages ipfw, which is an older BSD firewall. Still good, but not as good as PF.
