Spammers are catching up, and graylisting may not be as effective as it used to be. If you can, try using some RBLs, I find that they are more effective. YMMV.
AFAIK, postgrey cannot determine if the remote server's IP coresponds with the IP for the MX record of sender domain. Some other tools might be more adept to the task, take a look at postfix's add-on page for that.
As a side note, whichever technique you use, it may have a downside. RBLs might occasionally block some legitimate mail (although I find that quite rare with good RBLs and generally worth the trouble) and graylisting gives you a delay. Also, regarding your plan, sender box might not be the same machine the MX record is pointing to, so it might not be efficient if you're mostly dealing with mail from big domains. You'll need sender's address to be able to look up MX record for the domain, and, as we know, that's easily forged. Servers ocasionaly process mail for more than one domain.
On a brighter side, postgray implements auto-whitelisting of clients which repeatedly show to be able to pass the greylist. So, unless you really have to, maybe you shouldn't worry much about the delay.