Since the mime encoding can increase a message's size by 33%, what is the correct math I need to do to filter messages larger than 45Mb? Is the math needed at all?
Does the math change among different versions of Exchange?
The Exchange size-limit is indeed the size of the actual message, not the decoded attachment size. So yes, you will have to do the math. Base64 encoding doesn't change from Exchange version to Exchange version, they've consistently been limiting on the size of the message in the mailer-queue (i.e. encoded).
That size is the full size of the message in the DATA block of the SMTP conversation. That's headers, body, and base64-encoded attachments (and in exceedingly rare cases, UUENCODED attachments! Those still work.). Since this is MIME, not simple b64, the math works out to an expansion factor of 1.37 (link). That is the 3 bytes into 4 bytes expansion (+.33), plus the terminal CRLF characters after ever 72 characters (+.04). Above and beyond this are the fixed costs of headers, actual body of the message, and the MIME headers inside the body; depending on how large the attachments are these costs may be negligible.
Simple B64 is 1.33, but MIME is 1.37.
So, for a 45MB message size limit (46080 kb), you need to set a limit of 63130 Kb.