For the most part your mails will get delivered even if you run your mail system as IPv4-only. There are still lots of IPv4-only mail servers and very few IPv6-only. There are however some caveats to pay attention to.
One risk by running your mail on IPv4-only is that you may unknowingly have caused your SPF records to be invalid. Take a look at look at this live example of SPF validation with an address on a misconfigured domain (with a domain taken from the reject log on my mail server):
>>> import spf
>>> spf.query('192.0.2.1', 'firstname.lastname@example.org', 'bankdata.dk').check()
('fail', 550, 'SPF fail - not authorized')
>>> spf.query('2001:db8::1', 'email@example.com', 'bankdata.dk').check()
('permerror', 550, 'SPF Permanent Error: Void lookup limit of 2 exceeded')
As you can see if a spammer using IPv4 tries to forge an email from that domain the SPF validation will tell the receiver that the source IP is not authorized to send mail from that domain and the mail will likely be rejected or send to the spam folder because of that.
If a spammer using IPv6 tries to forge an email from that domain the SPF validation fails and the receiving server may not understand what that failure means and might proceed as if your domain had no SPF record in the first place.
If you are using SPF records as a way to prevent spammers from forging mails from your domain, this is a misconfiguration to watch out for. You can still avoid this problem by not using any of the SPF features which rely on