I found that adding
RewriteRule .* - [E=HTTP_IF_MODIFIED_SINCE:%{HTTP:If-Modified-Since}]
RewriteRule .* - [E=HTTP_IF_NONE_MATCH:%{HTTP:If-None-Match}]
To the bottom of my htaccess file (below all rewriterule) solved my issue (HTTP_IF_MODIFIED_SINCE was not being set) with caching dynamic generated images. Brilliant.
Although what does this actually do and why it is needed? Is it necessary I have this in every .htaccess or can I change the apache server config so I dont have to add this to every website project?