The issue here is that I did not have a gunicorn.socket file and this was in the service file as Required. Adding the following file into the same folder as the service file:
Fixed part one of the ...
Just in case anyone comes here with Apache using mod_wsgi, this can be solved by adding the lang and locale options to the WSGIDaemonProcess line in your site's config file.
Should normally be set to en_US.UTF-8 according to the docs: