Failed SPF for email imported to Gmail because of client IP instead of server's in message when sent through SMTP from one local box to another

Question

We have a linux (Debian) VPS with domain (let's say example.com with MX mail.example.com) that has SPF set up. There is dovecot+exim running. There is also Direct Admin on top of that.

When I send a mail to foreign server then everything is fine. There is server IP in the message and SPF check goes fine.

Some data changed (domain etc.):

Received: from mail.example.com (mail.example.com. [188.40.153.39])
    by mx.google.com with ESMTPS id ***.7.2015.02.18.04.09.46
    for <*@gmail.com>
    (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
    Wed, 18 Feb 2015 04:09:47 -0800 (PST)
Received-SPF: pass (google.com: domain of [email protected] designates 188.40.153.39 as permitted sender) client-ip=188.40.153.39;

But when I send it from a local mailbox to another local mailbox and then get mails into gmail through POP3 then I have problem with SPF because a message contains original client IP address and SPF check fails.

Authentication-Results: mx.google.com;
   spf=fail (google.com: domain of [email protected] does not designate 82.160.100.10 as permitted sender) [email protected]
Received-SPF: fail (google.com: domain of [email protected] does not designate 82.160.100.10 as permitted sender) client-ip=82.160.100.10;

82.160.100.10 is IP of original sender.

Because of that problem our internal corespondence tends to be marked as spam in gmail for people who check their boxes through it.

Any ideas how to fix that?

EDIT: headers of test mail (changed our IPs and domain)

1) Email sent from one box to another - headers from Thunderbird client:

Return-path: <[email protected]>
Envelope-to: [email protected]
Delivery-date: Thu, 19 Feb 2015 11:41:20 +0100
Received: from nat10.net08-g2.isko.net.pl ([82.160.100.10] helo=[11.0.0.22])
    by mail.example.com with esmtpsa (TLSv1.2:DHE-RSA-AES128-SHA:128)
    (Exim 4.83)
    (envelope-from <[email protected]>)
    id 1YOOPC-0005Ud-Qq
    for [email protected]; Thu, 19 Feb 2015 11:41:20 +0100
Message-ID: <[email protected]>
Date: Thu, 19 Feb 2015 11:41:18 +0100
From: Head Developer <[email protected]>
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0
MIME-Version: 1.0
To: Zbyszek <[email protected]>
Subject: This is test
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit

2) Same mail after being received by gmail (by automatic regular import by POP3):

Delivered-To: *@gmail.com
Received: by 10.140.86.210 with SMTP id p76csp775880qgd;
    Thu, 19 Feb 2015 02:47:12 -0800 (PST)
X-Received: by 10.140.102.165 with SMTP id w34mr10762910qge.26.1424342832562;
    Thu, 19 Feb 2015 02:47:12 -0800 (PST)
Authentication-Results: mx.google.com;
   spf=fail (google.com: domain of [email protected] does not designate 82.160.100.10 as permitted sender) [email protected]
Received-SPF: fail (google.com: domain of [email protected] does not designate 82.160.100.10 as permitted sender) client-ip=82.160.100.10;
Received: by 10.224.31.8 with POP3 id w8mf619596qac.5;
    Thu, 19 Feb 2015 02:47:12 -0800 (PST)
X-Gmail-Fetch-Info: [email protected] 2 mail.example.com 110 [email protected]
Return-path: <[email protected]>
Envelope-to: [email protected]
Delivery-date: Thu, 19 Feb 2015 11:41:20 +0100
Received: from nat10.net08-g2.isko.net.pl ([82.160.100.10] helo=[11.0.0.22])
    by mail.example.com with esmtpsa (TLSv1.2:DHE-RSA-AES128-SHA:128)
    (Exim 4.83)
    (envelope-from <[email protected]>)
    id 1YOOXn-0005j5-Tm
    for [email protected]; Thu, 19 Feb 2015 11:41:20 +0100
Message-ID: <[email protected]>
Date: Thu, 19 Feb 2015 11:41:18 +0100
From: Head Developer <[email protected]>
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0
MIME-Version: 1.0
To: Zbyszek <[email protected]>
Subject: This is test
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit

Edit: some extra info

method of delivery is SMTP (I send email from Thunderbird in home using mailbox [email protected] to another mailbox on same server [email protected])
82.160.100.10 is my home IP, nat10.net08-g2.isko.net.pl is my home hostname which resolves to that IP
10.140.102.165 is gmail server IP
11.0.0.22 is local IP in my home network
Mail server IP wasn't in any of headers (If it would be there it would be 188.40.153.39).
Hostname mail.example.com points to mail server IP
Domain IP wasn't in any of headers (if it would be there I would change it to 85.17.23.59)
domain has proper MX entry (that points to subdomain mail.example.com)
SPF record: "v=spf1 a mx ip4:188.40.153.39 -all"

Edit: uncovered IPs as not so sensitive

When you say local mailbox do you mean a mail address on the same machine using a local delivery method using only the local part of the address without the domain? Thus the domain is delivered locally instead over SMTP? If yes, that's expected, if no, please elaborate on the problem and your setup. — sebix, Feb 18, 2015 at 14:38
Yes, target mailbox is on the same machine. I wonder if there is a method either to configure it so message would contain IP of server instead of client's IP or turn off SPF check for such messages (which is probably on gmail side when importing mail, but really I'm not 100% sure, don't know how to check it). — Zbyszek, Feb 19, 2015 at 9:47
Just for correction, GMail doesn't support fetch email via IMAP. It fetch your email via POP3. — masegaloeh, Feb 20, 2015 at 15:25

Community · Accepted Answer · 2017-04-13 12:14:10Z

15

Disclaimer: This answer was speculation one until GMail person confirmed it.

Looks like it's GMail mishandle your fetched email here. Some peoples also report similar case with yours in here, here or here

The problem is: GMail also deploying SPF measure when scanning email after fetching it via POP3.

Normally SPF-checking take place in SMTP transaction by checking the domain parts of sender address and client IP address. But in POP3, GMail has to parsing header and find the last Received header.

Received: from nat10.net08-g2.* ([*.160.100.10] helo=[11.0.0.22])
    by mail.example.com with esmtpsa (TLSv1.2:DHE-RSA-AES128-SHA:128)
    (Exim 4.83)
    (envelope-from <[email protected]>)
    id 1YOOXn-0005j5-Tm
    for [email protected]; Thu, 19 Feb 2015 11:41:20 +0100

This is original email that fetched from your server. It states that your email accepts email from *.160.100.10 with sender [email protected]. In this stage Gmail pretends as your server and checks domain parts of sender address (example.com) and client IP address (*.160.100.10). The result was expected:

SPF softfail because domain of ldev@example.com does not designate *.160.100.10 as permitted sender

For workaround, you can set Gmail filters to never mark your email as Spam.

edited Apr 13, 2017 at 12:14

CommunityBot

1

answered Feb 19, 2015 at 13:27

masegaloeh

18.2k10 gold badges57 silver badges106 bronze badges

Thank you for info. I wonder if there is any other better way than this workaround. Will accept this answer on monday if no better resolution to the problem will be found.
– Zbyszek
Feb 19, 2015 at 14:07
How do the client (*.160.100.10) send email to exim? By submission port 587? Or by smtp port 25?
– masegaloeh
Feb 19, 2015 at 14:09
On smtp port 25
– Zbyszek
Feb 19, 2015 at 14:13
2

At first, I think that you can try bypass gmail SPF checker by add something in the header indicated that the client was using authenticated SMTP session. But looks like you already have ESMTPA header in your email and Gmail still ignore it :(. Proper spam checker will consider to this kind of header when calculating spam score. For example: SpamAssasin will reduce the score of authenticated email when there is ESMTPA header in Received header.
– masegaloeh
Feb 19, 2015 at 14:41
2

7 years later - Gmail still does it (just happened to us, another server)
– Zbyszek
May 24, 2022 at 19:41

| Show 1 more comment

masegaloeh · Accepted Answer · 2015-02-27 13:41:55Z

2

It seems to be a bug that Gmail does not respect the ESMTPA in the Received header to show the MUA is a trusted host. Some possible workarounds come to mind:

use split MX, that is, one Exim instance to receive and forward from authenticated clients, and a second to receive into mailboxes. That simulates the ISP-to-ISP mail that Gmail may be expecting, and there will be an IP address in the headers that matches the SPF record.
add a Received header at the top that simulates the above transfer, by adding add_header = Received: by 10.224.31.8 with ESMTP ....
for the users checking their mail in Gmail via POP, forward a copy to their Gmail account instead.

edited Feb 27, 2015 at 13:41

masegaloeh

18.2k10 gold badges57 silver badges106 bronze badges

answered Feb 21, 2015 at 18:19

Cedric Knight

1,1087 silver badges20 bronze badges

(2) looks interesting, could you elaborate why 10.224.31.8 though?. (3) is not an option for us as there is another problem with having a plenty of forwarders from one server with one IP - Gmail tends to think that those are "unsolicited bulk mails" and starts to put a rate limits on us (hosting active newsletter of one of our clients probably isn't helping)
– Zbyszek
Feb 22, 2015 at 17:56
Sorry, I saw 10.224.31.8 and assumed it was a private address of your mailserver, but it looks like Gmail. If you have a separate public address you could use, maybe fake that. The line would have to comply with RFC 5322 and be sufficiently realistic to be interpreted as a relay by eg SpamAssassin.
– Cedric Knight
Feb 22, 2015 at 18:16
Ok, thanks, will need to try it out soon. Do you know a way how to config that only for problematic mails eg. from local boxes to other local boxes? Or do I need to do that for all incoming mail?
– Zbyszek
Feb 22, 2015 at 20:57
@Zbyszek Looks like you need to do it in every email which its sender domain has published proper SPF record. Just be safe, you should added it in every email
– masegaloeh
Feb 27, 2015 at 4:00
1

anyway instead using received_header_text like this proposed solution, why not using add_header feature from exim?
– masegaloeh
Feb 27, 2015 at 15:25

Add a comment |

Digin Dominic · Accepted Answer · 2020-01-30 11:58:07Z

1

I had the same issue and I fixed it using he following Google doc. This might help you as well and worth trying.

https://support.google.com/a/answer/33786?hl=en

You will have to include the following record added to your existing spf TXT record. This helps Google to identify the source of the email address.

include:_spf.google.com

So you have to update your spf record as follows.

SPF record: "v=spf1 a mx ip4:188.40.153.39 include:_spf.google.com -all"

Then try sending the email and it should land directly to your inbox if your IP has good reputation.

edited Jan 30, 2020 at 11:58

answered Jan 29, 2020 at 10:35

Digin Dominic

637 bronze badges

1

Worked for me! Specifically, the -all part.
– Chris Happy
Sep 30, 2022 at 17:32

Add a comment |

Stack Exchange Network

Failed SPF for email imported to Gmail because of client IP instead of server's in message when sent through SMTP from one local box to another

3 Answers 3

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged
ip
dovecot
exim
spf
gmail
.

Linked

Hot Network Questions

Failed SPF for email imported to Gmail because of client IP instead of server's in message when sent through SMTP from one local box to another

3 Answers 3

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged ipdovecoteximspfgmail.

Linked

Related

Hot Network Questions

Not the answer you're looking for? Browse other questions tagged
ip
dovecot
exim
spf
gmail
.