Implement SPF records for email

[trixtah]

Title:
Implement SPF records for email

Area:
Administration

Summary:
Implment DNS SPF records to facilitate email delivery to large webmail providers

Description:
SPF is an industry standard way of guaranteeing which email servers are permitted to send mail on behalf of your domain. At present, there seems to be a perennial problem with Dreamwidth bulk email being rejected from time-to-time - the "big four" email providers (Gmail, Hotmail, Yahoo and AOL) -do- use SPF records to positively weight email spam scores in favour of bulk emailers.

Dreamwith.org sends mail from one server - it is simple to implement a DNS TXT record that reads "v=spf1 mx ~all" that will verify to any email receiver that checks SPF that your MX server is permitted to send mail on behalf of "@dreamwidth.org" senders.

It also makes the likelihood of future spammers spoofing dreamwith.org addresses in order to send mail much less.

SenderID is also a useful solution, but SPF is simple to implement and will assist with delivery of bulk email to most large email service providers.

Poll #3195 Implement SPF records for email

Open to: Registered Users, detailed results viewable to: All, participants: 45

View Respondents

This suggestion:

View Answers

Should be implemented as-is.
27 (60.0%)

Should be implemented with changes. (please comment)
0 (0.0%)

Shouldn't be implemented.
0 (0.0%)

(I have no opinion)
17 (37.8%)

(Other: please comment)
1 (2.2%)

Comments

[zarhooie]

I am in favor of anything that makes emails work.

[mstevens]

DKIM might also be worth a look if dreamwidth isn't doing it (yet).

Especially Yahoo apparently like this.

[trixtah]

Yes, when I mentioned SenderID, I was actually meaning to describe DKIM, duh me. DKIM is a little trickier to implement than SPF, though, and requires more infrastructure to do, but I thought adding a simple DNS entry would allow SPF to be the logical first cab off the rank.

I need to look at DKIM again for my organisation, now that I have a mail scanner that purports to implement it (inbound and outbound).

[pauamma]

*nod* IIRC, the prevailing opinion on postfix-users@ last time the topic came around is that DKIM is better overall. (And Google is also a strong believer in it, IIRC.)

[pauamma]

Do we know what triggers the "discard email from this site" reaction?
1- email sent from Dreamwidth normally (notification emails for comments and other small-scale events)
2- email sent from Dreamwidth at a high rate (notifications for newsposts)
3- email sent *through* Dreamwidth (sent to an @dreamwidth.org alias)
4- Spoofed HELO or MAIL FROM domains or addresses (joe jobs and similar)

SPF will only help if the cause is 4.

[trixtah]

Not true.

For 2 especially, SPF can help with the big mail hosts. Some domains will check sender rates and then up their spam scores if there is a big mailout from a specific host in a short period of time - it looks like speculative spammer bots.

However, if you have SPF, many of those same hosts will not negatively weight the spam score too much, because at least you have the record (you've made the effort), and the mail is originating from the approved host(s), so therefore it's less likely that those 20 msgs/sec from that host are a bot.

Hotmail: http://postmaster.live.com/Guidelines.aspx

AOL: http://postmaster.aol.com/spf/details.html

Gmail: http://mail.google.com/support/bin/answer.py?hl=en&answer=81126

Etc etc etc etc

I can't be bothered looking for it right now, but Yahoo prefers DKIM (all the big providers will check it) but will fall back to SPF and positively weight the spam score as well, if present.

[sophie]

As long as it doesn't cause providers to *negatively* weight mails sent from other servers, I like the idea.

However, if it would cause providers to negatively weight emails that weren't sent from DW's servers, I'd say no, because lots of people might send email from their @dreamwidth.org address via their own mail servers if they don't want to give out their real email address, and it would be bad for them to be marked as spam simply because it didn't come from Dreamwidth's own mail servers.

[trixtah]

Unfortunately, that's the point of SPF. If it "fails" the check, then your mail is generally adversely scored. There is a way of creating an SPF record to say "these are the approved servers, but they are not the only ones that might relay mail from this domain", but I personally think that defeats the purpose of SPF. But perhaps for the big mailers, what is known as a "soft fail" won't make much of a difference.

Regarding the question of allowing people to have a sender address of a particular domain, and yet be allowed to send mail from any random machine, as an email administrator, and in this day of endemic spam (at least 90% of the email sent around the world is spam), I have a strong aversion to the idea of any organisation allowing domain senders to use random servers which they do not control. I can't think of any spammers who use their own domains to send messages "from". By allowing your namespace to be potentially polluted (any sender address can be spoofed, but it's less likely that a target that doesn't seem so "easy" will be used - there's a lot less spam purporting to come from Hotmail senders these days), you open the door to your email reputation scores to get progressively eroded.

If you don't want to use your real email address for something, then either send via the service provider itself, or use a service like mailinator.com that is set up specifically for throwaway addresses.

If the DW admins think there is any utility whatsoever in allowing their namespace to be used with random relay servers, then they can get around the SPF issue by either giving a "soft" record (as I mentioned earlier, with its drawbacks), or create a different subdomain for either the notifications or the user email aliases. If you'd go down that path, then creating a subdomain for the user aliases would seem to me to be a better way to go, so as to at least keep the "@dreamwidth.org" sender reputation reasonably intact - in any case, for the alias namespace, you can either leave off the SPF record in its entirety, or give it a "soft" record.

But if someone sent mail purporting to be from @dreamwidth.org, and from a home IP address, my organisation would reject it anyway. We don't accept mail from home networks, and while I enforce a stringent ruleset in that way, it is not rare.

[sophie]

But if someone sent mail purporting to be from @dreamwidth.org, and from a home IP address, my organisation would reject it anyway. We don't accept mail from home networks, and while I enforce a stringent ruleset in that way, it is not rare.

I hope you mean you would block mail where the first mailserver was on a home IP address. Plenty of people send mail from a home IP address, but it's incredibly rare that any good mail would be sent where the mailserver itself was on a home IP address - instead, those mails would be mostly sent through their ISP's mailserver.

I agree with blocking in the mailserver/home IP case, but not blocking everyone who just happens to send their mail from home. How on earth would you check for that, anyway?

[trixtah]

Yes, that's precisely what I mean. Yes, of course almost home email has originated from a home/dynamic IP, but I do want the last-hop before my mail server to look like a real mail server.

There are blacklist providers who keep specific lists of home or dynamically-allocated IP networks that the big ISPs use which you can subscribe to, but I simply block everything with a certain kind of name from Comcast.com, RR.com and with something that looks like an IP address in the name (such as 123-456-789.crappy.isp.com). Some antispam engines allow you to block anything that's ever traversed a known-spamming network, but I'm not quite that stringent.

If places like Comcast in the US forced people to logon to their mail servers every time they sent a message (obviously you can set up the mail client to logon for you), the amount of spam that tries to come into my organisation (in Australia) would instantly drop by about 15%

My check is pretty crude - it's not the kind of thing I'd do if I were an actual email service provider - but I get rid of 70% of inbound message traffic just from that check, and have a list of less than a dozen "false positives" over the last 5 years. When you're talking getting rid of several thousand messages a day, and only having 10 or so false-positives out of literally millions of connections, I think the blunt tool can be pretty useful (if it's well-maintained).