Deal intelligently with bouncing email addresses

[pauamma]

Title:
Deal intelligently with bouncing email addresses

Area:
email notifications, email address validation, not spamming users

Summary:
When an email sent to a user is rejected or bounces, do something intelligent instead of just ignoring it.

Description:
Currently, when emails Dreamwidth sends to one of its users start bouncing (being rejected, or undeliverable because that email address no longer exists), the Dreamwidth email servers just ignore this, instead of using it as a hint not to deliver more emails to that email address. This behavior was inherited from LiveJournal, and has caused in the past much distress to users, ops people, support volunteers, and email providers, and occasionally drastic reactions by email providers, such as refusing or silently discarding all emails from LiveJournal, thus making the problem worse. LiveJournal recently committed a code change (http://community.livejournal.com/changelog/8801455.html) that changes that, but its approach may no be the best (http://bugs.dwscoalition.org/show_bug.cgi?id=2759). (Specifically, it makes that email address unconfirmed for all user accounts associated with it - http://www.dreamwidth.org/support/faqbrowse?faqid=3.)

Other, less drastic ways to deal with delivery problems include notifying the user via their inbox instead of disabling delivery, only acting on the specific account for which the email was sent, only acting if more than a threshold number of emails bounced, or only stopping emails to that address temporarily, instead of until the user takes action. (Or of course, keeping the current approach of ignoring all such reports.)

Poll #3747 Deal intelligently with bouncing email addresses

Open to: Registered Users, detailed results viewable to: All, participants: 41

View Respondents

This suggestion:

View Answers

Should be implemented as-is.
31 (75.6%)

Should be implemented with changes. (please comment)
7 (17.1%)

Shouldn't be implemented.
0 (0.0%)

(I have no opinion)
2 (4.9%)

(Other: please comment)
1 (2.4%)

Comments

[matgb]

I ticked 'as is' because I don't have a change to suggest, even though you're not actually proposing a solution. Ignoring it is definitely a bad idea, especially given the irregular havoc LJs failed notifications have caused.

[ct]

+1

[aedifica]

+1

[pauamma]

I didn't suggest a single solution, because I'm not sure the solution LJ chose is the best for DW. But on second thought, my preference would go to something that includes "inbox notification that email deliveries were suspended due to bouncing, together with a complete bounce message/SMTP session/whatever, for diagnostic and because email provider support near-invariably asks for it."

[matgb]

The main issue is of course not everyone reads their inbox, ever. Compounded by current inability (AFAIK) to stop stuff hitting it and instead just getting emailed, so it can get cluttered quickly, especially with a lot of subscriptions.

But that's a usability issue that I know's slowly being worked through, and frankyl if someone's email address is faulty then they ought to check their inbox when notifications fail. Perhaps a different category in the inbox for DW announcements and this sort of thing?

[thorfinn]

Yep, if email is bouncing, I would strongly suggest an inbox notification that includes at least the first few bounces. I don't agree with automatically switching the notifications *to* inbox notifications, but it should definitely have a notification of "your mail is bouncing".

If in the future DW has a DW Jabberbot, it can and should notify people if their mail is bouncing also.

[cesy]

I'd vote for doing all of your less drastic ways at once, so it's lowest impact to users who've just slipped over their mailbox limit, but still protects Dreamwidth.

[susanreads]

+1

[aithine]

I think disabling the email (stop sending emails from Dreamwidth to that email and the user needs to reconfirm that address) and also contacting the person via their inbox to let them know they're having email issues would work. Probably needs a threshold, though--after three bounces or whatever, so that it's not immediate when it's just the person's email server having issues that day. :)

[thorfinn]

Investigate the methods that Mailman uses to handle mail bounces. Copy those. They are excellent.

[pauamma]

You mean the ones listed in http://www.gnu.org/software/mailman/mailman-admin/node25.html ?

[thorfinn]

Bingo. :-)

[azurelunatic]

Actually, I do have a few ideas.

1) When a bounce is detected, put the address on automatic timeout -- queue all emails to be sent to that address (for all associated accounts), for a system-determined amount of time. This prevents loss. Retry later.
2) Notify via inbox and to all other previously-validated addresses (if still on account) that the validated address on the account is bouncing (without specifying the address in case of compromise on the old addresses).
3) Test with a single email to the bouncing address before sending the queued mail.
4) If it keeps bouncing after a certain amount of time and number of trials, declare that address Stopped Until User Action.
5) Create the ability for a user to create a fallback address, to which email will be sent as normal, in case of primary email bouncing. (This is different than sending a notification to a previously validated email address, in that a previously validated address not specified for fallback would not get the other regular notifs).

[kyrielle]

+1

[noracharles]

+1

[matgb]

I like these I think.

And that does remind me to go make sure my old email addresses are deleted from LJ, don't own that domain anymore...

+1.5 -3.5

[thorfinn]

1) Mail Servers can return hard-bounce or soft-bounce.

If the former, you are supposed to assume the mailbox no longer exists and that mail cannot be delivered. If you keep retrying, you are likely to get your mail server blocked as a spam originator.

If the latter, the transmitting MTA is supposed to retry later in any case.

2) That'll be very spammy, and what do you do with bounces from those notifications? Treat them as bounces? Problematic.

3) "Test" email is exceedingly annoying. If you are going to retry, you might as well retry with the email that's in the queue. And in fact, your MTA will retry it for you, if it is allowed to (i.e. soft-bounce).

4) Yes. See the link above to how mailman does it.

5) This one is maybe useful - it's opt in. However, it also needs to be verified and regularly tested itself somehow... Monthly email, maybe. If done that way, please make it rolling somehow, don't copy Mailman's 1st of the month "here's all your subscriptions" method - it's bad for MTAs.