Spam report feedback

[cesy]

Title:
Spam report feedback

Area:
spam reporting, comments

Summary:
Allow the anti-spam team some standard options for feedback, to thank users for reporting spammers, and redirect users who are reporting non-spam as spam.

Description:
When a member of the anti-spam team closes a spam report, it would be good if there was a way to send a note back to the spam report originator.

The proposed options would be along the lines of:
* Close this spam report with no feedback.
* Close this spam report with feedback of 'thank you for your report, spammer will be smacked'.
* Close this spam report with 'this did not appear to be spam'.
* Close this spam report with 'this is not the way to report a suspected ToS violation. Here is a handy link.'

People wouldn't be able to see the specific feedback on their spam reports, but would be able to see their percentages.

At over 75% instances of not-real-spam or dude-ToS-is-thattaway, someone might get an auto 7 day spamreportban.

Advantages are that helpful users who report spam could be thanked, and anyone who consistently and repeatedly reports comments they just don't like as spam wouldn't be able to waste the anti-spam team's time so easily.

Disadvantages are that people might be upset at getting either of the last two types of feedback, so they would need to be worded tactfully.

Poll #2860 Spam report feedback

Open to: Registered Users, detailed results viewable to: All, participants: 48

View Respondents

This suggestion:

View Answers

Should be implemented as-is.
35 (72.9%)

Should be implemented with changes. (please comment)
4 (8.3%)

Shouldn't be implemented.
2 (4.2%)

(I have no opinion)
7 (14.6%)

(Other: please comment)
0 (0.0%)

Comments

[the_shoshanna]

I've never reported (or gotten) spam and am not on the anti-spam team, so maybe this is obvious, but here:

People wouldn't be able to see the specific feedback on their spam reports

does "people" mean the spam reporter, or the anti-spam team, or random passers-by? I should think the detail would be useful for the first, possibly useful for the second (I don't have the experience to know), and none of the third's business; but it looks as though "people" and "their" mean the same group, so I am slightly confused. But still voting in favor, because, YES.

[cesy]

Random passers-by shouldn't see anything.

For the spam reporter, their only record of the comment would be in their email notification, since it would have been deleted when they reported it, so it wouldn't be possible to link the feedback neatly to a specific report from their point of view, I don't think.

I don't know how long old spam reports are kept, but for most of the anti-spam team, general percentages and percentages per user or IP address would be much more useful than specific links.

However, as you imply, this is probably a minor point that we could leave up to the coder who implements it.

[green_knight]

If you implement feedback, it ought to be feedback for everything - 'close without comments' leaves me as baffled as before. (Haven't had spam on DW, had a few on LJ, and I'm curious what the outcome was with no way of finding out.)

[cesy]

However, it means that the anti-spam team can still use the system in the same way they currently do, and are not forced into giving feedback, particularly if a report doesn't fall neatly into any category.

[green_knight]

'we're not sure' is feedback in itself. I just would like to avoid a situation where feedback depends on the mood of the person dealing with the spam. I'd be happier if you had a 'I don't want feedback' option.

Whoever deals with spam already needs to categorize it for internal feedback, so I see this as making that information public, not creating more work for anyone.

[cesy]

They don't need to categorise a report for internal feedback, actually. Using any of these options would be extra work, but in some cases it would be worth it, for the advantages outlined above.

I also don't really see how it depends on the mood of the person dealing with the spam - this would just be three clear and helpful standard responses, in addition to the option to continue as they would with the current system. The spam team don't have any of the same user-communication processes as the support team, and it's right that they don't - we don't want to force a change in the whole nature of the team.

[denise]

Yeah, as long as it wouldn't be required to provide feedback, I'm okay with this. I'd hate to have a setup where we'd create the expectation that the user would hear the end disposition of every spam report they make, especially because often it can seem like the site isn't taking action when they should to the user (selection bias) -- "they have something against me, they never deal with my spam" would be a bad impression to create when really what it is is that at least 50% of spam we get reported could be either one very clueless anonynmous commenter or the beginning of a spam campaign where spammers leave a comment with easily googleable text, and until/unless we get more volume from that IP/with that spam text, we can't make that determination.

Basically, we're lucky in that we get very little actual spam and what spam we do get is of the "testing the waters" variety, but that does make determining what's testing-the-waters spam and what's a clueless commenter hard, and there's always a judgement call involved. It is often hard to make that determination. Still, I'd like to be able to have the option for the antispam team to say "the last 400 spam reports you have made are not actual spam, so please cut it out" or something like that, as long as it wouldn't be a sort of system where the antispam team would be required to enter a public disposition for each spam report.

[kyrielle]

I'd almost suggest only three switches here - no comment, "not spam please" and "ToS violations thattaway please". That way, no one can form a false expectation of "I was told THIS one was spam, so no reply to the next 14 means they aren't taking me seriously!" Thanking helpful users has the drawback of potentially setting an expectation, while a standard response for non-spam reports might be useful.

[azurelunatic]

The spam system itself has had few upgrades from the original system we inherited. There is currently no structure for categorizing reports for internal feedback. This does mean that each spam report has to be categorized twice: once by the person dealing with the report at the time they deal with it, and once by the person doing a regular roundup of reports (if one is done).

I envisioned the report given to the user as being something like a page that a user could go to, to check what sorts of dispositions their reports had been getting; if a user did not want to see, they would not visit that page. (Though a subscription, for 'tell me when the numbers on my spam statistics page have changed' would be useful; I hadn't thought of that.) The user would always be able to see their total number of reported-as-spam items, both for their own journal and for any communities they administrate. They would then be able to see number and percentage of all the useful categories that get defined (uncategorized, which would cover reports that have not been dealt with, reports that were dealt with before there was categorization, and reports that had been disposed that way; spam; non-spam; non-spam with bonus ToS link). Though it strikes me (now that it's no longer 2am) that null/spam/non-spam/ToS is much more useful internally to the team than a user standpoint, and from a user feedback level, non-spam/ToS could probably be clumped, with a "not spam" score, and a helpful link on dealing with unwanted contact including how to report ToS violations, regardless of whether the spam team looked at it and gave it the hairy eyeball re: ToS. And a "the team is uncertain" disposition that is distinct from null categorization would be useful in the team; that could be flagged for someone else to look at to provide a second pair of eyeballs, and I see no reason to not share that with the user, unless it caused confusion on the user end.

At the time of the IRC conversation that spawned this suggestion, I had envisioned the user interface for the feedback being radio buttons on the spam reports page, both for individual reports, and (if there were multiple reports on the same page, currently the same page shows reports about the same (alleged/actual) spammer, but there's talk of getting reports on the same page from the same reporting user) for the whole page, if the whole page is stuff that falls into the same category.

[ninetydegrees]

Honestly, I would appreciate feedback if I made a mistake reporting something or the first few times I report something and it turns out it really is spam but getting a thank you if it's my 10th report? It doesn't seem necessary to me.

[azurelunatic]

I was envisioning less a push report, and more a page where the user could go and take a look, possibly with a subscription.

[ninetydegrees]

Well, I would prefer an e-mail.

[azurelunatic]

I would think a subscription would include an email notification; you would want the data in the notification itself? That sounds like a very good idea.

I was thinking a page because it's the sort of thing that I might want to take a look at out of random curiosity at any moment, rather than solely a notification that I might delete by accident.

[ninetydegrees]

Hmm. If this was somehow tied to Support requests I've opened and I could manage notifications for these as well then, yes, I would find it interesting.

[merlinpendragon]

This would quickly become too much of a burden to the anti-spam team, and I'm not on it! Just imagine having to send out a comment to all 257 people who reported ONE spam message on a group... Now multiply that by the number of groups... Or replace the number of groups with the number of DW members who have over 100 friends....

[cesy]

That's why we'd keep the "close without feedback" option, which would do the same thing as the anti-spam team currently do.

[green_knight]

If it's the same comment, does it get handled as seperate incidents? Because if that is so, we need a system that logs comment xxx on post yyy was flagged as spam by users [a,b,c] - so it will get looked at only once (maybe with a higher priority the more people report it?) and the feedback gets sent to everybody on the list.

[cesy]

No, each comment can only be flagged as spam once, when it is deleted by the journal owner or community maintainer.

[yvi]

Only journal owners and community admins can mark things as spam - and only when deleting them, so you can only have one report per comment.

[azurelunatic]

It may help to know that this suggestion was made by an anti-spam team member, as the result of a group anti-spam discussion.

[yvi]

I really like the idea (a bit like the stock answers in LJ's support system!) and I'd leave it up to the antispam team to decide whether this would be too much work for them. In the long run, this would probably reduce their workload.

[pauamma]

I'd like some kind of decaying exponential average thingy for the incorrectly-reported ratio, or a cutoff/reset at the same time the user gets sysbanned, so that good-faith misreporters don't get penalized once they level off the learning curve.

[azurelunatic]

I like this too.

[mskala]

If the spam-handler is going to take the time to recognize that something is a ToS violation instead of spam, then they should forward it to the ToS-handler directly instead of sending the person reporting a note telling them to report it as ToS violation instead. Forwarding would be no more trouble for the spam-handler; and not forwarding tells the person reporting "We are deliberately inconveniencing you," which is never a message Dreamwidth should send one of its members.

[mskala]

Come to think of it, spam is a ToS violation. Why is it handled separately from all other ToS violations, and why should it be the job of the person reporting to categorize a given ToS violation as spam or other?

I suspect the reason is because spam has a different profile from other ToS violations: it tends to happen in bursts, it tends not to be targeted at specific recipients, it tends to be easier to recognize, and maybe the people who handle spam don't need the same administrative powers and dispute-resolution skills that the people who handle other ToS violations need. But in practice what that's going to mean is that spam, being more clear-cut, gets handled faster and more punitively than other ToS violations. So if someone posts a comment I don't like, am I going to report it in the way that gets the asshole banned fast, or am I going to report it in the way that gets me a note pointing me at an FAQ entry telling me I have to delete the comment from my journal myself and just suck up the fact that they'll continue trashing me everywhere else on the site because Dreamwidth supports freedom of speech? Hmm. Tough question.

The current system creates a strong incentive for false categorization. Rather than collecting categorization information from submitters and then complaining that it's bad information, why not just stop collecting that information? It's not valuable information. Dreamwidth's people are going to have to make the categorization decision over again anyway when deciding whether to accept or reject the report.

Improving the quality of spam/non-spam categorization from 10% to 25% to 80% wouldn't make a difference; it would only really help if you could improve it to the point where you could actually trust submitters and stop checking it yourselves. That's not happening until you can eliminate the incentive to miscategorize, by guaranteeing resolution of all non-spam ToS-violation reports promptly to the submitter's satisfaction even when the submitter is wrong. Good luck.

[azurelunatic]

Also, the vast majority of spam on Dreamwidth is anonymous spam, which requires action against IPs/blocks of IPs, rather than action against a user account.

If something is very clearly an attempt from a user to get the spam team to act punitively against something that is non-spam ... that's not something the anti-spam team appreciates.

Reporting something that isn't actual spam results, in practice, in the spam report being closed without action taken: if it's a user, the user is not banned; if it's an anonymous comment, that IP address is judged based any other deleted-as-spam comments emitting from it.

An egregiously high volume of bad spam reports from any given user leads to antispam personnel complaining to the owners, and polite notes of clarification from the owners sent to the user, and to temporary banning from the ability to file spam reports.

[azurelunatic]

To date, I've seen only perhaps one non-spam item coming in through spamreports where I was moved to contact ToS. I did, so they could keep their eyes on the situation, although I believe that the results were very much anti-climactic.

All the rest of the non-spam items have been of the sort where ToS would most likely give as detailed as necessary instructions on deleting the comment and banning the user.