A System For Handling 'Impostor' Complaints

A System For Handling 'Impostor' Complaints 165

Posted by ScuttleMonkey on Monday May 18, 2009 @01:01PM from the just-let-everyone-fight-to-the-death dept.

Frequent Slashdot contributor Bennett Haselton writes "A woman sued Yahoo because they wouldn't remove a page created by her ex-boyfriend pretending to be her and soliciting strangers for sex. What would be an effective system for large companies like Yahoo to handle 'impostor' complaints, without getting bogged down by phony complaints and unrelated disputes? This is a harder problem than it seems because of the several possible cases that have to be considered. One possible solution is given here." Read on for Bennett's analysis.

When I first heard that Yahoo had been sued because they refused to remove a page created by the ex-boyfriend of a woman named Cecilia Barnes to impersonate her -- portraying her as a slut looking for sex with strangers (who obliged by hounding her office with phone calls and e-mails) -- I thought Yahoo's conduct was indefensible. Even though, as the court ruled, they may have been exempt from liability under the Communication Decency Act of 1996, what possible excuse could Yahoo have had for the way they handled the situation, exposing Barnes to months of harassment, when it would have taken them only seconds to review the page, see that it was obviously causing harm, and remove it?

Then I thought more about the consequences of the rule that I was implicitly advocating by making that argument. Obviously, if an ISP has a policy of removing a user's page if some third party merely complains that the page is impersonating them, then one of your enemies could get your page removed by filing a complaint saying that they were really "you", and that your page was impersonating them. But if the ISP has a policy of not acting on such complaints, then someone could create a user account pretending to be you, and you wouldn't be able to get it removed.

In both cases, there are two problems. One is the fact that the ISP has to have a way to figure out who is telling the truth. The second is that the solution has to scale well, even for a company like Yahoo that probably gets so many complaints about user conduct every day that it would be impossible to read them all. It should be possible for genuine complaints about impostors, to reach the attention of the right people and get an account closed, without accounts being shut down because of (a) people who file complaints about 'rude behavior' that get unintentionally mixed in with 'impostor' complaints by someone who is too overworked to read them all very carefully; or (b) people who file outright false complaints that a given account is an 'impostor', just to get it shut down; or (c) people who are really sneaky, and file complaints about things like rude behavior, but who craft the complaints in a way that is deliberately designed to get them mixed in with the 'impostor' reports, in order to get the account shut down (this way, if the complainer ever sued or otherwise confronted about the complaint that they filed, they can say that they "didn't lie"!).

It's hard to think of a solution that covers all of these bases. For example, John Morris of the Center for Democracy and Technology explained how many ISPs use faxed driver's licenses to decide impersonation complaints:

In many cases involving real people, the challenged site (whether it is a legit site or a bogus site) contains one or more photographs of the person involved. What service providers do in this case is to get the person to submit a copy of their driver's license, and the provider decides whether the person submitting the license is the same person depicted in the photos. And if so, that person is the one who can control whether the site stays up or not. This works in lots of cases (because pictures are often, but certainly not always, involved).

The problem is that even this could be abused when used against a company like Yahoo that handles an extremely high volume of complaints. Suppose that Yahoo publishes a standard procedure for submitting complaints about impersonation, that includes the requirement of a faxed driver's license. Abusers of the system would figure this out, and they could start filing "complaints" against users and websites by faxing in complaint letters along with a copy of their driver's license, where the letters were not complaints about impersonation at all, but just bogus complaints about other things like "This guy was mean to me". Because the driver's license accompanying the letter is real and the statements in the letter are true (or at least a matter of opinion), the complainer can't be accused of lying or forging government documents. And if anyone ever challenged them and asked, "Why did you send your driver's license with the complaint letter? Weren't you trying to trick the ISP into thinking that this was an impersonation complaint so they would take it seriously?", the complainer could play dumb and say, "Well, I heard that if you file a complaint against someone, you're supposed to fax your driver's license with it." But if Yahoo is still getting too many messages to sort through them carefully, some of these crank complaints could still get users' accounts shut down.

So now you have an interesting, non-trivial problem. Before reading further, it's worth thinking about how you would solve this. What's a good policy that would honor legitimate complaints, without giving cranks a way to get their enemies' pages shut down for no reason, and that would scale well for large companies like Yahoo? There are really two questions here: (1) What would you do if you were drafting an ISP policy and trying to balance the interests of all parties? and (2) What would you do if you were drafting a law requiring ISPs to implement certain policies, also while balancing the interests of all parties? (The best solution may be no law at all, but I think you would have to argue that position, rather than taking the default libertarian stance and simply assuming that. After all, the "no law" status quo didn't do much good for people like Cecilia Barnes who had a legitimate grievance and couldn't get anybody to listen.)

The non-verifiability of complaints is the same problem that I've posed to hard-core anti-spam advocates who have said that ISPs should have a zero-tolerance policy towards spam and cancel any account that is generating spam complaints. The problem with that is that unless the ISP has logs of all mail sent out by a customer (and if the customer is leasing a dedicated server, this would usually not be the case), the ISP can't tell for sure if a spam complaint is real or not. If they adopt a policy of removing a site in response to a complaint (or three or ten complaints), then someone could easily get one of their enemies' sites shut down by filing phony spam complaints sent from multiple Hotmail or Gmail accounts. (You would have to forge some e-mail headers to make it look convincingly like the spam came from the site in question, but this is not very difficult.) If the hosting company has a policy of kicking customers off in response to some threshold number of spam complaints, then a dedicated adversary could just file that many complaints until the customer was terminated. On the other hand, if the hosting company won't kick off customers for any number of spam complaints, then they have no deterrent against their customers spamming. (This is mostly an academic question, because I tried filing complaints against all the dozens of spammers who spammed me in a given one-day period a few years ago, and none of the hosting companies terminated any of the sites I complained about. I wouldn't have expected any of them to terminate a customer based on one complaint, but I assume that some of the hosting companies were getting spam complaints about those customers from other people as well.)

The big difference between spam incidents and impersonation incidents, is that while there may be no reliable record of whether a piece of mail was sent in the past or not, the fact of whether the Yahoo user "bennetthaselton" really is Bennett Haselton is something that can be determined with evidence that still exists in the present day. Some kinds of evidence are more readily available than others. If I were drafting an internal policy for an ISP on when to remove pages in response to an impersonation complaint, I would take care of the low-hanging-fruit cases first:

If the page directs people to contact the page owner at an e-mail address or phone number (as the page created by Barnes' ex-boyfriend did), and you e-mail the address or call the number and someone answers by saying, "No, I didn't create that page, it's a fake", then you don't need to do any checking of the real-world identities of the parties involved -- all you need to know is that the page purports to be created by the owner of that phone number, but it isn't, so it's a fake and should be removed. This would take care of the most vicious cases of goading visitors into harassing someone directly.
(Although I'd make clear in the policy that this wouldn't apply to consumer pages about companies, telling visitors to call such-and-such a company to complain about their conduct. Encouraging people to air their grievances is legitimate as long as the page owner isn't claiming to actually represent the company. I'm ducking the question of whether this should apply to pages about individuals -- if I make a page saying, "My ex is a skank, call her at this number for a 'good time'," am I infringing on her rights? But since I'm not claiming to be her, the situation wouldn't be covered by a policy about impersonation pages.)
If the page is created by a paid user, then you can check if the real name on file with their credit card information, matches the name on the site. If it doesn't, that doesn't necessarily mean the page is a fake (possibly one person paid for the account while another one created the content), but if it does match, the page owner is probably not guilty of impersonating anyone. (Here I'm ducking the question of what to do if someone shares their name with a celebrity -- for example, if your name really is Julia Roberts and you create a page saying "Hi, I'm Julia Roberts", that's probably not enough to count as impersonation. But what if you talk about your interest in film and your exploits as an actress in local community theater, how much are you allowed to let people think that you might be "the Julia Roberts?)
If the page violates the hosting company's Terms of Service in other ways, then it can be removed without determining whether the page owner is guilty of impersonation or not. The Yahoo Terms of Service doesn't actually mention sexual content (they used to allow users to post "adult profiles" in their Yahoo Profiles accounts as long as the profile owner flagged them as such), but the document prohibits content that is "vulgar" or "...otherwise objectionable". I haven't seen the page created by Barnes's ex-boyfriend soliciting strangers for sex, but it probably violated the Terms of Service in itself.

And there may be other low-hanging-fruit options that I'm not thinking of. But what if there is no easy call, because none of these simplifying factors apply? A user creates a profile on a free site claiming to be Mr. X. A third party complains that they are the real Mr. X and that the profile is fake. What should the ISP do, if they don't want to spend money verifying the real-world identities of the parties involved, every time they get a crank complaint about any users on their system?

This is essentially an economics problem. Cecilia Barnes wasn't asking Yahoo to do anything that would have been too burdensome for them -- the "labor" required to look at a faxed copy of her driver's license probably wouldn't have cost more than $5, at which point Yahoo could have initiated the process of shutting the page down, which they already have built-in procedures for. The benefit to her of getting the page shut down could have been valued in the hundreds or thousands of dollars. Normally, when you need someone else to do something that costs them $5 worth of effort and brings you $1,000 worth of benefit, the natural arrangement is to pay them, but Yahoo doesn't offer this as an option.

In fact, I assume the real cost to Yahoo here would not have been actually reviewing Barnes's complaint, but actually finding it buried among all the bogus complaints that they receive, and noticing that it had real merit. Again, including a $5 payment would be one way to ensure that your complaint gets taken more seriously than all the others. But while the $5 fee might have helped in this specific situation, it's easy to imagine how that could set a bad precedent -- ISPs charging exhorbitant fees for users to submit abuse complaints to them, or users not filing complaints because they didn't want to share their payment information or pay money at all.

So, rather than paying a small fee directly, a better approach might be to require complainants to post some sort of "bond" -- which may not be something financial, as some examples will show -- in order to get their complaint to the front of the queue. Recall the example of submitting your driver's license along with an impersonation complaint. It's important to understand the subtle reason why this procedure actually works. It's not because someone couldn't still file a bogus complaint with a phony ID. (While it's somewhat hard to create a fake driver's license that you can hold in your hand, creating a fake faxed driver's license would be easy.) It's because if the complainant is lying, now they can be prosecuting for forging government documents. Essentially the complainant is posting their freedom as a "bond", going out on a limb and saying: "I can't prove to you that I'm telling the truth. But now you know that if I'm lying, I'll go to jail. Bet you the other guy won't be willing to make a binding promise like that."

So naturally I'd put that in the ISP's policy as well: If someone sends in a complaint about our user impersonating them, and they're willing to fax in a copy of their government ID proving that they are who they say they are, and we can verify that the page owner is claiming to actually be that person (and not merely complaining about that person or their business), then we would remove the page unless the account owner can submit even more compelling evidence that they are who they say they are.

This addresses the problem of the impersonation complaints that are completely fake. However, you still have the problem of what to do about people who fax in their driver's license along with letters saying "This guy is a jerk", hoping to get someone's account closed down. If a company like Yahoo is too big to read through all the complaints carefully, then it becomes hard to sort through the complaints to see which ones are really about impersonation and which ones are about other behavior that doesn't violate their TOS.

What might be a solution would be to borrow some of the non-terrible aspects of the Digital Millenium Copyright Act. The two most controversial provision of the DMCA are (1) a ban on software that enables the user to circumvent copyright restrictions, and (2) a requirement that ISPs have to respond to copyright-violation "takedown" notices in a certain manner. As I've said before about the DMCA, I'm opposed to #1 in principle because I think software should be protected by the First Amendment; I'm not against #2 in principle, but just concerned about how it could be abused in practice.

But one thing the DMCA does is solve the "sorting problem" -- how to get complaints about copyright violations to the top of the pile. Service provides often have a procedure for handling DMCA complaints that is separate from the regular complaint channels. The DMCA also provides protection for users against phony complaints, by stipulating that anyone who files a false complaint can be sued for statutory damages and attorney's fees, as in a case where Diebold, Inc. agreed to pay $125,000 as a penalty for sending false "takedown" notices. In other words, the DMCA solves the "bonding" problem too -- by sending a DMCA complaint, a user is effectively saying, "I agree to pay big money if I'm lying. So, I'm probably telling the truth."

So, a law addressing how ISPs should handle "impersonation" pages, modeled after the DMCA to solve the "top of the pile" problem and the "binding promise" problem, might go something like this:

For a user to file a complaint, the complaint should cite the name of the anti-impersonation law, as in, "This complaint is being filed under the Anti-Impersonation Act of 2009". This gives ISPs an easy way to sort these complaints to the top of the pile, the same way that they have specialized channels for handling DMCA complaints.
In the complaint, the user has to assert unambiguously that the page they are complaining about is impersonating them, and is not merely posting gripes about them or their business.
The complaint should include a copy of a government-issued ID. (Again, this is not because this is hard to forge, but because now the complainant is promising, "If this is fake, I'll go to jail.")
If the impersonation page is directing visitors to call a phone number or e-mail an e-mail address, and the takedown notification to the ISP includes a request to call that number or e-mail that address to verify that it doesn't actually belong to the page owner, then the ISP should follow up on that within a given time period of receiving the complaint. (And once they call that number or e-mail that address and get a response saying, "No, that page is definitely not mine", then the ISP should shut the page down.)
Anyone who files a phony complaint citing that statute, can be held liable for statutory damages and attorney's fees, and if they faxed a phony government ID, then they can be prosecuted for that as well.

The problem-solver in me says that this is one way to ensure that legitimate complaints will be acted on, while making phony complaints much harder and riskier. It also seems to me that this is a minimal solution, in the sense that if you remove any part of it, it no longer solves the problem. For example, if you remove the part about complaints having to cite the anti-impersonation law, then you no longer have an effective means for these complaints to get to the top of the pile. And if you remove the part about civil penalties for filing phony complaints, then you no longer have any disincentive for people to tie up the system with crank complaints trying to get their enemies' accounts cancelled. Perhaps others can come up with an alternative solution that meets the logical requirements of enabling real complaints while discouraging fake ones. Meanwhile, the civil libertarian in me doesn't get a queasy feeling from it right away. It seems that it could only be used to stop cases of actual impersonation, and even as a free speech advocate I don't think that you have the moral right to impersonate someone else in a non-satirical manner for the purpose of actually deceiving or harassing people.

But even the absence of such a law is hardly an excuse for what Yahoo did. All they had to do is go to the page, look at the phone number, call the number and hear her say, "Yes, this is me and no that's not my page", and shut it down. The fact that they couldn't do this, shows a contempt for the process of handling legitimate complaints. Apart from the harm caused to Cecilia Barnes directly, incidents such as these might lead to Congress narrowing the scope of the immunity given to providers for hosting content posted by their users. Of course I'm technically suggesting a law that would narrow the scope of that immunity too, but only in a very narrowly prescribed way. If, on the other hand, Congress or the courts ever adopt the vague principle that providers can be held "jointly responsible" for whatever their users say once they've been "made aware" of it, it's going to get a lot harder for people to find Web hosting who have anything controversial to say.

This discussion has been archived. No new comments can be posted.