Why Google's Gmail Phishing Warnings Give False Positives (vortex.com) 49
Vortex.com is one of the oldest domains on the internet -- one of the first 40 ever registered, writes Slashdot reader Lauren Weinstein. So why does Google sometimes block the email he sends?
Here's why. First, my message had the audacity to mention "Google Account" or "Google Accounts" in the subject and/or body of the message. And secondly, one of my mailing lists is "google-issues" -- so some (digest format) recipients received the email from "google-issues-request@vortex.com"... Apparently what we're dealing with here is a simplistic (and frankly, rather haphazard in this respect at least) string-matching algorithm that could have come right out of the early 1970s...! [A]t least in this case, it appears that Google is basically using the venerable old UNIX/Linux "grep" command or some equivalent, and in a rather slipshod way, too.
In addition, the article concludes, "I've never found a way to get Google to 'whitelist' well-behaved senders against these kinds of errors, so some users see these false phishing warnings repeatedly.
In addition, the article concludes, "I've never found a way to get Google to 'whitelist' well-behaved senders against these kinds of errors, so some users see these false phishing warnings repeatedly.
Probably an acceptable trade-off for Google (Score:5, Interesting)
With the huge volumes of data that Google handles, it's probably hard to do any better.
AI style approaches can fail in quite unpredictable ways, and I think Google likely much prefers that too much is blocked than failing to find something obviously fishy but that gets through the algorithm for some obscure reason.
Sometimes simple approaches are the way to go. You're going to have false positives and false negatives no matter what, the question is how much and in what circumstances. And this particularly scenario is unlikely to be all that common.
Re: (Score:3)
On the other hand, it wouldn't be hard to provide some kind of whitelist procedure; or better yet a way of slotting email verified from certain domains into certain algorithmic tracks.
Re: (Score:2)
But domains - even old, established ones - change hands somewhat regularly. So maintaining a useful and effective whiltelist would likely involve significantly more work than one might think.
Re: (Score:2)
Sure. And if people start *reporting* spam coming out of formerly whitelisted domains, you must un-whitelist them.
Re: (Score:2)
Sure. And if people start *reporting* spam coming out of formerly whitelisted domains, you must un-whitelist them.
I think it is more complicated than that if you want to use domain name alone.
What if a new domain is created, how do you put it in the white listing? If your domain was compromised without your knowledge and was used in spamming/phishing, how do you know that your domain is removed from the white list? And how do you prove that your domain is now safe to be enlisted in the white list again? How much would you lose while you are trying to get it back on the white list? There are a lot more cases going on th
Interface could be better (Score:4, Interesting)
With the huge volumes of data that Google handles, it's probably hard to do any better.
GMail may be "hard to do any better", but dealing with spam is complex and labyrinthine.
My client began having conversations with a vendor last week, and as a result GMail put *all* subsequent E-mails into my spam folder, including ones from my (whitelisted) client addressed to the vendor CC'ing me. I only found out by accident.
One might *expect* a quick, easily identified control that says "whitelist this person" or "whitelist this company", but there isn't. You have to go to "Settings->Settings->Filters and blocked addresses", none of which terms are "spam", so the casual user can't just scan headings for the term.
You can't, apparently, just refer to the spam and say "whitelist that person", you need to create a new filter. You can't, apparently, say "@example.com" as a wildcard for the business, you have to identify an actual sender by complete address.
And of course, you have to discover that you need to do this, because GMail doesn't give any warning. (Surprising, since every time I use GMail from a different location it sends me a warning E-mail. Every. Single. Time.)
I'm not even sure why everything went to spam in the first place - I had sent E-mails to both the vendor and the client, so they should have been in my "recently used" list.
GMail has a pretty cryptic interface, compared to some of the other mail readers I've used.
Re:Interface could be better (Score:4, Interesting)
If they gave you options & explanations it would spoil the flat simple look!
Re: (Score:2)
A false positive implies that Google is wrong. Google is not wrong this case. Vortex.com is just not keeping its identity secure. Any email you receive from that domain should automatically be treated as suspect because it could have been sent by anyone.
I may not be able to send email directly from vortex.com because vortex.com has an SPF record, so at least, they secured that much, but anyone can easily forge an email header with the vortex.com domain name because they didn't [mail-tester.com] bother to implement DomainKeys
Just a thought... (Score:4, Interesting)
Tweak your mailer so that it sends mail from gi-request instead of google-issues-request, and don't mention "Google Account". Granted, this sucks, but the Internet routes around brokenness, and that's what you need to do in a situation like this. Is that a sad thing? Yes, of course. If we had a mail architecture that was pull- rather than push-based, maybe we could have nice things, but until that magic day, the whole thing is bubble gum and bailing wire, and it's honestly not Google's fault that that's so.
As another example of brokenness, I often get mail that is marked spam because it went through a mailing list expander and the headers didn't get rewritten, so that it fails DKIM validation. Yes, we can all rail about how evil and awful DKIM is, but the bottom line is that if you don't want that to happen, you rewrite the headers. Again, a system that's pull-based rather than push-based would make this a lot better.
Re: (Score:2)
It's push-based because of history. The number of connections is the same either way: the sender has to announce that a message is available. The difference with a pull-based solution is that the receiver ignores announcements from senders it doesn't know, and decides when/if to pull. You use pull-based solutions every day. Facebook is pull-based. Reddit is pull-based. The reason SMTP isn't pull-based is that back in the day, we didn't realize that there would be assholes. It's really that simp
Re: (Score:2)
It is actually a mix of both. SMTP is push-based but POP/IMAP are pull-based. A purely push-based or pull-based system would require 24/7 connectivity for all clients (for receiving and emitting respectively).
Making the internal connectivity pull-based would just make things slower. Mail is push-based by nature, sending mail is the active part, receiving it is passive. With real-life post offices, the sender is the one who do all the procedures and pays for the stamp, the receiver just needs to check his ma
Re: (Score:2)
No, you'd still have servers, and it's servers that would be on 24x7. Your client would use IMAP or JMAP (hopefully not POP).
Re: (Score:2)
What I mean by "the headers didn't get rewritten" is that the sender didn't get rewritten to a sender that would validate. If I forward your message from my server, you aren't sending it, and so the DKIM isn't going to validate. I have to send it as me.
Re: (Score:2)
Sure. So what happens if From: isn't on that list? Answer: the message is rejected, if the recipient is e.g. yahoo or google.
Too whiny (Score:5, Insightful)
C'mon, Lauren, with the 10's of millions of spams that google catches every day, some things are going to get caught by the filter that shouldn't be. Even if the filter is 99.99 effective that means there will be 1000 false positives in there...and yours is one of them. Shit happens. Adjust and move on.
Apparently what we're dealing with here is a simplistic (and frankly, rather haphazard in this respect at least) string-matching algorithm that could have come right out of the early 1970s...! [A]t least in this case, it appears that Google is basically using the venerable old UNIX/Linux "grep" command or some equivalent, and in a rather slipshod way, too. is drawing a trend and a conclusion from one data point.
Don't look for constructive ideas on Slashdot (Score:2)
Even for today's Slashdot, that was a remarkably low-insight comment to get an insightful moderation. There are constructive approaches to consider, but at this point, why bother?
Implement SPF and DKIM (Score:5, Informative)
Re: (Score:2)
So you're SPF and DKIM signed - you can still be marked by Google as spam based on content, which is what their problem is here. I know - I've had to deal with it, and it's very annoying because no ESP out there cares about senders - you're not their customer.
Re: (Score:2)
SPF and DKIM can be useful for resenders, but it adds nothing to security for original senders except maintenance costs and delays. If the A record for the sender address points to the IP of the remote MTA, it can be generally be trusted to come from that domain. Unless the DNS server is taken over, in which case SPF and DKIM won't do much good either.
1970s (Score:5, Funny)
You mean like that vortex.com [vortex.com] front page?
Re: (Score:2)
Easy to read, links are unambiguous.
What's to hate?
Bullshit! They use AI. (Score:1)
The AI is so advanced it looks exactly like a human running grep.
first impressions usually right (Score:3)
Bad guys only have to succeed once. (Score:1)
A million false positives are much more preferable than one false negative.
Re: (Score:2)
Really no.
Having all my incoming and outbound email thrown away because I might be a phisher or one of the people sending something to me might be is not a good thing. (And there are random days when for no obvious reasons suddenly some portion of mail in either direction for one of my mail accounts appears to be treated as SPAM by the large mail handlers for example, and I don't know.)
I am capable of spotting and avoiding responding to phishing attempts myself, without assistance.
I get 10,000 SPAM deliver
Regardless of what you think of Vortex... (Score:1)
Setting the source and tone aside is crucial to any good analysis of a subject, so setting the Vortex source aside, I've noticed two things that seem to be relevant here. The first is that while Google is usually pretty good at blocking spam without false positives, it seems to be getting worse at that task, rather than better. Furthermore, it is notably bad detecting when an email is or is not a scam.
The second, more important, point comes at the end of the original note, and it's that Google as a whole