The Evil in E-Mail 211
Frenchy in Ontario writes "An Ontario university researcher is devising ways to help law enforcement agencies better pinpoint likely criminal behavior in e-mails. His theory is that people who are "up to something" are more likely to write differently than people who aren't - either by avoiding using certain words at all that could be flagged for possible criminal context (like "bombed) or to examine patterns that might indicate criminal activity - like several people e-mailing one person but not each other, which is how some criminal networks operate. There's also an interesting paragraph on why Enron's emails aren't as valuable as you might think for this sort of work."
Dumbest thing I've read all week... (Score:5, Insightful)
From TFA:
Super. I'm predicting a whole lot of false positives...especially during the initial phase of this operation...
Also from TFA:
Great...so words like 'bombed' get the email flagged...as well as an absense of the word 'bombed'? So far, Skillicorn's test appears 100% sensitive...too bad it's 0% specific.
Some more from TFA:
OMG! This is the pattern of emails in my company! My whole company is a giant terrorist organization! I had no idea!
But here's the kicker...again with the quoting:
So let me get this straight...if criminals are okay with their criminal activity (like...say...terrorists), they'll 'slip under the radar'??? Great test, Skillicorn...sounds a lot like a standard polygraph test, which experienced criminals can fool at will, while innocent people fail them 50% of the time. That's what the War on Terror really needs...another inaccurate 'test' that does nothing but throw false positives.
I'm just glad that this method is so obviously stupid that it will never be implemented by our government...
Oh, wait...one more from TFA:
Crap.
Agreed (Score:5, Funny)
This line in the lead jumped out at me: We have an addresses "techsupport@internaldomain" which matches this pattern to a T.
--MarkusQ
P.S. Back when we were on MS-Windows, it would have been OK, because the people asking for TechSupport were often sending each other worms at the same time.
Re:Agreed (Score:2, Funny)
Re:Agreed (Score:2)
Re:Agreed (Score:5, Insightful)
In a court, you can question the evidence used against you. Considering that the creator of this evidence indicated that he didn't need to know how it works, it's highly likely that you could get this evidence thrown out because it fails the test of provablility.
So this technology will "flag" people, and they will be watched "just in case". However, there's not going to be a court case, just continued monitoring until the budget to watch this person dries up. And it's very easy to get a bigger budget because you can argue, "We are watching 400,000 people who have been flagged as possible terrorists, we can't keep up. We need more money." Even when your flagging system has worse odds of finding a terrorist than the Lottery.
Re:Agreed (Score:2)
Re:Agreed (Score:3, Funny)
Suspected terrorist has posted an apparently coded message on Slashdot indicating connections with terrorist supporters in Middle Eastern countries.
Suspect has possible sexual relations with both his wife and his sister based on frequency of email contacts.
Suspect is apparently concealing his connections with his wife's mother from his wife. His wife, however, is also in contact with the terrorist leader. Indications are his wife is part of a different cell than the suspect. {See "Mr. and M
Re:Agreed (Score:2)
We have an addresses "techsupport@internaldomain" which matches this pattern to a T.
Yeah, and I'd never realized that all of those geek mailing lists that I'm on are centers of illegal activity.
After all, it's only the newbies that use "reply to all" and produce messages between subscribers. The experienced list members usually figure out that getting two replies to a message is dumb, and just
Re:Agreed (Score:2)
Re:Agreed (Score:2)
like several people e-mailing one person but not each other, which is how some criminal networks operate.
Yeah... And classes?
We used to have a right to assemble somewhere, now if only I could find where that was written. Ah, there it is. No, wait, that just says, "Some rights are more inalienable than others." Hmm...
Re:Dumbest thing I've read all week... (Score:3, Insightful)
SPAM.
Re:Dumbest thing I've read all week... (Score:5, Interesting)
Another, Skillicorn says, is that research shows
people speak and write differently when they feel guilt about a
subject, for instance using fewer first-person pronouns, like I and we.
Because people always use first person pronouns in messages. That's just what's done. And alot of them should be used.
Sounds like a way to track messages with "substance" rather than the "hai h u r? heer are the pictures of my vacation." messages.
Think about that. This man has just come up with a way to measure the relative interest of what the sender has to say to people in the government.
Yet another way to cut down on the messages that the government has to read and be bored with. Yet another way to enable the government to read out communications more effectively
Yet another reason to look into using real encryption.
Re:Dumbest thing I've read all week... (Score:4, Insightful)
Yeah, sure, until using encryption is flagged as a likely indicator of criminal activity, too...
Remember, if we don't all walk around with our pants down in public, that means that we've got something to hide.
Re:Dumbest thing I've read all week... (Score:2)
Re:Dumbest thing I've read all week... (Score:2)
Re:Dumbest thing I've read all week... (Score:2, Interesting)
Re:Dumbest thing I've read all week... (Score:2)
No kidding. I can smell my tax dollars burning...
Re:Dumbest thing I've read all week... (Score:5, Insightful)
1) Devised a theory
2) Tested it on a sample set of emails from Enron
3) Gotten poor results
4) Blamed the failure on Enron, for being just *too* evil for his theory to work!
Yawn. Maybe he should save the press release until he's gotten something to work.
Re:Dumbest thing I've read all week... (Score:2)
Re:Dumbest thing I've read all week... (Score:4, Insightful)
Re:Dumbest thing I've read all week... (Score:2, Insightful)
Crap.
But of course. It is the nature of the monitoring beast and the very reason such monitoring is offensive to freedom.
First you monitor. Then you monitor for the people avoiding the monitoring. Then you monitor for the people avoiding the . .
Monitoring, if it is to work at all, is an all or nothing sort of deal. Once started it innately progresses toward the end of a secret cop in every pocket. If you know t
Re:Dumbest thing I've read all week... (Score:2)
ATTN: department of homeland security (Score:2, Funny)
I gonna get bombed Friday = Terrorist Threat (Score:2)
Re:Dumbest thing I've read all week... (Score:2)
Re:Dumbest thing I've read all week... (Score:2)
A related trick, he says, is to examine patterns in who e-mails whom. As an example, in criminal networks it is common to find several people communicating regularly with the same person, but never with each other.
OMG! This is the pattern of emails in my company! My whole company is a giant terrorist organization! I had no idea!
Not that I agree with all this emphasis on monitoring, but think about the utility of this. This technique isn't useful in mapping out "terrorist organiza
Re:Dumbest thing I've read all week... (Score:2)
Re:Dumbest thing I've read all week... (Score:2)
He's basically right, though. If you have a technique that works, and you're explaining it for general audience, wouldn't you mention the technique's successes? If Skillicorn has any success stories, he's done a good job of hiding them.
It seems clear that all he's doing is applying standard do
Re:Dumbest thing I've read all week... (Score:2)
Oddly enough, I find myself wondering how your last couple posts would be analyzed by the "evil-filter".
Re:Dumbest thing I've read all week... (Score:2)
What about other languages.? (Score:4, Insightful)
Re:What about other languages.? (Score:2, Funny)
Re:What about other languages.? (Score:3, Insightful)
-
That's false. (Score:2)
http://www.paulgraham.com/spam.html [paulgraham.com]
You just need a good corpus for your text classification.
Re:What about other languages.? (Score:2)
It sounds like the guy...
a) had a half-assed theory (that should've probably
b) came up with little data to support his theory
c) explained the lack of supporting data for his theory on the material being studied
d) used a lot of pretty awful reasoning to shoehorn what little supporting data was left i
Re:What about other languages.? (Score:4, Interesting)
It was written during Cold War and of course referred to socialist governments of the time but I see new paralles now.
Re:What about other languages.? (Score:2)
Actually, you don't even have to do that -- the whole point of word frequency analysis is that sensitive words are detected automatically.
(You do need native speakers to help you with syntactic constructions and stuff if you want to do any
A couple of reasons for it not to work (Score:4, Interesting)
- Many languages are conjunctive/agglutinating in nature (e.g. Turkish [wikipedia.org], Finnish [wikipedia.org], Swahili [wikipedia.org]). This means that words of sentences aren't isolated (like most European languages) but are in fact formed from 'parts' that change depending on the surrounding words. Moreover, modifying pre-/suffixes are used as inflections for e.g. verb paradigms. This results in language that effectively have literally billions or even an infinite number of possible "words". It is impossible to do keyword-based analysis on such languages without a full morphological parser for each language to break a word into its 'parts' - such a parser is a massive task.
- Chinese is the opposite, it is a totally "isolating", meaning each word is distinct with no inflections, and because different characters are used for different words there are NO SPACES between words. So you cannot begin to analyse Chinese data at all unless you have a full "Chinese segmenter" to locate word boundaries.
The need to do further disambiguation further complicates all of this analysis.
There is pretty much no way for this type of analysis to be really accurate under the current level of written language analysis technologies.
Re:A couple of reasons for it not to work (Score:2)
So just to add to that --- given that these technologies can only work primarily for English and a few other major European languages --- who are they truly intending to watch with this? The terrorists, or their own populace?
if you're really up no good.. (Score:5, Insightful)
Real criminals aren't dumb, only the bad ones who get caught are.
What you should do... (Score:2)
Re:if you're really up no good.. (Score:2)
The only way out of this scenario (, and I would advise it ONLY if you are truly up to no good,) is to register as a Republican and make substantial financial contributions to the GOP. Only by such means can you escape detection. As a positive side note, you might also obtain additional "cover" when asked to serve in the administration of the regime currently in power.
For everyone else out there, well, yo
The idea isn't new... (Score:4, Insightful)
I do disagree with his statement about not being useful to fight spam - recognizing patterns ins spam is already in use, applying the idea that the same or significantly numerous occurrences of the same words from either the same person to multiple users at the same sight and different sites, or the same basic message sent to multiple users from different mailers / return addresses might be a good indicator of spam. The challenge is how do you monitor all the traffic?
Someone set us up the Bombed (Score:3, Insightful)
Re:Someone set us up the Bombed (Score:2)
Subject: FidelCatsro|861135|Slashdot.org
Message ID: 12794056|Slashdot.org
Keyword Trigger: "BOMB"
Analysis: Subject is likely not a terrorist
EOM
-
Bad sample? (Score:5, Insightful)
I don't know if using the Enron e-mails as his test material is such a good idea. Corporate malfeasance is probably not conducted the same way that every other criminal (or terrorist) network runs. At least their communication might be different due not to a "lack of guilt" but due to the fact that it's probably so easy to make a naughty memo sound like an innocent one without being obvious. After all these memos would be mixed in with a lot of legitimate company business the conspirators are also conducting.
How does automated analysis separate a memo saying "I think we should go ahead and promote Price out of the mailroom" - which means "Have Price-Waterhouse cook those spreadsheets I sent you", from one which just leads to some dude getting promoted out of the mailroom? Of course if they are not bothering to use code words then the system might work very well.
A related trick, he says, is to examine patterns in who e-mails whom. As an example, in criminal networks it is common to find several people communicating regularly with the same person, but never with each other. This is meant to ensure that if one lawbreaker is caught, he or she is unlikely to lead authorities to too many others. But it can also be a clue to suspicious activity.
Traffic analysis is probably more promising, since you can reconstruct relationships between players with it. The traffic pattern could look like a terrorist cell, or it could look like a bunch of guys who know each other - as he says, there's a difference. But this is old news, though automating it would make snoops' lives easier.
At any rate I find this line of inquiry disturbing for civil rights reasons, but I don't believe we should attack the researcher for working on it. Academic freedom is a very useful concept and ultimately does us more good than harm, IMO.
Whatever (Score:2, Insightful)
Re:Whatever (Score:2)
Great deal on Hidden Al-Qaeda messages. aff.
eBay.com
Word bombs.... (Score:5, Funny)
That should keep me safe for a few days.
Re:Word bombs.... (Score:2)
Mail bomb.
Assassination.
Fertilizer.
Same-sex marriages.
Patagonia.
Nader for President.
Re:Word bombs.... (Score:2)
Re:Word bombs.... (Score:2)
Computer: =-O
I can't believe this got funding... (Score:5, Insightful)
Personally, I can't see how this would ever work. It is typical of the attitude that "all terrorists are bad, they are all the same and we just have to deal with them all in the same way".
Isn't it obvious that different terrorists will have different styles, different levels of literacy, different levels of security awareness, different languages, different aims, different approaches - the list goes on and on. Normal emails all have these traits too. I can't imagine there is any way of applying Bayesian filtering to help with this task.
GPG (Score:2, Insightful)
Re:GPG (Score:2)
The more likely scenarios would be (1) nonsensical SPAM messages with a hidden message within, (2) a message steganographically embedded within images posted to a public forum, or (3) some pre-arranged and totally innocent message that has been assigned some other meaning.
The use of encryption for email messages represents such a small proportion of the total number of emails sent that it would
Big Brother right here (Score:4, Insightful)
Re:Big Brother right here (Score:5, Funny)
Re: (Score:2)
Re:Big Brother right here (Score:2)
Social Networks = Criminal Networks? (Score:3, Insightful)
Not to mention most social networks. Or is everyone you know equally popular?
Too narrow (Score:2, Interesting)
or something which I tried out in a few experiments last year. It is supposed to analyse writing and determine whether its author is female or male.
It works rather well given the conditions that the authour is also is American, white and middle class. Any samples outside that field and it fails spectacularly actually getting more wrong than right (worse than chance).
These sort of ideas are cute in their ambitions
but not science of any kind at all. The tests giv
Re:Too narrow (Score:2)
Well, son, pass it along to homeland security! Those are exactly the sort of terrorists they should be after!
If Terrorists are Smart (Score:2)
Oh dear (Score:5, Insightful)
Bah (Score:3, Funny)
Hmm (Score:2)
Like <president@whitehouse.gov> for example?
Re:Hmm (Score:2)
you encrypt - you're a terrorist (Score:3, Interesting)
First they start using some very un-smart word-scanning piece of crap filtering system [and god help you if you write foreign language letters, or have a different style than the average], then they will punish the use of mail signing and encryption software [which is something I regularly do], then if the filtering still has a false positive rate above 99% they will ban e-mailing. Then they will find out other forms of efficient communication exist.
Re:you encrypt - you're a terrorist (Score:2)
If you're innocent, you have nothing to hide and therefore don't need to encrypt things. Only terrorists need to use strong encryption.
And while I'm at it, why do you hate America, and won't somebody think of the children?
Wasted effort (Score:3, Funny)
Do they believe in the effectiveness of this.....? (Score:3, Insightful)
Re:Do they believe in the effectiveness of this... (Score:2)
Yes, but just remember, it might not be american law enforcement doing the violating. AFAIK ECHELON (search google for it) exists and is working. If say the british government did the searching through american email it would not actually be the american government spying on their own citizens and vice versa.
"How much more of this are we willing to accept."
I'm hoping not a lot more.
"This will NOT be effective for the worst thieves or tererorists."
this research is a wonderful example of... (Score:3, Insightful)
Privacy? (Score:2)
P147 1n 5lt.5n1nym1U6 u64 kn1wn crypt1, l23v2 n1 tr5c4. 4m52l != s5f4.
num=num-1
Ofcourse I know, if the system is sortof good inwhat it does, the above wont be too effective. But who wants will bypass such a thing. What about PGP?
I honestly doubt that the system will result in displaying the emails of those who actually are paranoid about their activity and doing things that aren't too legal. It'd be disturbing if they will be successfull with this type of privacy infringement. It's
Re:Privacy? (Score:2)
Regarding snail-mail vs. email...with snail-mail you seal the message inside an envelope to be opened at the other end, thus a reasonable expectation of privacy. Email, on the other hand, is generally transmitted as plaintext all the way from A to B...more like a postcard than a letter. Thus no expectation
Re:Privacy? (Score:2)
Re:Privacy? (Score:2)
The Belgian constitution [senate.be], article 29 states that the "Letter secrecy" CANNOT be violated. (this means, noone can open the letter other then the adressed.)
According to the Belgian MailOffice [www.post.be] they obey this law stating:
Only a select few DePost employees are allowed to open letters which are undeliverable in very rare conditions to find who the letter is adressed to. (0.02% of the mail would be undeliverable)
The Belgian const [senate.be]
Oh Dear (Score:3, Insightful)
So if you don't talk about things which a terrorist would talk about, you are a terrorist?
like several people e-mailing one person but not each other, which is how some criminal networks operate.
Yes, it's also how every other nuclear network of friends operates. Not all my friends know eachother. Not all a bank's customer's know eachother, not all a mailing list's users know eachother.
Government sucks. (Score:5, Interesting)
-Tacitus
Government is already too invasive. I'm already forced to seek a building permit before I can erect a structure on my own property. The fines for ignoring this, (and say, having the gall to build a solar powered house which is not connected to the AC power grid, or (horrors!) a straw-bale house), are huge and the government's reasons for these laws are utterly ridiculous.
Any professor who suggests that we should be looking to monitor email content is not thinking clearly. The Government already has their nose in everything, and telling us that, "It's For Our Own Good," is NOT a valid excuse.
It's MUCH more important that people be able to make mistakes -and even die through their own faults- than live ensnared in the safe-keeping of a bunch of ignorant civil servants who are trying to build a Starfleet future where everybody dresses the same, and nobody is allowed to think or act outside a bunch of pre-set 'safe' boundaries designed for middle-class suburbanites who exist in eternal ignorance of the real world, who actually believe in the Discovery Channel, who drink milk, and live in absolute terror of anything you can't experience beyond the confines of a nice, respectable department store.
-FL
Re:Government sucks. (Score:2)
Generally, a densely populated society requires straightjacket laws to make sure everyone more or less gets along. The less behaviours permitted, the fewer things will piss off your many neighbours.
I think the West needs to get a little closer to 'my right to swing my fist ends at your nose' - but keep in mind that if that happened tomorrow, we'd start campaigning for a return to more restrictive laws as the worst 19% of t
Regulations for the ignorant, and Fire safety. (Score:2)
letter from college home (Score:5, Funny)
Hi Mom,
I blew it and bombed the final exam. The physics
prof put the gun on my head and told me to work harder.
I could kill him. I feel like having a knife
at my throat. The anger feels like poison in my
blood but I know it is my fault and the all is
blamed to that virus, I had been laboring with
for quite a while. I'm working on it mom! I promise
to make you proud. I can not wait to be on the subway
home to work on my final project on weapons of
mass destruction in my political science class. Its
mental terror.
Love
Your son
P.S. The powder you sent me works well for my
skin infection. Strong agent.
He must be up to something fishy. (Score:2, Funny)
Been There Done That (Score:2, Interesting)
Why ? The size of the token database increases massively to the point where it becomes un maintainable. Every spelling mistake, word variant, not to mention foreign language, gets included. Eventually you are unable to separate the wood from the trees. Let alone make statistically significant assertions about a single message.
And lets not
First things first (Score:2)
I don't know if they then pay special attention cuz we think we can bypass their filters, or completly ignore us cuz 99.9999998% of
Just stupid (Score:3, Insightful)
This is clearly just designed to appeal to the government of Police State America, probably to get more funding.
This whole obsession with 'terrorists' is just becoming tiring. There are very few 'terrorists' in the world that the Americans didn't create through their own acts of terror. If America would stop its interference in the affairs of other countries, there would probably be almost none at all outside of the White House.
Re:Just stupid (Score:2)
A naive way of looking at the world at best, extremely dangerous at worst. The canard that "America is to blame" is as tiring as the faux security around terrorism (yes, I agree a lot of the attempts to rein in terrorism are bogus and do little to stem
And you are wrong (Score:2)
(*) and I am not even counting the number of country where the good old US of A support a dictatorial govt, or attempt to o
I concur (Score:4, Funny)
"please verify your credit card information", etc.
Holy Thought Police Batman! (Score:2)
News: Chocolate rations are up from 200g to 150g this week.
War is Peace (Score:2)
Re:War is Peace (Score:3, Interesting)
Then all that will be left is futile, self-destructive petty rebellion.
Re:War is Peace (Score:2)
Comments highlight a blind spot of geeks (Score:2)
This kind of syst
zerg (Score:2)
pipe dreams (Score:2)
I Guess George Bush is the perfect example of this (Score:2)
Re:Spam is a criminal conspiracy (Score:2)
BANG (Score:2)
See you in the place where there is no darkness.
Re:This should be easy... (Score:2)