The Dark Side of the Web 156
Barence writes "Beneath the web pages indexed by Google lies an online world that few know exists. It's a realm of huge, untapped reserves of valuable information containing sprawling databases, hidden websites and murky forums. It's a world where academics and researchers might find the data required to solve some of mankind's biggest problems, but also where criminal syndicates operate, and terrorist handbooks and child pornography are freely distributed. Interested? You're not alone. The deep web and its 'darknets' are a new battleground for those who want to uphold the right to privacy online, and those who feel that rights need to be sacrificed for the safety of society. The deep web is also the new frontier for those who want to rival Google in the field of search." The melodrama is tempered, though: "The deep web isn’t half as strange or sinister as it sounds. In computer-science speak, it refers to those portions of the web that, for whatever reason, have been invisible to conventional search engines such as Google."
interested in teroirsm and cihld pron? (Score:1, Insightful)
TFA:
>terrorist handbooks and child pornography are freely distributed. Interested? You're not alone.
No, actually, speak for yourself!
Re:interested in teroirsm and cihld pron? (Score:4, Insightful)
Re: (Score:3, Funny)
Holy shit, a vagina that I didn't have marry anyone in order to see, this is awesome!
Re: (Score:2)
But does he abhor them enough to support trampling all over the rights of everyone else? That's the true test of a patriot.
Terrorism (Score:1, Interesting)
I find it ironic that in the online world there are places where the Terrorists and those who support them are calling us, the Freedom Loving People, terrorists !
By the original Terrorists I mean those who strap bombs to themselves and go KABOOM ! taking themselves with innocent people around them.
What I find ironic is THEY call us terrorists !!
Re:Terrorism (Score:5, Insightful)
Re: (Score:2)
Christianity says Jews are the Chosen People so we must support them at ALL costs and be horny for the Apocalypse.
That superstition just happens to conflict with Islam, an even more (and that's quite an accomplishment!) toxic supersition.
The only problem with such a religious war is that innocent atheists may be harmed in the process.
Re: (Score:1, Insightful)
I mean really, what in god's name made the UN believe that it was ok to throw a few million Jews in the middle of several different groups of people that hate them?
Up until the 20th century, the Palestinians did not hate Jews. Zionist immigrants (trying to get backing from the British colonial power), however, managed to change that in the early 20th century. And most people really don't like having *their* country signed over to someone else. I find it surprising how anyone could miss that it would lead to a permanent state of war.
Re: (Score:2)
Up until the 20th century, the Palestinians did not hate Jews.
Look buddy ! /. isn't the place for you to spread your lie.
Crawl back under that rock of yours and spread your lies there !
I have had enough of you terrorist vermins !
Re: (Score:1)
Re: (Score:1, Interesting)
Re: (Score:1)
But that isn't where they wanted to go.
Well, okay, sure, some of them probably would have taken you up on it, especially right after the holocaust. Goodness knows some of them have moved just about everywhere else in the world.
But that wouldn't have stopped Jews from moving to the middle east. They'd already been wanting to get back into "the land" for a while (roughly, since the first century), and increasing numbers of them h
Re: (Score:1)
Re: (Score:2)
Re: (Score:2)
The U.N. had nothing to do with it. Jews had lived there for three thousand years; it's where Jews come from. Jews lived throughout the Middle East for two thousand years, with a significant minority population in all of the Arab states until the pogroms following Israel's independence. The "few million" Jews moving to Israel were refugees fleeing persecution in Europe who bought the land they moved onto and had every right to move there. The Arabs at the time were not opposed to Zionism (the progressive idea of social equality between the Jews and everyone else) except for a few rogue right-wingers who started riots, gained control of the mosques through violence, and were continually appeased and promoted by the British occupation until they became the predominant force. The 1948 war came at the end of a thirty year pro-war propaganda campaign that included the executions of dozens of Arab leaders who supported peace with the Jews. The U.N. only signed off on the deal after Israel survived what the Arab League had promised would be the greatest massacre since the Mongol invasions.
Even if the creation of Israel was a morally good thing to do, it wasn't very good from a practical point of view. Supporting Zionism put the western powers at odds with the Arab states which control a significant part of the world's oil supplies and exacerbated the instability in the region. It would surely have been more cost-effective to have maintained a large Anglo-American base in Palestine (which could have been done legitimately, since Palestine was held by the British under a League of Nations Man
Re:interested in teroirsm and cihld pron? (Score:5, Insightful)
It’s a world where academics and researchers might find the data required to solve some of mankind’s biggest problems, but also where criminal syndicates operate, and terrorist handbooks and child pornography are freely distributed.
At the same time, the underground web is the best hope for those who want to escape the bonds of totalitarian state censorship, and share their ideas or experiences with the outside world.
Interested? You’re not alone.
Re: (Score:1)
It's not my fault the quote was manipulated. I was quoting the poorly-worded Slashdot article.
And it should have been modded "funny."
Pretty much ruined it for me (Score:2)
I was thinking about exploring those 'darknets' but child porn? I don't even want my browser pre-caching its way into those websites much less directly stumble onto one. At least I was warned. Anyone foolish enough to go there can't expect to feel like a victim if they get caught in a dragnet for showing up on a bad site's web access log.
Re: (Score:3)
Re: (Score:3, Funny)
You could stumble on that stuff on the 'regular' intertubes too. Although in some places it is more likely than others (e.g. 4chan).
Been there, done that (Score:2, Funny)
It's called the Metaverse, created by Neal Stephenson in Snow Crash. We now know it as Second Life.
http://www.robotstxt.org/ (Score:4, Insightful)
http://www.robotstxt.org/ [robotstxt.org]
Says it all really, no need for a melodramatic "article" trying to draw parallels between the non indexed and page ranked portion of the net, and kiddie porn.
Some of us just don't want google indexing out stuff on general principles.
FX, types "brain tumour" into google
up pops page full of links asking me if I want to buy a brain tumour on fleabay...
Re: (Score:2, Interesting)
But not all robots honor the file you see...
Re: (Score:2)
Re: (Score:2)
Perhaps they don't index that data (although not necessarily true for some engines). That doesn't mean it's not crawled (and kept for another reasons).
That is exactly what it means. They can't crawl the data without exhaustively visiting the website. If a webmaster sees a bot hitting their restrictive robots.txt, and then continuing to hepelumph through the rest of the site grabbing crap, the webmaster can let his attorney decide which of a thousand roads to take next.
Google may be big, but they pay heed to the possibility of negative PR from spidering where they are not welcome.
It's hidden on a purpose (Score:1, Interesting)
From what I've seen and heard this 'hidden' information is hidden on a purpose - most such sites I've ever encountered are trafficking (child) porn, software, audio and video - there's next to zero informational value in this undernet. As someone once said "Information wants to be free" and if it isn't let it die.
The real problem which this article doesn't even touch is that sometimes it's getting very hard to find the information buried in millions of pages Google returns to your query.
Re:It's hidden on a purpose (Score:5, Interesting)
Re:It's hidden on a purpose (Score:5, Insightful)
You must be new here. "Darknets" have been around since the start of the internet, and there's nothing necessarily illicit about it. Sure, some people take advantage of the privacy, just like some of our neighbours/colleagues/priests do, but that doesn't mean we put cameras in every house or monitor every phone call to catch the terrorists/kiddie-fiddlers/drug-users/speeding-drivers/child-punishers/blashphemers/etc. And if you're looking at websites rather than places or resources, you haven't even scratched the surface.
Basically, people like to communicate online, but that doesn't give you (or Google) the right to index it or even access it just 'cos it's on the internet - whether you like it or not, it's their communications not yours. Don't see value in it? Don't spend your time there. Think it's illegal? Call the cops with details. Just like IRL.
Re: (Score:2)
While I agree with all you say in terms of privacy rights, I disagree with things being "off limits" just because it's implied.
If you put data on the internet which isn't access protected or encrypted, then like it or not it is accessible to the public. Putting a robots.txt on your site is like putting a "please don't read me" sticker on a book and then leaving it on the bus; sure it's a common courtesy to obey the label, but you can't really start citing invasion-of-privacy spiels if someone doesn't.
If you
Re:It's hidden on a purpose (Score:5, Insightful)
"Think it's illegal? Call the cops with details."
You: "Officer, I was interested in these so called 'dark nets' so looking around I happened to find a website with child porn on it. Clicking about the site I found there were literally hundreds of images so I thought I best report it to the police."
Police: "Please stay by your computer and we'll be around to arrest you shortly. Enjoy your 25 years in prison."
I think you need to review the "Don't talk to the police" video:
http://www.youtube.com/watch?v=6wXkI4t7nuc [youtube.com]
If you see a house being burgled, ignore it and continue on. If you see somebody being raped keep walking. If you see a child in trouble, absolutely never go near them. The last one is particularly important since children are the greatest risk to your freedom in the current political climate and should never be approached under any circumstances.
Re:It's hidden on a purpose (Score:5, Insightful)
Police: "Please stay by your computer and we'll be around to arrest you shortly. Enjoy your 25 years in prison."
That has never happened to me. I've called the police three times over finding child porn online. The first time was in the mid-90's when I found some on a local (popular) BBS, and the police were concerned but utterly clueless on what to do. This was before they had an actual "cybercrimes" division. I don't think anything ever happened with that one. The second time I called on another complaint, they referred it up to the FBI, I don't know if anything ever happened because I never went back to the site again for obvious reasons. The third time, the local police took all my information, said thank you, and I never heard about it again. I'm guessing they handled it so nonchalantly the third time because they finally got their act together, and knew what to do.
Never once did the FBI knock on my door, or the police harp at me or treat me like a criminal.
If you see a house being burgled, ignore it and continue on. If you see somebody being raped keep walking. If you see a child in trouble, absolutely never go near them. The last one is particularly important since children are the greatest risk to your freedom in the current political climate and should never be approached under any circumstances.
Are you a sociopath? If I saw any of the above I would try to help, especially in cases of direct harm, like rape and children in trouble. It is my civic, and human duty to step in. I couldn't sleep at night if I just walked away and tried to forget it. But then again I'm the type who stop and try to help injured animals, and swerve to avoid hitting squirrels.
Re: (Score:2, Interesting)
[troll]
Until the law grants me preemptive indemnity if I pull out a pistol and double tap the perp, no questions asked I'd probably walk. I have no civic duty to my neighbors--over half are on welfare, don't work, and are living off of my taxes. I'm not exaggerating--I'm in the middle of an apartment complex, and of the 8 apartments around me, there's one employed middle-aged woman.
Hell, I nearly got in trouble once for giving directions to a kid that was clearly lost in the supermarket and telling him "w
Re: (Score:1)
Right on the money! You should write a newsletter. I'm wondering how many of us there are.
Although I no longer live in an environment like the one you describe, I empathize with you completely, beca
"swerve to avoid hitting squirrels" (Score:1)
Re: (Score:2)
If you see a child in trouble, absolutely never go near them. The last one is particularly important since children are the greatest risk to your freedom in the current political climate and should never be approached under any circumstances.
I've seen people act as if they really believed that. A small boy crying desperately, any parent would realize that he was distressed. Plenty of adults within hearing, nobody seemed to care. I talked to him, found out he was lost in his new neighborhood and helped him find his way. And yes, I did think about the possibility that I would be taken for a child molester, but - suppose he didn't find his way and was picked up by someone not out to help him?
Re: (Score:2)
I think you need to review the "Don't talk to the police" video: http://www.youtube.com/watch?v=6wXkI4t7nuc [youtube.com]
This is among the dumbest things I've ever seen on the internet. Guilty or innocent, you're going to get more scrutiny from the cops if you don't cooperate. Once you're indicted everything the guy says is probably true, but when there's a bar fight and someone gets killed, do you really want to be the one person in the crowd who says, by omission "I will not deny that I killed that guy?"
Adopt a long term perspective... (Score:3, Interesting)
Just maybe not everyone in the world wants to be Google's bitch and allow them to mine their precious information for profit. Information may want to be free, but information is also power. Secrets are valuable to those who hold them, and in a near future world where information becomes increasingly more valuable, those who hold the secrets will be the most powerful.
Now just ask yourself, are you willing to submit to the likes of Google and give up the right and freedom to decide what to do with your infor
Re: (Score:2)
Re: (Score:2)
Re: (Score:3, Insightful)
Maybe you just didn't look in the right places.
It's possible.
Re: (Score:2)
From what I've seen and heard this 'hidden' information is hidden on a purpose - most such sites I've ever encountered are trafficking (child) porn, software, audio and video
My first reaction was to respond "bullshit!", but on further reflection I realized that parent post is legitimate within the context of an extremely narrow point of view. It isn't so much an example of bullshit as an example of limited thinking of someone who knows very little about a subject, but thinks he knows it all.
As a writer of fiction, I use the darknet extensively: I've got hidden wikis and websites where I collect information I don't yet want to share publicly, and where I compose rough drafts t
Re: (Score:2)
As a writer of fiction, I use the darknet extensively: I've got hidden wikis and websites where I collect information I don't yet want to share publicly, and where I compose rough drafts that I want to share with only a selected few. I am sure that I am not alone in this approach. I can't imagine any new authors of the 21st century who are not doing the same kind of thing.
This approach goes back way before the Web. The mailing list systems that developed back in the late 1970s and 1980s had "moderated" an
Re: (Score:2)
I have always interpreted this asa meaning that information wants to be free just like apples want to fall: information has a tendency to get out, and you ahve to actually try to stop it.
I learned a lot from this article (Score:4, Funny)
I heard something about a robots.txt file somewhere before, but I thought that all robots where smarter than me anyway.
I also learned about something called freeweb that may or may not be used for good or bad things. I then learned about TOR which also may or may not be used for good or bad things.
This article really opened my eyes to the vastness of the Internets
Time Travel? (Score:4, Interesting)
Re: (Score:2, Informative)
Either the poster traveled back in time or the date and month are transposed. I'm betting on time travel.
The article says it was "Posted on 3 Sep 2010 at 15:47". Unless I've missed something, we're still in March 2010...
Re:Time Travel? (Score:5, Insightful)
The american date format must die.
Re: (Score:2, Funny)
The american date format must die.
Quoted for truth.
Re: (Score:3, Insightful)
Re: (Score:2)
Re: (Score:1)
The [A]merican date format must die.
Agreed. [wikimedia.org]
Re: (Score:2)
Re:Time Travel? (Score:5, Insightful)
If someone asks you when you were born, you don't say:
"1964, 22nd, May", you say, "May 22nd, 1964" - which is the American format. Enough.
I would say the 22nd of May, 1964.
When someone asks you for the time, do you tell him in HH:SS:MM format? The units must be ordered either from least to most significant (dd-mm-yyyy) or most to least significant (yyyy-mm-dd). I don't care which you chose. But don't put the least significant unit in _the_middle_!!!
See this horrible Open Office bug on the subject:
http://www.openoffice.org/issues/show_bug.cgi?id=5556 [openoffice.org]
Re: (Score:2)
An advantage to most significant first, numeric, with leading zeros is that the dates will sort correctly.
Re: (Score:2)
But don't put the least significant unit in _the_middle_!!!
Except that the date of the anniversary of my birth or of my next dental appointment are difficult to think of as the least significant. : - )
Re: (Score:2)
They say 3rd of October, and don't think its awkward. It's just a question of what you are used to hearing, like so many of the differences between en-US and en-GB.
Re: (Score:2)
You have missed something. There's plenty of available information on time travel in the dark side of the web, or at least there will be in 6 months time.
Re: (Score:2)
You need to spend more time looking into the info available on the darknets then, clearly. You have missed something. The truth is out there.
Re: (Score:3)
this article is a bad idea. (Score:3, Interesting)
This kind of article will only make things worse for a future defendant trying to explain he wasn't coordinating with 'the deep' in the distribution of his movies from his computer to his Mythbuntu box.
Why include Deep Web? (Score:3, Interesting)
I thought the article might get to the point by the last page, but it was still talking about child protection and terrorism (in company databases???) I had wondered whether this confusion was down to an incautious academic, but the doesn't seem to suggest it: http://ai.arizona.edu/research/terror/ [arizona.edu]
About Privacy (Score:5, Insightful)
The deep web and its 'darknets' are a new battleground for those who want to uphold the right to privacy online, and those who feel that rights need to be sacrificed for the safety of society
Corporations, wealthy individuals and people in power keep their right to privacy. That's not good for the "safety of society". See the ACTA negotiation. Most of the calls about the future of society are made in a non transparent way, by corporations, the psychopaths that run them and corrupt politicians. If I don't keep my right to privacy ( and this looks like a lost cause) then I want them to lose it as well. I want a full public database with detailed information about every dollar owned and every move made by politicians and members of a corporation board. I want every government contract to be published on an easily searchable database. I want all meetings between corporation boards and/or government officials transcribed and published on another publicly search able database.
Re: (Score:2)
Re: (Score:3, Insightful)
I agree with that, but the message people in power is throwing around is that privacy is a lost cause... for us.
If they realized that in that case we are not willing to allow them to keep their own privacy and anonymity, they may change their minds.
In fact, it may be a good idea to keep individuals privacy, but making institutions and corporations transparent.
Re: (Score:2)
Re: (Score:3, Insightful)
I want a full public database with detailed information about every dollar owned and every move made by politicians and members of a corporation board. I want every government contract to be published on an easily searchable database. I want all meetings between corporation boards and/or government officials transcribed and published on another publicly search able database.
Before you ask for more intrusion of gov't into corporate affairs, step back and realize most corporations are small businesses owned b
That's computer-science speak? (Score:4, Funny)
> In computer-science speak, it refers to those portions of the web that, for whatever reason, have been invisible to conventional search engines such as Google."
Spare me your mumbo jumbo, doctor!
In other news... (Score:4, Insightful)
What is the problem? (Score:5, Insightful)
If child pornography is being freely distributed amongst anonymous networks of paranoid people, what is the problem?
The vast majority of people who use onion routing are very cautious people, so very few will be stupid enough to leave a trail which could identify them (such as a payment) as doing anything which is seriously controversial or illegal. It would be absurd to suggest that anybody is going to profit from producing child pornography and distributing it through anonymous networks.
If somebody produced child pornography as a "hobby" (instead of for profit, which would result in a swift arrest anyway), it's pretty obvious that the producer would produce the pornography for themself regardless of whether they distributed it. So again, anonymous networks are not contributing to a problem, nor is the alleged availability of child pornography.
The majority of perpetrators of child sexual abuse are the parents of the child. If people genuinely wanted to stop child abuse, they would focus on protecting children from abusive parents. Instead, politicians and police chiefs tend to focus on matters which score politicial points and win votes; parents are not an acceptable target because they constitute a major component of the electorate. Claming to fight child pornography is much easier for politicians and police chiefs, as they will not lose significant support and they can easily claim a victory without any risk of being exposed as liars; after all, who is going to check the evidence?
Re: (Score:2)
This is a brilliant post, wish I had the mod points.
Re:What is the problem? (Score:5, Insightful)
The problem is that it offends people who love reading about these stories. Those who are not offended enough to really care will still go along with those who do for fear of being targeted. We live in a democracy, ergo the will of the vocal minority will make it illegal. Moreover, this same will leads to draconian restrictions and state surveillance of the internet and indeed the general population.
The "problem" here is less the child pornographers than it is the people who go into irrational emotional meltdowns whenever someone mentions the password("pedophiles") and who then proceed in their hysteria to tear down the great society that has been built over the last 200 years. Child pornography is at worst an unpleasant nuisance. These crusaders on the other hand are a direct and immediate threat to our way of life--or at least what our way of life used to be.
Re: (Score:1, Insightful)
"If child pornography is being freely distributed amongst anonymous networks of paranoid people, what is the problem?"
Because children are abused during the making of child pornography. You really don't understand this?
"It would be absurd to suggest that anybody is going to profit from producing child pornography and distributing it through anonymous networks."
I bet it still happens. Child porn is highly illegal and as a result, it probably has a high value on the black market. You could easily make paym
Re: (Score:2)
"Children are sexually and emotionally abused. Would you say that it's okay to molest children as long as its just a "hobby?""
Neither I nor any of the respondents suggested that children being abused is "okay"; that would be a ridiculous assertion. My argument was that distribution and availability of child pornography on anonymous networks is extremely unlikely to encourage people to produce it.
I also question your assumption that "child pornography" is necessarily a depiction of children being molested. I
Re: (Score:2)
Theory 2 overlooks the possiblility that the "profit" comes not in the form of money, but that if two child molesters exchange materials, each gets "exciting, new" child pornography in exchange for easy and cheap to make copies of stuff they already have on hand that may have, from their point of view, have lost some of its "inspirational power".
Some years ago... (Score:4, Informative)
here is an analogy of the danger to consider (Score:1, Offtopic)
Remember, this is an analogy using an unrelated but example with hindsight already established, unlike what this article is about.
The idea of the stock market was to allow people in invest in companies they believed in, sort of a put your money where you mouth is vote.
But due to the higher and higher levels of abstraction and manipulation at these higher levels of abstraction, putting your money in companies you believed in, is no longer what moves the market. Instead there have been some very bad things re
Re: (Score:2)
missing link in above: The Trillion Dollar Bet [pbs.org]
NOYBnets (Score:1)
It is like going to garage sales. You can expect to see a ton of the same old garbage, but usually someone has 'stored' something they don't care about but you really want. Occassionally there are priceless buried treasures that should be shared with the whole world. If marketers could regularly go through them all, it would provide unparalled insight into what we buy and store. Unfortunately, (or fortunately depending on your point of view) everyone doesn't get to ramble through our garage unless we give t
BBS, IRC (Score:3, Interesting)
If you still remember how to use those BBSs, the IRCs, then you can get back into the 'darknet' world I guess, though today it is also about appearing/disappering websites and botnets. Botnets like Zeus got that whole 'FreeNet' idea and have their own implementation, only it's not exactly free.
13% of the web explored by search engines? (Score:1)
Re: (Score:2)
and how exactly do they know the number of non-indexed pages? it's not as if google could do a "SELECT COUNT(*) FROM sites WHERE indexed=FALSE" on their database, because the basic idea is that these sites are not in any extern databases.
Re: (Score:2)
not really (Score:1)
Bottom line (Score:2)
So the bottom line is there is more private data then public data floating around.
Go figure. *snooze*
Slow news day? (Score:1)
Must be that time of year again (Score:1)
Every 12, 16 months or so I go "Oh yeah, Freenet exists...I should check it out again." But every time, I can't answer the question: Is there anything actually HERE?
Freenet: Is there actually anything worthwhile on it?
eh (Score:1)
Darkweb (Score:1)
I thought Darknets were private and encrypted vpn based file sharing systems. They work based on invites and trust.
I think the author meant Darkweb
Undernet (Score:2)
So this is the new term for IPv6?
Yes, believe it or not Google cannot do everything (Score:1)
Re: (Score:2)
Google is actually a good way to find scientific papers, their search engine gives better results than the academic databases offer most of the time... the problem is you usually have to pay. The best thing to do alot of the time is google what you want then log in through your schools VPN or go their library to get the paper without paying.
Hiding is not always sinister or suspicious (Score:1)
"The deep web isn't half as strange or sinister as it sounds. In computer-science speak, it refers to those portions of the web that, for whatever reason, have been invisible to conventional search engines such as Google."
In reality, we might be better off if more of the Web were "dark" in this sense. Google (and the other major search sites) are fairly good at indexing text in English (and other human languages). But there's a lot of data online in those "sprawling databases" that are not encoded in huma
"Dark Side of the Web" moniker already taken... (Score:1)
options (Score:1)
I see you never tried the -robot-follow option in google...
Fravia's Research (Score:2)
Fravia's Research:
http://www.searchlores.org/ [searchlores.org]
http://www.searchlores.org/indexo.htm [searchlores.org]
He was a great guy - it's his "fan club" I couldn't stand.
Watch yourself on the links to some of the "discussion boards".
There's some really good knowledge in what he left us.
One print page... (Score:2)
http://www.pcpro.co.uk/features/356254/the-dark-side-of-the-web/print [pcpro.co.uk]
Re: (Score:1)
no quite a bit of it is lit by fiber optics.
Re: (Score:1)
Re: (Score:2)
I know a place (Score:2)
where all the junkies go.
Where everything is fast
And everyone gets slow.