All of Gopherspace Available For Download 200
An anonymous reader writes "Cory Doctorow tells us that '[i]n 2007, John Goerzen scraped every gopher site he could find (gopher was a menu-driven text-only precursor to the Web; I got my first online gig programming gopher sites). He saved 780,000 documents, totalling 40GB. Today, most of this is offline, so he's making the entire archive available as a .torrent file; the compressed data is only 15GB. Wanna host the entire history of a medium? Here's your chance!' Get yourself a piece of pre-Internet history (torrent)." Update: 04/30 00:16 GMT by T: As several readers have pointed out below, our anonymous friend probably meant to say "pre-Web," rather than "pre-Internet."
Shame on Slashdot (Score:5, Insightful)
Here's your chance!' Get yourself a piece of pre-Internet history
I think Jon Postel is rolling in his grave right now.
Re:Shame on Slashdot (Score:5, Informative)
Beat me to it. The summary should read "Get yourself a piece of pre-world wide web history," since gopher came AFTER the birth of the internet (1981) but before the widespread usage of the web (circa 1993).
Re: (Score:3, Funny)
since gopher came AFTER the birth of the internet (1981) but before the widespread usage of the web (circa 1993).
I hope you don't mean the birth of the Internet was in 1981. Or maybe you typoed 1991 (when Wikipedia says gopher was released)? I thought gopher was actually a bit older than that.
I just wish people would stop holding onto FTP like they were Charlton Heston.
Re: (Score:3, Informative)
Um, the generally accepted start of the Internet is by activities surrounding the start of ARPANET in the late 1960s. ARPA in its name still lives on as part of reverse DNS entries. Some people say it started in 1967, some say 1969, either way, it was much earlier than 1981 and there are a lot more protocols that are part of what we call "the Internet" than just TCP/IP, although of course not all of it is routed globally. Check your /etc/protocols file sometime, the first line says Internet (IP) protocols
Re: (Score:2)
Lost cause trying to get even IT people to know about networks and how / why they came - nobody (well, most) knows or even want's to know the history today. It's so funny when the youngsters, maybe had their first experience in 80's or maybe even coded "assembler" in 70's (not really but that's another story), talk about network, protocols, multi tasking / threading / programming / etc, whatever. The knowledge and education is (very!) bad today, has gone down (IMHO) since 70's. Ask about SDLC today - you t
Re: (Score:2)
Maybe he just got his words in the wrong order: a piece of Internet prehistory.
Re:Shame on Slashdot (Score:4, Insightful)
I have an EE degree. What's a good 2nd degree? CMP ENG or Comp Sci? I want to be eligible to apply for more jobs.
You are eligible to apply for all jobs now. The trick is actually getting one.
Second degrees are a net loss in the market. One degree means you are of at least average intelligence and can show up on time when it counts. Two degrees means pretty much the same thing.
Re: (Score:2)
Re: (Score:3, Insightful)
Seriously. The story submitter was anonymous (probably a good thing!) but I'm really shocked that any Slashdot editor could let that line go through without comment. And spare me the "you must be new here" line -- I know perfectly well that /. editing standards can get pretty sloppy, but this is particularly egregious. Calling Gopher "pre-internet" is the kind of crap I'd expect on a mainstream news site, not from "News for Nerds."
Re: (Score:3, Insightful)
Really? You are shocked that a slashdot editor doesn't check and correct the stories he posts? You must be new here.
Re:Shame on Slashdot (Score:4, Insightful)
Really? You think that someone asking if you're new here means they think you're new here?
Are you new here?
Re: (Score:3, Informative)
I've been around a while, and I can't think of any time a Slashdot editor fact-checked, spell-checked, or proofread a submission. Look at it, they put the entire thing into a quote. That way they can just say they're quoting the submitter and that's what he said.
They might add the "UserXXX writes," part themselves, but a couple characters of perl could probably do that part just as well.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
The world wide web is all the websites accessible by using an Internet browser.
Fail
Re: (Score:3, Funny)
Where do the tubes come in? Are they buried under the superhighway?
Re: (Score:3, Informative)
This definition is probably looser than most, but here's a quick and dirty view:
The Web is a huge collection of interlinked documents addressable by URLs and served with HTTP. The Internet is the world-wide TCP/IP network over which the Web and many other services operate.
Re: (Score:3, Informative)
Actually, the Internet is the world-wide IP network. TCP is just one of many protocols that are used to transmit information across it.
Internet vs Web (Score:2)
Someone. Please, please, PLEASE enlighten me on the difference between web and Internet. Yeah, I know they're different and it's a matter of protocols, but I've heard this for years and honestly still don't quite get it.
I know someone else has just answered, but here goes:
The Internet is a global network of computers, or more precisely a global collection of interconnected networks that happen to use the Internet Protocol (the "IP" in "X over IP") to talk to each other.
The Web is a global collection of documents and various media files stored on web servers around the world.
The Web can also refer to the global collection of web servers which store these documents and media files.
In other words, the Web is part of the Inter
Re:Shame on Slashdot (Score:4, Informative)
Re: (Score:2)
Re: (Score:3, Informative)
History of the Internet from 1957 to present:
http://vimeo.com/2696386?pg=embed&sec=2696386 [vimeo.com]
Quite educational, even if you think you know all about it.
Re: (Score:3, Informative)
Having just watched it again, it may not fully answer your question. With what you learned from the video in mind, the OSI model [wikipedia.org] is the layers the video talked about. There are seven layers altogether, with the lowermost layer being the physical hardware everything runs on, followed by the network connecting the hardware, then how data is passed over the network, and so on until you get to the application layer. You've heard of TCP/IP? That's TCP (layer 4) running on top of an IP (layer 3) network. ICM
Re: (Score:2)
Haven't read all the replies but Internet isn't just TCP/IP !!!! The many exclamation marks because that's a common error - Internet isn't even an pure IP network, much what happens in there is on Ethernet level or even using other lower level protocols, still! Yes - I know, it's defined in Wiki as a TCP/IP network but, after designing / developing / delivering a lot of systems using "Internet" and sometimes UDP, sometimes other IP based protocols, sometimes working just on Ethernet level / link layer / eve
Far cry from "all of gopherspace" (Score:5, Insightful)
Re: (Score:2)
Maybe not.
I think the growth of HD space since 1997 might ahve you thinking that 40GB isn't much. . . today it wouldn't be, in 1997? That was huge.
I don't have any figures, and wuld welcome actual numbers, but is does seem like a 'raising the bar' issue.
Re: (Score:2)
I think the growth of HD space since 1997 might have you thinking that they are the only way to store large amounts of data. You could have stored 40GB with a handful of tapes in 1997.
Re: (Score:2)
Re: (Score:3, Informative)
"Do you have any facts or figures underpinning your statements ?"
That would indeed be interesting, but GP makes a reasonable assumption, akin to "There were more horse carriages out and about before the automobile." No?
Re:Far cry from "all of gopherspace" (Score:5, Interesting)
On a regular basis? Yes. Than exist in barns today for special occasions or limited use, possibly not.
It has been indicated that more people know how to properly shoe a horse today than in the late 1800's. Lower percentage of the population, and not something they do every-day, but a larger total number of people.
I wouldn't be surprised if the total number of documents on Gopher continued to climb despite the percentage of content on Gopher decreasing rapidly. The cost to host has rapidly decreased and amount of content in general has increased significantly that the total number of items could still be higher today than in the 90's.
Re: (Score:2)
Good point. A similarly unintuitive fact is that there are more slaves today than ever before in history.
Re: (Score:2)
Re:Far cry from "all of gopherspace" (Score:5, Informative)
Yes.
In 1997 we had a 100Gb disk array holding the research data from our lab, all of which was available via gopher (and ftp, and the web). We moved to a 200Gb array shortly after, and then a 400Gb after that. And then 3Tb, around 2008.
Sometime around 2007 or 2008 the SunOS system that ran the gopher server died permanently and was replaced by a virtual linux server without gopher. Even without that server, I found not long ago that I was still creating .cap files -- which were gopher, as I recall, but maybe archie.
Quantitatively, online currently I have more than 15Gb of data for just 1997, all of which was gophered at the time. In 1998, another 18Gb.
So, I would say, had the gopher scraping been done in 1997 instead of 2007, the result would have been a lot more data. In fact, a few months earlier in 2007 and it might have BEEN a lot bigger.
Re:Far cry from "all of gopherspace" (Score:4, Funny)
Because as most users of the internet he wasnt accurate about the unit.
From the context one can assume (without that big a risk of error) that he is indeed speaking about gigabyte, and not gigabit.
Buuut, anal responses are more important than content. We know this :-p
Re:Far cry from "all of gopherspace" (Score:4, Funny)
Stop making fun of him, he's not Turing tested yet. In a few years, he'll start noticing some changes and then he'll grow up to be a big boy AI that can interact with the rest of us.
Re: (Score:2)
Ah, that explains it ;)
The Ultimate Lesson in Open Source and Standards (Score:5, Interesting)
That move by the U of MN is a great lesson in how licensing can kill innovation. Standards should always be open and guaranteed open.
Re: (Score:2)
If Gopher might had became the Internets: Imagine all those VT-terminals that wouldn't be in landfills!
And we'd be working on Gopher-5, the Flash-killer!
Re: (Score:3, Funny)
I prefer to believe that Gopher failed because the world wasn't ready for the awe-inspiring virtual reality experience that was TurboGopher VR.
Re: (Score:2)
As a gopher user in the early 90's, my impression was that the web behaved like gopher, but with a working mouse and actual visuals. Gopher was essentially a way of networking old BBS's together. The web was like that too, but with actual visuals, real page layout, and ugly backgrounds.
I seriously doubt Gopher would have caught on to the same degree, any more than command-line interfaces being prevented from reaching their full potential by crafty GUI licensing. Gopher just didn't go far enough for the a
Re: (Score:3, Interesting)
Re: (Score:3, Informative)
That's more the fault of the clients than the protocol. There's no reason you can't serve hypertext documents over gopher, and no reason a gopher client couldn't display graphics.
Re:The Ultimate Lesson in Open Source and Standard (Score:4, Informative)
Re: (Score:3, Informative)
To be somewhat more accurate, it's not "now" called hypertext: it was called hypertext before gopher even existed. Gopher was first released in 1991, while Ted Nelson coined "hypertext" in 1965, and there were dozens of implementations before the WWW (the most popular outside academia was probably Apple's HyperCard, released in 1987).
Re:The Ultimate Lesson in Open Source and Standard (Score:5, Insightful)
There's no markup for hypertext in HTTP either.
Re:The Ultimate Lesson in Open Source and Standard (Score:5, Informative)
The original pre-RFC HTTP states that a response is an HTML message [w3.org].
Pre-internet history? (Score:5, Insightful)
The web is NOT the internet. (Though sadly it essentially has become so, nowadays.)
Re: (Score:2)
Re: (Score:3, Funny)
They teach us the difference and why it no longer matters;P
Re: (Score:3)
Re: (Score:2)
Re: (Score:2, Insightful)
Because the internet is not restricted to what you can do on a handful of ports with little more than a handful of protocols. That so many technical professionals limit themselves to the "web" tends to restrict creativity.
Re: (Score:2)
I suppose that's fair enough. Still, when I think about culture and the like, the web is all that really comes to mind. Even things that may happen in other protocols still end up on the web if they are more fleeting than a moment or public than a few people.
If someone told me to archive the entire internet, I'd consider the www both necessary and sufficient.
Re: (Score:2)
Interesting claims, given that things like World of Warcraft, instant messenger, voip, doom, and even BitTorrent don't run over "the web" by these semantics. BitTorrent is facilitated by torrent files most frequently downloaded over the web... A lot of people clock a lot of time on the internet in ways that is not "the web."
Really is shocking to me to see so many people even on a site like slashdot clearly not understand the difference, or try to minimize it. Then again, I guess colloquially the web is tcp/
Re: (Score:2)
To put some perspective on it for non-nerds / Slashdot editors.
Port 80 and 443 are the Web ... the other 65534 ports are the Internet, where everything interesting happens.
Re: (Score:2)
There has been a "webalization" process that took place, even encapsulating database connection in HTTP (special type of database driver) and all sorts of other protocols. Further more, a tendency to use port 80 has also prevailed even when not using HTTP. Last time I checked, Skype uses port 80 by default to listen on the local machine.
One of the logical explanation I see which might have caused this: It all started to occur when corporations started to tighten their security, installing firewalls and star
Re: (Score:2)
Maybe, but isn't the 'web' basically like a turing machine that can emulate everything else if need be?
Re: (Score:3, Insightful)
The web is NOT the internet. (Though sadly it essentially has become so, nowadays.)
Hardly. Most traffic is bittorrent and email (mostly spam).
Gopher isn't dead. (Score:4, Interesting)
http://www.tekeeze.com/geeky/7-fun-sites-you-can-only-find-on-the-gopher-internet/
Includes things like Twitpher (which might not be working right now) Twitter for Gopher.
Firefox (others?) supports gopher://
Re: (Score:2)
Sigh...
Fortune goes back much farther than Linux.
It's not on my Mac OS X machine, Wikipedia's page says:
Gopher (Score:5, Funny)
So does this mean we're getting 6 more weeks of winter or not?
Re: (Score:2)
LOL. That would be a groundhog.
Re:Gopher (Score:5, Insightful)
No, just another ten years of November.
Re:Gopher (Score:5, Informative)
No, just another ten years of November.
I believe you mean September. [wikipedia.org]
Re: (Score:2)
D'oh. Well, it was almost a good joke.
Maybe I should get outside more. Like most slashdotters, I can't actual tell the difference between the months!
Re: (Score:2)
You sir, are awesome.
Re: (Score:2)
What a terrible Marketting line (Score:2)
I can look around the room and find hundreds of pieces of pre-internet history.
Is there any other point you can try and sell me on?
Re: (Score:3, Funny)
new furniture?
Not pre-internet history (Score:2)
Pre WWW history sure but GOPHER was a protocol for use on the internet.
1996 bookmarks with gophers (Score:2)
"gopher://cwis.usc.edu/11/Other_Gophers_and_Information_Resources/Gophers_by_Subject/Gopher_Jewels/Istuff/fun/fun" (a list of cool resources...)
"gopher://gopher.lysator.liu.se:70/11/lysator-Science_Fiction_Archive" (I think this is where I got the Blake's 7 scrips from)
"gopher://www.library.ucsb.edu:70/11/journals/usenet" (not sure what this was about)
"gopher://wiretap.spies.com:70/11/Library/Fringe/U
I miss Gopher (Score:2)
As much as I love the more advanced technology of the modern Internet, there's a soft spot in my heart for Gopher and the Internet circa 1993. Gopher is the way I found the first MUDs I ever played, how I found and was granted access (via telnet) to a Free-net (freenet.calgary.ab.ca) which gave me my own email address and access to newsgroups. Then came the Web, and Yahoo still looked a bit like a Gopher site, and I continued to use Gopher through my provider's PPP connection until it became a niche thing.
Re: (Score:2)
Sweet! Vindication for Pointy-Haired Boss (Score:2)
Now you really CAN download an internet, in the loosest definition. :D
(But it still won't fit on a floppy disk.)
I wonder if someday... (Score:2)
...futureslashdot.future users will be futuretorrenting the history of the www when it gives way to the next iteration.
Oh and we'll all be plated in gold, because that's what happens in the future.
Index anyone? (Score:3, Interesting)
Is there a plaintext index of URLs this archive includes anywhere? I'm connected via 3G and pulling a 15gig torrent isn't feasible. I'd love to wander thru some of my personal archived bookmark lists and such just to see if any of them wound up being preserved.
Compression routine (Score:2)
Since it's all text, I'm surprised that 40GB only compressed down to 15GB. I wonder how small it would be if he used lrzip [kolivas.org] with max settings instead... I didn't see mention of which type of compression was used in the short article.
I am (Score:2)
I am a piece of pre-Internet history, you insensitive clod!
Gopher lives! (Score:5, Informative)
What do you mean, "was"? Gopher still works fine. There are dozens of servers out there. See quux.org [quux.org] or just install your Linux distribution's gopher package and fire it up.
Re:Gopher lives! (Score:4, Funny)
Dozens of servers ought to be enough for anybody.
Re: (Score:2)
Are any of them secure?
I've been wanting to run a gopher server, for nostalgia, I'd just rather not get pwnt if possible.
Something that will run on openbsd would be groovy.
(I can't check the site because the router here at work drops everything that isn't on 80 or 443).
The real surprise (Score:2)
Is that there were still gopher sites in 2007! I RTFA expecting the real date to be 1997, but apparently not. How come the sites survived until 2007 but not 2010?
Re: (Score:2)
Re: (Score:3, Informative)
To a lot of people, WWW=Internet. Us old greybeards who remember when the Internet was telnet, FTP, e-mail and Usenet know better.
Re: (Score:2)
Wasn't the pre-WWW Usenet technically a separate network (like Fidonet) from the internet?
Re: (Score:3, Informative)
Re:Wrong (Score:4, Informative)
So, yes, Usenet preceded the Internet in the sense that it did not rely in IP, though both generally evolved around the same time.
But, there was a rather vibrant pre-WWW internet where the protocols of choice were smtp (mail), ftp (file transfer), and gopher and archie for repositories of places to find stuff. News could be carried via nntp (net news transfer protocol).
What some may not know was that sendmail could work over transiently connected points as well, rather like usenet. Anyone still remember bang path notation? One would address mail using the sequence of hosts required to get it from one's own to the destination, using names understood by each successive host in the sequence. One of the reasons sendmail configuration files were so horrendous was to permit relaying between networks using different host naming conventions.
Re: (Score:3, Informative)
Ahhh the good old days.
You post a question on rec.arts.tv like, "When does the new season of TNG start?", wait for the midnight syncing between your local BBS and the rest of the nation, and then you come back tomorrow morning to learn the answer. If you're lucky. Sometimes you had to wait 2 days for a reply.
Re: (Score:2)
Then, in the Web's infancy, 2400bps data connections were almost bearable for browsing. I dunno if the Web "grew up" as much as it "grew fat".
Re: (Score:2)
Re: (Score:2)
Don't forget UUCP, via 300 baud modem, it could couple multiple nodes similar to Fido.
Re: (Score:2, Informative)
Re: (Score:3, Insightful)
>>>I always accessed my Usenet groups via the Internet anyway
I used a 1 kbit/s modem (yes very slow). My messages are still archived on google groups, and I wish there was a way to erase them, because it's somewhat embarrassing to read posts from your teenage self 25 years ago (especially the typos). ;-)
Re:Wrong (Score:5, Interesting)
A funny thing happened to me a while back.
I was trying to build Nethack for a server, and it was failing linking on some missing curses library. So I did a google search to try to find out which library I was missing so I could find which -dev package I needed to install to get this library.
The first Google search result was... ...a post by *me* asking *exactly* the same question ("Which lib do I need") almost 15 years earlier on one of the linux newsgroups!
Re: (Score:3, Funny)
I remember when fingering the gopher was totally normal.
Re:Interesting (Score:4, Interesting)
I thought Gopher was okay, but not near as exciting as my first exposure to Amiga Mosaic web browser. After all, it had teh 4000-color pron. ;-) Plus exciting sites like this one: http://web.archive.org/web/19961114151757/http://scifi.com/ [archive.org] - I mean how cool is that? It's animated and colorful. :-)
Aside -
Looking at that schedule reminds me how cool Sci-Fi Channel used to be. Weekend Anime. Inside Space reports. Sci-Fi Trader. Sci-Fi Buzz. FTL Newsfeed. It was like a geek paradise for fandom. Today's channel is more akin to watching the TNT channel - ordinary and nothing special.
Re:Interesting (Score:4, Interesting)
Ever since the RSS vs ATOM war peaked (and fizzled) i've been waiting for a re-gopherisation of the internet, where files, videos, music, audio and pictures are all linked and indexxed by interconnected RSS feeds that dont include all the crud you have to wade through in web pages to get anywhere. Something akin to MRSS with png thumbnails and optional links to "buy the dvd box set now" where you could create your own Channels (feeds full of links to shows directly, or other feeds) and then browsable from your telly directly, i think i'm rambling
Re:Anyone remember ARCHIE servers? (Score:4, Funny)
Sure, back when people knew the IPs of their local archie and simtel archives.
Those where the days...
Re: (Score:2)
the dark, dark days.
I don't miss them at all.
Re: (Score:2, Insightful)
Does anyone give a frak?