Internet Archive Challenges Google

richards1052 writes "The Internet Archive, whose main claim to fame is the Wayback Machine, designed to archive the internet's web history, has created a new project: the Open Content Alliance. It's purpose is to open the nation's library collections to universal web search. A number of major library systems, including the Boston Public Library and Smithsonian, have refused to sign up with competing ventures by Microsoft and Google because they do not provide for universal access to digitized books. These commercial ventures prohibit books being accessed by competing search engines. So far, 80 libraries and research institutions have signed on with Open Content Alliance. They must pay for the scanning of their books while Google and Microsoft offset that cost for their participating institutions."

Society lost

[packetmon (977047)]

I believe I've commented on something like this before. Might be a good idea to archive the books lest somewhere in the future we re-live something like the Spanish Inquisition where important literature was lost. Its also making this society a bunch of couch potatoes. What ever happened to walking into a quiet library, the smell of stale books, looking around at people. Its slowly being replaced by reading books online and hitting ctrl-w to close annoying popups while you read. Currently I have about 30+ Cisco (CCIE/NP/IP/etc) books and each come with their PDF's. At first I thought, neat I can read them on my laptop... Nowadays I find its easy to just open the book, nothing like butchering my books up with highlighters... This world is coming to one where companies will be fighting to keep us locked in our houses. Call me a troll, just speculation

Re:Society lost

[CRCulver (715279)]

I enjoy very much sitting in a university library with piles of books around me to work through. However, as someone who spends most of the year traveling, usually lugging around a bag or two of hardbound specialist literature in addition to an already heavy backpack, the more I can put on my notebook the better. PDFs don't weigh anything.

Re:Society lost

[Actually, I do RTFA (1058596)]

PDFs don't weigh anything.

Not true, ones have slightly more mass than zeros, so they weight a tad more than zeroing out that section of your drive!

Note, my source is Dilbert. But it was right about smaller font sizes leading to smaller .doc sizes as well.

Re:

[luder (923306)]

What ever happened to walking into a quiet library, the smell of stale books, looking around at people. Its slowly being replaced by reading books online and hitting ctrl-w to close annoying popups while you read.

Funny, I bet someone said the same thing when the average person began to have enough purchasing power to buy individual copies of books...

Re:Society lost

[RsG (809189)]

Might be a good idea to archive the books lest somewhere in the future we re-live something like the Spanish Inquisition where important literature was lost.

Oh really, does that seem likely?

I don't expect the Spanish Inquisition...

Nobody...

[jefu (53450)]

Nobody expects the Spanish Inquisition.....

(Can't believe I'm the first one to respond with that. Of course by now I'm probably not. )

Nobody...

[benhocking (724439)]

Nobody expects the Spanish Inquisition!

(I couldn't bear to leave you hanging.)

Re:Society lost

[hasbeard (982620)]

It seems to me that in today's climate, religious conservatives (You would probably consider me one) are also threatened with censorship. Some Christian leaders have pointed out the possibility that "hate speech" laws could be utilized to limit the ability of Christians to free practice their faith and voice their beliefs about homosexuality and other matters. It isn't that difficult to posit a future in which the Bible could be banned as "hate speech" and Christians jailed, denied employment, or are otherwise discriminated against because of their beliefs. Secular, totalitarian states have also banned and burned books. There are Christians enduring persecution in many places in the world even as I type this. And if you want to ask someone about government censorship, you may want to talk to the Chinese people where Christians and others are regularly censored by their non-religious government.

Re:

[ConceptJunkie (24823)]

I guess you didn't get the memo, but Christians don't have the same rights as everyone else. And God forbid you're one of the many Christians who actually embraces and insists on reason. People don't know what to do with you so they figure you're an evil influence and should be shut out of society. Meanwhile, PhD's are being handed out for "discoveries" that were common sense just a few generations ago. "Birth control leads to the objectification of women." "Divorce hurts kids." "Not having a father h

Re:

[Achromatic1978 (916097)]

Birth control leads to the objectification of women

Show me this study, I'm curious. Or was it just one thrown in there that seemed like it might fit, and just so happens to further the 'birth control is evil' agenda?

Re:

[Joe Tie. (567096)]

It was also common sense that killing a woman was a good test of whether or not she was a witch, or that it was impossible that a drop of pond water had little invisible animals swimming around in it. That's the whole point of science, testing common sense because humans are so bad at interpreting the world around us.

It isn't that difficult...

[benhocking (724439)]

It isn't that difficult to posit a future in which the Bible could be banned as "hate speech" and Christians jailed, denied employment, or are otherwise discriminated against because of their beliefs.

Yeah, conspiracy theories are usually quite easy to posit. That doesn't mean they have a bit of merit. Get over yourself—you're the majority, and you're not being persecuted in this country. (Yes, there are Christians being persecuted in countries where they're not the majority, and it is genuinely a tr

Re:Society lost

[triffid_98 (899609)]

It's not just the right you need be concerned with, the left is equally likely to purge "inappropriate content", they just make up different reasons for it. It espouses racism (Huckleberry Finn), it's pornographic (The Scarlet Letter), it's dangerous (The Golden Book of Chemistry Experiments). Big brother or nanny government, it all boils down to the same thing. 2008 is coming people, can we please can all of these assholes before I end up having to describe this as double plus good?

(read Christian religious right) have "won" and science as it now stands is banned. Books could easily be "misplaced" by these folks, and the library of congress purged of "inappropriate content".

The left is against pornography?

[benhocking (724439)]

It's not just the right you need be concerned with, the left is equally likely to purge "inappropriate content", they just make up different reasons for it. It espouses racism (Huckleberry Finn), it's pornographic (The Scarlet Letter), it's dangerous (The Golden Book of Chemistry Experiments).

OK, so you have a point with Huckleberry Finn (although I'd argue there are far fewer on the left trying to censor Huck Finn than on the right trying to censor Harry Potter). However, in what world is the "left" the

Re:

[absorbr (995554)]

But there is advantage in being able to search those technical manuals. Leave the couch potato business to me, there's already enough reason (health) to get outside, soak up sun, and exercise.

Re:Society lost

[beadfulthings (975812)]

I don't think you're a troll, and until recently I would have agreed with every point in your post. Now I'm not so sure. Anything that preserves the written word from future loss, and makes books and literature available to more people, is a good thing. Freeing this activity from commercial restraints is a good idea; somehow the commercial route would make me think my choices might be somehow limited by what the commercial software "wanted" to show me.

It's the quiet library...dusty books part that has me a bit concerned at the moment. My city recently built a brand-new neighborhood "anchor" library within striking distance of my house. That eliminated the need for a tedious drive or bus ride downtown to the main library, the expensive necessity of finding scarce parking, and the tyranny of downtown "business hours." The branch is convenient, has good parking, and is open for people who have to work during the daytime. It's bright and attractive. Endowed with an outdoor "reading garden" for good weather, it will shortly boast an indoor coffee area. It has banks of computers to be allocated to either youngsters or adults, a pretty good periodicals section, and a pretty good reference section. It also manages to cater to our increasing Hispanic population. The kids' section, which takes up half of the main floor, appears to be excellent. There's a huge section of fairly current DVD films. Tables (and to my surprise) comfortable easy chairs are invitingly scattered throughout.

I guess the problem is that with all the beautiful amenities, there's actually not much room for books. To be fair, they are just starting out, and their holdings will certainly increase. I'm just not sure where they will locate a collection that's much larger. I would divide my reading into "work-related," "feel like I ought to read it," and "just for pleasure." It's not easy to find new or interesting titles in any of those areas. They're trying, but there's just not enough space.

I've resolved that, to an extent, with a nifty little Mac utility called "Library Books." By entering my online access information, plus the library's catalog system (iBistro/Sirsi), I suddenly have complete, convenient access to the entire city and state library system. I can browse, search, and reserve. I could do all of that simply by going to the library's main Website and logging in, but the utility does me little favors like alerting me to the arrival of reserved titles and putting a big, red star up on days when books are due. It's a convenience that makes online browsing just a bit easier.

I've thrown myself into the new anchor library, in tandem with the simplified online access of the Library Books utility. I've become an evangelist, of sorts, for requesting and reserving books. It's amazing the number of people who don't know you can do that. The sheer square footage required to hold a large collection isn't feasible for neighborhood libraries, and I feel as though this sort of system gives one the best of both worlds. No yellow markers, though . . .

Re:

[liquidpele (663430)]

Not just books! Anyone know of a place where research papers can be searched and downloaded for free? I was trying to find some yesterday, and I ended up having to log into my School's library system to search because I couldn't find even one free website with articles in their entirety!

Re:

[Metaphorically (841874)]

I don't have to do this myself but I have heard about this problem (and wrote a little about it [latenightpc.com]). Your school pays publishers for access to journals and you can't necessarily get to them from home. There is a plugin for Firefox called LibX [libx.org] that makes it easier to search what your school's library has available. It has to be customized somehow to make it work with the school's system. The University of Windsor has customized LibX as Foxy Leddy [uwindsor.ca] (Leddy is the school library). Maybe your school has something s

Re:

[liquidpele (663430)]

Thanks for the plugin link, but I still wonder why there are no free sources online... I mean, it's not that I couldn't find the same sources online for free that I could get through my school's library, it's that I couldn't find ANY sources online... not one! And I know for a fact that many research papers are distributed under creative commons. Just seems really strange to me that such a system doesn't exist yet, especially for something like research papers, for which the internet is almost perfectly s

Re:

[Steauengeglase (512315)]

Why not go to the library?

Easy, because many public libraries are becoming nothing more than places where people go to check their e-mail. Of course a nice university library is something completely different, but the small town public library is pretty much giving up on those bulky paper things. Why buy new books when you can offer poor people a place to get on-line (and get a bunch of grants)?

My local library only buys 10% of the books that it bought 8 years ago.

Digital has its advantages

[JCSoRocks (1142053)]

I agree that a nice, hard bound book is, at the moment, more pleasant to read. However, technologies such as e-Ink and others that allow you to read something digitally without the eye-strain of using a back lit monitor are catching on. I think a few factors make digital copies more advantageous - cost of duplication, storage, protection from damage, searchability.

Storage: I just moved, and I moved three bookcases full of books. That sucked. If those were all digital, I'd have hauled my computer from A t

Re:

[mikey_boy (125590)]

I think this is also about what you are used to, and how you learn to interact with the written word and research in general. As today's younger generations grow up being more used to everything being available on line, they will demand content to be available online and be able to work with it as effectively (if not more so given storage, cross referencing, dynamic searching etc).

I'm with you, I much prefer reading stuff on paper. I'm just not convinced that it will hold true in years to come.

I don't see

Project Gutenberg...

[Sherloqq (577391)]

... but on a much larger scale?

Relevant Link

[Chapter80 (926879)]

In case you missed this discussion back on October 2, [slashdot.org] Carnegie Mellon has a service which helps to better digitize these books. It's called Recaptcha, [recaptcha.net] and it uses otherwise wasted human cycles to convert text that was hard for computers to OCR.

Re:

[oliverthered (187439)]

Yes, what's your problem?

Re:

[eldepeche (854916)]

Maybe they'll move toward an ad-supported revenue model. Did you know Harry Potter wears Reeboks?

Re:

[SnarfQuest (469614)]

Why not hear it from several authors, from one publisher:

http://www.baen.com/library/ [baen.com]

Several authors use older releases as bait to tempt you to buy more recent books.

Project Gutenberg is concerned mostly with old, out of copyright books, author usually long dead. Many of these books would be unavailable otherwise.

A Better Link

[value_added (719364)]

The Libraries Shun Deals to Place Books on Web [nytimes.com] story in The New York Times covers the subject fairly well.

Way to go!

[Voltageaav (798022)]

Recognising the restrictions of the current iterations currently available and working to provide a better resource that most or all libraries will support. The free exchange of ideas (not entertainment for those of you who download your entire music libraries from Kazaa) will promote progress across the board.

Re:Way to go!

[kebes (861706)]

The free exchange of ideas (not entertainment for those of you who download your entire music libraries from Kazaa) will promote progress across the board.

It's interesting that you draw such a sharp difference between information and entertainment. I agree that there are differences between content intended to transfer knowledge, and content intended to amuse... but certainly there are strong similarities between the two.

In particular, if you accept that free exchange of ideas will promote intellectual progress, then is it not also reasonable to suggest that free exchange of artistic content will promote cultural progress? This is the central notion that Lawrence Lessig advocates: that overly restricting the distribution, reuse, and remixing of art and entertainment will inherently stifle culture. (Note that Lessig does not advocate wanton infringement nor abolition of copyright: merely a 'sane' balance between the rights of content creators and the rights of content users.)

With respect to this current initiative, it would appear that they intend to scan and index books that are oriented towards information, as well as those oriented towards entertainment. In my opinion, this is a good thing. There is much that people can learn and grow by having easier access to ideas, where "ideas" means both informational sources, as well as artistic sources.

Re:

[Nazlfrag (1035012)]

Indeed, it's the same reason you can find CDs and DVDs at libraries. GP is stuck in the past.

Re:

[mochan_s (536939)]

not entertainment for those of you who download your entire music libraries from Kazaa

It's exchange of ideas if you're a musician or a record producer.

Re:

[moderatorrater (1095745)]

Music is an exchange of ideas. Listen to a hymn, an ancient poem, dark side of the moon, stadium arcadium, etc. In fact, music is even better as exchanging ideas since it has more "bandwidth" than just the word: the tone and speed of the music. Songs are the oldest way of preserving and extending knowledge.

Total scan time?

[192939495969798999 (58312)]

is there any estimate on how long it will take all these projects combined to scan the entire existing catalog of books, accounting for expansion and development of better technologies to do the scanning, etc?

Scan My Books

[Doc Ruby (173196)]

I buy a lot of books. I've got probably 10,000 or so. I wish I could search through them. Some for reference, sometimes because I read something that sounds familiar that I want to find where I first read it. I'd also like to read them on my PC sometimes, or even on my phone like when I'm waiting for a while somewhere. And I'd like to copy/paste short passages from them into messages I send on the Internet.

If this project is really "open", can I have my own libarary scanned? How much does it cost? I own the rights to copy my own books for my own personal use. Does something make these other "official" libraries eligible to use their full rights to their content in a way that I cannot?

Re:

[Red Flayer (890720)]

If this project is really "open", can I have my own libarary scanned? How much does it cost?

There are plenty of document scanning services around, I know for low-volume (less than 100k pages) I've priced them out at about $0.08 per page at high resolution. I'm not sure what kind of surcharge you'd pay for them not being able to batch-feed (since you're talking about books).

Or, do what I did and rent a good scanner and pay a couple high school kids a fair wage to do it. Or, offer them piece rate of a nick

Re:

[Doc Ruby (173196)]

I buy a lot of books for $1-5 used. I might pay up to a dollar per 300pg book to be scanned, which would cost $10K for my whole library. At $0.25 a book, about $0.008 per page, I'd do it.

Even HS kids wouldn't work for $0.42:h. This project calls for a scanner with automated pageflipping. If this Open Library project doesn't have that, then I expect that no one does (yet). I'll wait.

Re:

[Red Flayer (890720)]

Even HS kids wouldn't work for $0.42:h

Depends on what country they are in... but the cost of shipping the books to China/India/etc would likely be prohibitive. This is where the scanning services can sometimes be more cost-effective.

Or, you could keep tape flags in your wallet (for me, much more useful than a condom :)) and mark pages for scanning later. I do this if I note a particularly interesting magazine article or passage from a book that I want to keep on file. It actually makes me feel like the

Re:

[vertinox (846076)]

Some for reference, sometimes because I read something that sounds familiar that I want to find where I first read it.

Yeah, I've got the same problem. Often times I just want to quote something rather than paraphrase a particular section I remember reading but it takes an hour or so because the included index at the end of the book was sorely lacking for a 900 page book.

Re:Scan My Books

[Doc Ruby (173196)]

Yeah, except it's already bad enough that Google has a log of every Web search I make. I don't need them having a log of every book I've read. They'd have a map of my mind. Next they'd require I upload my genome as the "encryption key". Then I'd face an army of imprinted clones, each backed by Google's budget.

Re:

[Doc Ruby (173196)]

No, I am sure I own the book. Nowhere does the publisher even claim that I bought a license or anything else less than ownership of the object and the rights inherent in real property to use that object. The uses I described are extremely well documented in law as fully protected by law.

Re:

[Doc Ruby (173196)]

So? I don't need a copyright exception to do what I described with my own books.

Wondering...

[140Mandak262Jamuna (970587)]

How many of these libraries think of Open Source and software platform choice? How many of them make sure their web sites are platform agnostic, equally accessible from all browsers? These people are willing to stand up and are willing to pay more to preserve their liberty. Hats off to them. But does this stand also extends to not having their documents locked down in a proprietary format encumbered with licenses and restrictions? I would very much like such ideas, being independent of vendors, would extend to Corporate America too.

False Dichotomy?

[internic (453511)]

There's a story about this in The New York Times [nytimes.com] this morning (free reg required). It begins:

Several major research libraries have rebuffed offers from Google and Microsoft to scan their books into computer databases, saying they are put off by restrictions these companies want to place on the new digital collections.

The research libraries, including a large consortium in the Boston area, are instead signing on with the Open Content Alliance, a nonprofit effort aimed at making their materials broadly available.

The opposition between the Open Content Alliance and Google may not be as much as it seems at first glance. From the NYT article:

Adam Smith, project management director of Google Book Search, noted that the company's deals with libraries were not exclusive. "We're excited that the O.C.A. has signed more libraries, and we hope they sign many more," Mr. Smith said.

It looks like Google will digitize the collection for free in exchange for exclusive rights to offering searches of the digital data, but the libraries don't give up rights to have someone else digitize the stuff again and do with it as they see fit. So they can go with Google for now if they want and the O.C.A. later as they have the resources. This seems pretty reasonable to me. I don't know what the deal Microsoft is offering looks like, but I wouldn't be surprised if it's much more restrictive.

Re:

[internic (453511)]

Hmm...so I apparently missed page 2 of the article. :-)

Microsoft is part of the O.C.A., except that, "A year after joining, Microsoft added a restriction that prohibits a book it has digitized from being included in commercial search engines other than Microsoft's." Sort of an "embrace and extend" approach?

Jokes aside, this does seem to be a bit more of an open approach than Google's.

Re:

[Frosty Piss (770223)]

Of course Google would say something like this. So would Microsoft. It's what's called "PR Spin".

"Of course we're happy our competiton is starting great new projects that compete with us in ways we are not able! Competition is good! Yadda, yadda, yadda..."

People, in less than 2 years, Google and Microsoft will be indistinguishable, both being IP whores (and I'm not just talking about network addresses...) driven by mercenary stockholders interested in nothing but cash, raping and pillaging the Interweb

Re:

[internic (453511)]

I agree that no matter what they'll try to spin it to minimize the conflict, but the important question here is what are the facts of the matter? It sounds as if the facts are that one is perfectly free to work with both Google and the O.C.A. (based on the quotes in the article from other parties besides Google). This seems like a fairly reasonable arrangement to me; you have the option to get things scanned for free by Google but with restrictions and/or for a fee by O.C.A. without those restrictions.

What about copyright?

[causality (777677)]

Which books are digitized anyway? With copyright being as ridiculous as it is (what is it, 50-100 years after the death of the author?), are we likely to see anything modern in such a collection? I would hope that libraries would have some sort of exemption from this, except that in this case it sounds like the data might be used for commercial searches. I also wonder if these will be regular PDFs or if there will be some sort of DRM on them. Can anyone more knowledgable weigh in on this?

Do we really have to scan them?

[140Mandak262Jamuna (970587)]

Most modern books are created in electronic form to begin with and are printed with high speed offset printers from files. Only older books have to be scanned.

Other countries

[Bromskloss (750445)]

I hope libraries in other countries will be included as well. Please start with Sweden. :-)