Follow Slashdot stories on Twitter


Forgot your password?
Google Businesses The Internet

Internet Archive Challenges Google 115

richards1052 writes "The Internet Archive, whose main claim to fame is the Wayback Machine, designed to archive the internet's web history, has created a new project: the Open Content Alliance. It's purpose is to open the nation's library collections to universal web search. A number of major library systems, including the Boston Public Library and Smithsonian, have refused to sign up with competing ventures by Microsoft and Google because they do not provide for universal access to digitized books. These commercial ventures prohibit books being accessed by competing search engines. So far, 80 libraries and research institutions have signed on with Open Content Alliance. They must pay for the scanning of their books while Google and Microsoft offset that cost for their participating institutions."
This discussion has been archived. No new comments can be posted.

Internet Archive Challenges Google

Comments Filter:
  • Society lost (Score:5, Insightful)

    by packetmon ( 977047 ) on Monday October 22, 2007 @09:58AM (#21071587) Homepage
    I believe I've commented on something like this before. Might be a good idea to archive the books lest somewhere in the future we re-live something like the Spanish Inquisition where important literature was lost. Its also making this society a bunch of couch potatoes. What ever happened to walking into a quiet library, the smell of stale books, looking around at people. Its slowly being replaced by reading books online and hitting ctrl-w to close annoying popups while you read. Currently I have about 30+ Cisco (CCIE/NP/IP/etc) books and each come with their PDF's. At first I thought, neat I can read them on my laptop... Nowadays I find its easy to just open the book, nothing like butchering my books up with highlighters... This world is coming to one where companies will be fighting to keep us locked in our houses. Call me a troll, just speculation
    • Re:Society lost (Score:5, Insightful)

      by CRCulver ( 715279 ) <> on Monday October 22, 2007 @10:21AM (#21071847) Homepage
      I enjoy very much sitting in a university library with piles of books around me to work through. However, as someone who spends most of the year traveling, usually lugging around a bag or two of hardbound specialist literature in addition to an already heavy backpack, the more I can put on my notebook the better. PDFs don't weigh anything.
    • Re: (Score:3, Insightful)

      by luder ( 923306 ) *

      What ever happened to walking into a quiet library, the smell of stale books, looking around at people. Its slowly being replaced by reading books online and hitting ctrl-w to close annoying popups while you read.
      Funny, I bet someone said the same thing when the average person began to have enough purchasing power to buy individual copies of books...
    • by RsG ( 809189 ) on Monday October 22, 2007 @10:26AM (#21071895)

      Might be a good idea to archive the books lest somewhere in the future we re-live something like the Spanish Inquisition where important literature was lost.
      Oh really, does that seem likely?

      I don't expect the Spanish Inquisition...
      • Nobody... (Score:5, Funny)

        by jefu ( 53450 ) on Monday October 22, 2007 @10:39AM (#21072041) Homepage Journal

        Nobody expects the Spanish Inquisition.....

        (Can't believe I'm the first one to respond with that. Of course by now I'm probably not. )

      • Nobody... (Score:3, Funny)

        by benhocking ( 724439 )

        Nobody expects the Spanish Inquisition!

        (I couldn't bear to leave you hanging.)

    • Re: (Score:2, Insightful)

      by absorbr ( 995554 )
      But there is advantage in being able to search those technical manuals. Leave the couch potato business to me, there's already enough reason (health) to get outside, soak up sun, and exercise.
    • Re:Society lost (Score:4, Interesting)

      by beadfulthings ( 975812 ) on Monday October 22, 2007 @10:51AM (#21072195) Journal
      I don't think you're a troll, and until recently I would have agreed with every point in your post. Now I'm not so sure. Anything that preserves the written word from future loss, and makes books and literature available to more people, is a good thing. Freeing this activity from commercial restraints is a good idea; somehow the commercial route would make me think my choices might be somehow limited by what the commercial software "wanted" to show me.

      It's the quiet library...dusty books part that has me a bit concerned at the moment. My city recently built a brand-new neighborhood "anchor" library within striking distance of my house. That eliminated the need for a tedious drive or bus ride downtown to the main library, the expensive necessity of finding scarce parking, and the tyranny of downtown "business hours." The branch is convenient, has good parking, and is open for people who have to work during the daytime. It's bright and attractive. Endowed with an outdoor "reading garden" for good weather, it will shortly boast an indoor coffee area. It has banks of computers to be allocated to either youngsters or adults, a pretty good periodicals section, and a pretty good reference section. It also manages to cater to our increasing Hispanic population. The kids' section, which takes up half of the main floor, appears to be excellent. There's a huge section of fairly current DVD films. Tables (and to my surprise) comfortable easy chairs are invitingly scattered throughout.

      I guess the problem is that with all the beautiful amenities, there's actually not much room for books. To be fair, they are just starting out, and their holdings will certainly increase. I'm just not sure where they will locate a collection that's much larger. I would divide my reading into "work-related," "feel like I ought to read it," and "just for pleasure." It's not easy to find new or interesting titles in any of those areas. They're trying, but there's just not enough space.

      I've resolved that, to an extent, with a nifty little Mac utility called "Library Books." By entering my online access information, plus the library's catalog system (iBistro/Sirsi), I suddenly have complete, convenient access to the entire city and state library system. I can browse, search, and reserve. I could do all of that simply by going to the library's main Website and logging in, but the utility does me little favors like alerting me to the arrival of reserved titles and putting a big, red star up on days when books are due. It's a convenience that makes online browsing just a bit easier.

      I've thrown myself into the new anchor library, in tandem with the simplified online access of the Library Books utility. I've become an evangelist, of sorts, for requesting and reserving books. It's amazing the number of people who don't know you can do that. The sheer square footage required to hold a large collection isn't feasible for neighborhood libraries, and I feel as though this sort of system gives one the best of both worlds. No yellow markers, though . . .
    • Not just books! Anyone know of a place where research papers can be searched and downloaded for free? I was trying to find some yesterday, and I ended up having to log into my School's library system to search because I couldn't find even one free website with articles in their entirety!
      • Re: (Score:3, Interesting)

        I don't have to do this myself but I have heard about this problem (and wrote a little about it []). Your school pays publishers for access to journals and you can't necessarily get to them from home. There is a plugin for Firefox called LibX [] that makes it easier to search what your school's library has available. It has to be customized somehow to make it work with the school's system. The University of Windsor has customized LibX as Foxy Leddy [] (Leddy is the school library). Maybe your school has something s
        • Thanks for the plugin link, but I still wonder why there are no free sources online... I mean, it's not that I couldn't find the same sources online for free that I could get through my school's library, it's that I couldn't find ANY sources online... not one! And I know for a fact that many research papers are distributed under creative commons. Just seems really strange to me that such a system doesn't exist yet, especially for something like research papers, for which the internet is almost perfectly s
          • by lgw ( 121541 )
            The only information you can't ind online these days is information published in scholarly journals. Horray for education! It's no wonder that Wikipedia is in such demand - sure, it's not the most reliable source, but it is certainly the most available source!
            • That's true and it's a pity.

              However, scholarly work in journals is already paid for by the public through grants, so its only a matter of time for unscrupulous left leaning students of major institutions worldwide to make their ids and passwords to EZProxy servers available on so called free literature sites so that anybody in the world can access these common resources of mankind.

              Also, if more people were interested in books than music, it's conceivable that some terrible people would put many types of

          • You could start your own Journal with OJS [] or look around in the Public Knowledge Project [] though you'll probably find (again) that dismally few journals publish good free articles. The reasoning I heard was that the most prestigious journals are expensive and the prestige is tied to that cost. So the cc distributed papers aren't as well-regarded as those with a high price tag behind a firewall.

            fwiw I did install OJS just to say I did it and I could've started publishing my own journal within a day if I had
    • by Threni ( 635302 )
      > Currently I have about 30+ Cisco (CCIE/NP/IP/etc) books and each come with their PDF's. At first I thought, neat I can read them on my laptop...
      > This world is coming to one where companies will be fighting to keep us locked in our houses.

      Dude, PDF is `portable`, right? And they're on your laptop? And you can search for stuff instantly, and not just hope the index is up to it. So you can get your work done more quickly and spend more time outside, away from libraries (which in my experience smell
    • Re: (Score:2, Insightful)

      Why not go to the library?

      Easy, because many public libraries are becoming nothing more than places where people go to check their e-mail. Of course a nice university library is something completely different, but the small town public library is pretty much giving up on those bulky paper things. Why buy new books when you can offer poor people a place to get on-line (and get a bunch of grants)?

      My local library only buys 10% of the books that it bought 8 years ago.
      • My local library only buys 10% of the books that it bought 8 years ago.
        And why would they have to, when the digital age has made tracking and requesting books through inter-library loan easier than ever?

        I'll bet the collection at your local library is better than you think, when you compensate for the books that can be requested from other libraries in the network.
    • I agree that a nice, hard bound book is, at the moment, more pleasant to read. However, technologies such as e-Ink and others that allow you to read something digitally without the eye-strain of using a back lit monitor are catching on. I think a few factors make digital copies more advantageous - cost of duplication, storage, protection from damage, searchability.

      Storage: I just moved, and I moved three bookcases full of books. That sucked. If those were all digital, I'd have hauled my computer from A t
      • Who hasn't spent 30 minutes skimming a book trying to find THAT ONE PAGE!?

        That's precisely why I've got pirated copies of the Harry Potter series on my laptop (JK Rowling is against ebook versions of her books, so there aren't legit ebook editions). There's a Meetup group in my area, and we often want to refer back to a specific page. Sometimes a few of my fellow nuts are able to find it in their meatspace versions faster than I can look in finder on my Mac, but it sure is convenient.

        The same goes for Lord []

    • I think this is also about what you are used to, and how you learn to interact with the written word and research in general. As today's younger generations grow up being more used to everything being available on line, they will demand content to be available online and be able to work with it as effectively (if not more so given storage, cross referencing, dynamic searching etc).

      I'm with you, I much prefer reading stuff on paper. I'm just not convinced that it will hold true in years to come.

      I don't see
    • by vimh42 ( 981236 )
      You have a point, to an extent. The issue is that a great deal of content is not available at the library or at least at your local branch. How many of those Cisco books could you get at your local library? The extent of my reading is a bit different, but not much of what I read can be found at the library. Perhaps your argument is more about printed media versus digital?
  • by Sherloqq ( 577391 ) on Monday October 22, 2007 @09:59AM (#21071601)
    ... but on a much larger scale?
    • by absorbr ( 995554 )
      I like the idea, but what about copyright issues? If you were an author, would you be happy about your work being accessible at no cost by anyone with a computer?
      • Yes, what's your problem?
        • by absorbr ( 995554 )
          Well I'm not an writer, so it doesn't affect me, but I would imagine that they would like to get paid for their work. You can't live on welfare! If something like this existed in the form of a free, searchable website, you can bet I'd probably never buy another book again.
          • Re: (Score:2, Funny)

            by eldepeche ( 854916 )
            Maybe they'll move toward an ad-supported revenue model. Did you know Harry Potter wears Reeboks?
          • by lahi ( 316099 )
            Here in Denmark (and I guess there are other places with a similar system), Danish authors and translators who have their books in libraries, can apply for a grant, which depends on the number of copies of their works in Danish libraries. This is a form of Public Lending Right [] compensation, and is not based in copyright law. It is only paid for works in Danish (originals or translations to Danish), and should be viewed more as a way to support Danish language and culture. The pay varies between (this year)
      • Re: (Score:3, Interesting)

        by SnarfQuest ( 469614 )
        Why not hear it from several authors, from one publisher: []

        Several authors use older releases as bait to tempt you to buy more recent books.

        Project Gutenberg is concerned mostly with old, out of copyright books, author usually long dead. Many of these books would be unavailable otherwise.
        • by absorbr ( 995554 )
          it's an interesting strategy. I would just expect that the authors (or the owners of deceased authors' estates) get the choice of whether or not to include their works.
    • Relevant Link (Score:3, Interesting)

      by Chapter80 ( 926879 )
      In case you missed this discussion back on October 2, [] Carnegie Mellon has a service which helps to better digitize these books. It's called Recaptcha, [] and it uses otherwise wasted human cycles to convert text that was hard for computers to OCR.
  • A Better Link (Score:5, Informative)

    by value_added ( 719364 ) on Monday October 22, 2007 @10:02AM (#21071637)
    The Libraries Shun Deals to Place Books on Web [] story in The New York Times covers the subject fairly well.
    • Not really. It ignores all of the other digitization projects taking place.
    • by Guedon ( 756177 )
      Katie Hafner's article is crucial. In the area of mass digitization, the main problem is that most people think paper while dealing with digital files. Gregory Crane calls this "incunabular" and he is right. This question also points toward something a little more fundamental: we should be weary of quick, pracmatic-looking, solutions to digitization because we may end up paying a great deal more down the line. Clifford Lynch, in a short article published in the volume on Open Access edited by Neil Jacobs, a
  • Recognising the restrictions of the current iterations currently available and working to provide a better resource that most or all libraries will support. The free exchange of ideas (not entertainment for those of you who download your entire music libraries from Kazaa) will promote progress across the board.
    • Re:Way to go! (Score:5, Interesting)

      by kebes ( 861706 ) on Monday October 22, 2007 @10:15AM (#21071769) Journal

      The free exchange of ideas (not entertainment for those of you who download your entire music libraries from Kazaa) will promote progress across the board.
      It's interesting that you draw such a sharp difference between information and entertainment. I agree that there are differences between content intended to transfer knowledge, and content intended to amuse... but certainly there are strong similarities between the two.

      In particular, if you accept that free exchange of ideas will promote intellectual progress, then is it not also reasonable to suggest that free exchange of artistic content will promote cultural progress? This is the central notion that Lawrence Lessig advocates: that overly restricting the distribution, reuse, and remixing of art and entertainment will inherently stifle culture. (Note that Lessig does not advocate wanton infringement nor abolition of copyright: merely a 'sane' balance between the rights of content creators and the rights of content users.)

      With respect to this current initiative, it would appear that they intend to scan and index books that are oriented towards information, as well as those oriented towards entertainment. In my opinion, this is a good thing. There is much that people can learn and grow by having easier access to ideas, where "ideas" means both informational sources, as well as artistic sources.
    • not entertainment for those of you who download your entire music libraries from Kazaa

      It's exchange of ideas if you're a musician or a record producer.

    • Music is an exchange of ideas. Listen to a hymn, an ancient poem, dark side of the moon, stadium arcadium, etc. In fact, music is even better as exchanging ideas since it has more "bandwidth" than just the word: the tone and speed of the music. Songs are the oldest way of preserving and extending knowledge.
  • is there any estimate on how long it will take all these projects combined to scan the entire existing catalog of books, accounting for expansion and development of better technologies to do the scanning, etc?
    • TFA (or at least the NY Times article I read) said that's not the intention. they're at least starting by "filling gaps" in specialized material that only a few libraries have. it's not "make common books even more available than they are now," it's "for specialized collections like the Mark Twain papers, finally get to 100% available" because 100% is so important for certain applications like scholarship and research.
  • Scan My Books (Score:4, Interesting)

    by Doc Ruby ( 173196 ) on Monday October 22, 2007 @10:10AM (#21071707) Homepage Journal
    I buy a lot of books. I've got probably 10,000 or so. I wish I could search through them. Some for reference, sometimes because I read something that sounds familiar that I want to find where I first read it. I'd also like to read them on my PC sometimes, or even on my phone like when I'm waiting for a while somewhere. And I'd like to copy/paste short passages from them into messages I send on the Internet.

    If this project is really "open", can I have my own libarary scanned? How much does it cost? I own the rights to copy my own books for my own personal use. Does something make these other "official" libraries eligible to use their full rights to their content in a way that I cannot?
    • If this project is really "open", can I have my own libarary scanned? How much does it cost?

      There are plenty of document scanning services around, I know for low-volume (less than 100k pages) I've priced them out at about $0.08 per page at high resolution. I'm not sure what kind of surcharge you'd pay for them not being able to batch-feed (since you're talking about books).

      Or, do what I did and rent a good scanner and pay a couple high school kids a fair wage to do it. Or, offer them piece rate of a nick

      • I buy a lot of books for $1-5 used. I might pay up to a dollar per 300pg book to be scanned, which would cost $10K for my whole library. At $0.25 a book, about $0.008 per page, I'd do it.

        Even HS kids wouldn't work for $0.42:h. This project calls for a scanner with automated pageflipping. If this Open Library project doesn't have that, then I expect that no one does (yet). I'll wait.
        • Even HS kids wouldn't work for $0.42:h

          Depends on what country they are in... but the cost of shipping the books to China/India/etc would likely be prohibitive. This is where the scanning services can sometimes be more cost-effective.

          Or, you could keep tape flags in your wallet (for me, much more useful than a condom :)) and mark pages for scanning later. I do this if I note a particularly interesting magazine article or passage from a book that I want to keep on file. It actually makes me feel like the

    • by Cemu ( 968469 )

      I own the rights to copy my own books for my own personal use.
      Whoa, whoa, whoa, whoa, whoa... whoa... Own I dare ask? Are you sure you don't just have a license to read it in the format dictated by the publishing company? If you get a PDF of the book would you need to verify that you have a genuine copy of that PDF or risk the possibility that the pages would turn black?
      • No, I am sure I own the book. Nowhere does the publisher even claim that I bought a license or anything else less than ownership of the object and the rights inherent in real property to use that object. The uses I described are extremely well documented in law as fully protected by law.
        • by lgw ( 121541 )
          So why would you want to give up that well-established legal protection for the license to an eBook? As you pointed out, scanning them yourself is problematic.
          • The point is that there is no license required. I don't need any extra rights or privileges to scan my own books into another format for my personal consumption. I just need an automated book scanner. That scanning doesn't delete any of my rights.
            • by lgw ( 121541 )
              Sure, but it's not very practical for most people. So people will tend to buy eBooks of one form or another, which thus far have been horribly DRM encrusted, even time limited (for textbooks). Hardly what you'd want as a replacement for physical books.
              • I'm not most people. But my point was that the new project should be open to most people. No one seems to know whether it is, or what it costs.
    • Some for reference, sometimes because I read something that sounds familiar that I want to find where I first read it.

      Yeah, I've got the same problem. Often times I just want to quote something rather than paraphrase a particular section I remember reading but it takes an hour or so because the included index at the end of the book was sorely lacking for a 900 page book.
    • These projects mostly include works "out of copyright." They also include some works still in copyright but which are out of print. THey can't include books in copyright & in print unless the copyright holders authorize this.
      • So? I don't need a copyright exception to do what I described with my own books.
        • I'm no expert on this subject. But it seems to me that if you want to digitize yr own library for yr own personal use you should be able to do that. But if you make use of the material in ways that are no longer personal (uploading material to a website or blog for example), you might have copyright issues. Also, OCA costs 10 cents/pg. for scanning. That's a significant cost for a 10,000 volume library. Don't know if this answers yr question or not.
  • Wondering... (Score:3, Insightful)

    by 140Mandak262Jamuna ( 970587 ) on Monday October 22, 2007 @10:11AM (#21071733) Journal
    How many of these libraries think of Open Source and software platform choice? How many of them make sure their web sites are platform agnostic, equally accessible from all browsers? These people are willing to stand up and are willing to pay more to preserve their liberty. Hats off to them. But does this stand also extends to not having their documents locked down in a proprietary format encumbered with licenses and restrictions? I would very much like such ideas, being independent of vendors, would extend to Corporate America too.
    • I dare you to pick five cities, of random size, and look up their library websites. I guarantee you at least four of them will be poorly laid out, with broken links, and at least one will have eye-bleeding banners in fantastic 1996 style. The fifth will be seamless, make sense, and (hopefully) work in multiple browsers. The fifth will be the one who had an actual webmaster, instead of "well Jill here is under 30 so she knows something about computers."
  • It has got many more documents. And often /all/ versions ever of the document. And it doesn't just store HTML but much much more.

    Internet archive is an excellent service. Best of luck!
  • After all.. (Score:1, Interesting)

    by Anonymous Coward boils down if I can use my torrent client to download the stuff (=good) or not (=irrelevant to my life).
  • False Dichotomy? (Score:5, Interesting)

    by internic ( 453511 ) on Monday October 22, 2007 @10:33AM (#21071959)

    There's a story about this in The New York Times [] this morning (free reg required). It begins:

    Several major research libraries have rebuffed offers from Google and Microsoft to scan their books into computer databases, saying they are put off by restrictions these companies want to place on the new digital collections.

    The research libraries, including a large consortium in the Boston area, are instead signing on with the Open Content Alliance, a nonprofit effort aimed at making their materials broadly available.

    The opposition between the Open Content Alliance and Google may not be as much as it seems at first glance. From the NYT article:

    Adam Smith, project management director of Google Book Search, noted that the company's deals with libraries were not exclusive. "We're excited that the O.C.A. has signed more libraries, and we hope they sign many more," Mr. Smith said.

    It looks like Google will digitize the collection for free in exchange for exclusive rights to offering searches of the digital data, but the libraries don't give up rights to have someone else digitize the stuff again and do with it as they see fit. So they can go with Google for now if they want and the O.C.A. later as they have the resources. This seems pretty reasonable to me. I don't know what the deal Microsoft is offering looks like, but I wouldn't be surprised if it's much more restrictive.

    • I apparently missed page 2 of the article. :-)

      Microsoft is part of the O.C.A., except that, "A year after joining, Microsoft added a restriction that prohibits a book it has digitized from being included in commercial search engines other than Microsoft's." Sort of an "embrace and extend" approach?

      Jokes aside, this does seem to be a bit more of an open approach than Google's.

    • Of course Google would say something like this. So would Microsoft. It's what's called "PR Spin".

      "Of course we're happy our competiton is starting great new projects that compete with us in ways we are not able! Competition is good! Yadda, yadda, yadda..."

      People, in less than 2 years, Google and Microsoft will be indistinguishable, both being IP whores (and I'm not just talking about network addresses...) driven by mercenary stockholders interested in nothing but cash, raping and pillaging the Interweb

      • I agree that no matter what they'll try to spin it to minimize the conflict, but the important question here is what are the facts of the matter? It sounds as if the facts are that one is perfectly free to work with both Google and the O.C.A. (based on the quotes in the article from other parties besides Google). This seems like a fairly reasonable arrangement to me; you have the option to get things scanned for free by Google but with restrictions and/or for a fee by O.C.A. without those restrictions.

    • I didn't mean to claim in my original post that MS or Google were trying to prevent libraries fr going with OCA though I wouldn't be surprised if they did they quietly out of public view. But an important distinction for the commercial ventures is their restrictiveness in not allowing competing search engines to access their digitized content. That's why OCA's mission is so important & why it's important that OCA become the digitization method of choice for all libraries. I don't care if GOogle or MS
  • All is fine just as long they don't resort to shredding the books ;)
  • Which books are digitized anyway? With copyright being as ridiculous as it is (what is it, 50-100 years after the death of the author?), are we likely to see anything modern in such a collection? I would hope that libraries would have some sort of exemption from this, except that in this case it sounds like the data might be used for commercial searches. I also wonder if these will be regular PDFs or if there will be some sort of DRM on them. Can anyone more knowledgable weigh in on this?
    • If you read the NYTIMES article on this which has been cited twice prior to this post and on the paper NYTIMES front page above the fold, you will be reassured.

      The Open Content Alliance plans to digitize expired copyrights of 1922 and before. Then Boston Library Consortium (34 million expired copyright books) is seeking to digitize later dated in-copyright but out of print books.

      Google's approach for in-copyright is close to that of various on-line Journals. A reader can only read a few pages of a copyright
  • Most modern books are created in electronic form to begin with and are printed with high speed offset printers from files. Only older books have to be scanned.
    • by Jonathan ( 5011 )
      Right. But it's really only the older books that are relevant, unless publishers allow free access to the newer books, which would be cool but unlikely.
  • I hope libraries in other countries will be included as well. Please start with Sweden. :-)

  • by nomadic ( 141991 )
    The wayback machine...I don't think I've ever used a website that was so lagged yet still technically functioned. Sort of.
  • This project predates google's scanning project by several years. Brewster tried to get google involved, but as usual they decided to go alone. While the OCA was announced in 2005, it was an offshoot of the Internet Archive/CMU/Raj Reddy's Million Book [] project which was started in 2001 with books being scanned in India.
  • and the preservation of books (knowledge). I've also read many of the discussions about Google vs. Project Gutenberg, Corporate vs. Non-Corporate, etc. Personally, I hope preservation of knowledge for all wins out over greed and ignorance. I also find it interesting that in a game, how many years old now?, there are several prophetic quotes including the following:

    As the Americans learned so painfully in Earth's final century, free flow of information is the only safeguard against tyranny. The once-chained

Someday your prints will come. -- Kodak