Bringing the Library of Congress Newspapers Online 240
smooth wombat writes "If you want to read a newspaper article from sometime in the past (say 1920 for example) your only options right now are to go to your local library and hope they have a microfiche file of that paper or take a visit to Washington, DC and the Library of Congress. That may soon change. CNN is reporting that by 2006 the government will have the first of 30 million digitized pages from papers published from 1836 through 1922 which will be available to anyone who has a connection to the net. The project is a joint cooperation between the National Endowment for the Humanities and the Library of Congress.
The span of the joint project is limited because type faces of printers used before 1836 are too difficult for optical scanners to read, and copyright restrictions are in force on papers published after 1923."
Copyright limits (Score:5, Insightful)
Re:Copyright limits (Score:5, Interesting)
Re:Copyright limits (Score:2)
Paid access to its historical archives?
Advertising money for the ads you would see while browsing such archives?
Re:Copyright limits (Score:3, Informative)
The New York times has free registration (an
Re:Copyright limits (Score:3, Insightful)
One of the very real problems with copyright law is that it allows publishers to "capture" our history and prevent access to some of
Re:Copyright limits (Score:3, Insightful)
While I approve the impulse, I think this would be a nightmare to maintain, particularly if the "expire after a year" idea was appli
Re:Copyright limits (Score:3, Informative)
Plagarism can occur with or without there being copyright, and with or without permission from the author. If copyright determined plagarism, students who copied papers would be all fine and kosher because they had permission to copy the paper from the copyright holder.
Also, plagarism is legal with regards to the copyright code and people who hire ghostwriters d
This is a great idea... (Score:2, Informative)
This sucks (Score:5, Funny)
Re:This sucks (Score:5, Funny)
Re:This sucks (Score:2)
Re:This sucks (Score:2)
Besides, who said a harddrive array can't burn? It just take more work than paper.
Copyright restrictions (Score:2, Interesting)
The local library has every edition of the local papers on microfilm, and I suppose they could put it all on DVD too.. When does it become a copyright issue?
Re:Copyright restrictions (Score:3, Interesting)
The same as the law regarding any other form of duplication and distribution. Why would it be any different just because it's online?
The local library has every edition of the local papers on microfilm, and I suppose they could put it all on DVD too.. When does it become a copyright issue?
Assuming the microfilm was legally purchased, they're entitled to show it to as many people as they'd like. It doesn't become a copyright issue until they start making
Re:Copyright restrictions (Score:5, Insightful)
Why would it be any different just because it's online?
In the online world, it is completely impossible to show somebody something without similtaneously giving them a copy of that same something. If the library shows you a html version of the copyrighted work, then it had to do so by sending you the contents of that work as a second digital copy, independant of the copy that's on their hard drive. If the library shows you a GIF image of the copyrighted work, then it hd to do so by sending you the contents of that work. No matter what scheme is used, no matter what technique for encryption is used, the fact of the matter is that at some point, even if just temporarily, your computer has to have its own copy in one way or another.
On the other hand, if I show you a physical book, this doesn't cause two seperate copies of the book to appear.
Unless the online library is willing to delete their copy (even from backups and from the hard drive) while you have your copy (and then trust you to send it back to them when you are done or pay them for it if you lose it), then there cannot be a working analogy between online and physical libraries as far as copyright law goes. Even someone not intending to make use of their copy is still technically breaking copyright law every time they look at a copyrighted work. Your browser's cache is filled with copyright violations if you've ever visited any website with any copyrighted content recently (which is most people who surf the web, probably).
The problem is that the original law was not written with this technology in mind, and the attempts to update it are written by people who just don't understand what they're doing, don't understand how the technology works, and aren't listening to those who do, and instead are listening to those with a vested interest in lying to them about the issue. Hence we get laws that if interpreted literally would outlaw the entire world wide web, but then get enforced selectively. (ALWAYS a bad situation to be in, where it is nearly impossible to avoid violating a law - then the law becomes a means to randomly smack-down on people for whatever you wish to discriminate against them for.)
Re:Copyright restrictions (Score:3, Interesting)
Umm, no. If they site you were browsing had the right to distribute the materials, they're not violations. If the site's TOS doesn't allow caching, and they make use of HTTP headers that are supposed to forbid caching, and you knowingly modified your browser to ignore those headers, it might possibly be a violation. Even then it would most likely fall under Fair Use. I doubt
Re:Copyright restrictions (Score:3)
I'm inclined to agree with geoffspear on the caching angle.
Fair use seems pretty permissive in practice. Law being statute plus interpretation plus enforcement when applied to copyright allows much more than a conservative interpretation of the statue would suggest. Especially considering that there has never been a 'photocopier at the library' war.
There are practical matters too. We don't see people trading books over the P2P nets. It's a PITA to read a book on line and the cost of printing a copy while
Re:Copyright restrictions (Score:3, Interesting)
Therin lies the problem: Copyright law starts by making every living being a criminal, then has poorly defined grey areas of vague exemptions like "fair use" that more often than not have been defined through court cases that cost people money and livelihoods. It wasn't made with any technology in mind, the authors were lawyers who realized they could make money by making sure that every new advance and situation would requ
Re:Copyright restrictions (Score:3, Insightful)
Just because you're not aware of the legal history of copyright law doesn't mean the issues you raise haven't been considered.
We can analogize, for example, to the issue you mention above with copyright law-making from almost 30 years ago. It's been long realized that using a computer program almost always requires making a
Re:Copyright restrictions (Score:3, Informative)
No, it's not. These guys (http://www.authentica.com/ [authentica.com]) have done quite a good job of document control and management. I can show you whatever I want and you can't see it once you're done and I revoke access. (requires a plug in for acrobat to use).
We use this system to control restricted access and above documents at my office. Not even a screen capture works!
-
Re:Copyright restrictions (Score:2)
And I quote The local library has every edition of the local papers on microfilm, and I suppose they could put it all on DVD too.. When does it become a copyright issue?
which really is quite interesting. Why is it not a copyright issue for the local library to do it, while it is for the national library?
Re:Copyright restrictions (Score:2)
The LOC is making and distributing copies of the articles. The local library is not. What's so hard to understand?
Re:Copyright restrictions (Score:2)
Why couldn't the LoC just do this with things after 1923?
Re:Copyright restrictions (Score:2)
Because your local library is paying thousands of dollars to the copyright holders to do that.
Fees (Score:2, Interesting)
Typeface ? (Score:4, Interesting)
Surely the OCR process could be recalibrated to identify a different typeface ?
Re:Typeface ? (Score:5, Funny)
"Purfuit of Happineff"
Re:Typeface ? (Score:4, Insightful)
That's not hard. It would be easy to get the OCR to recognize the long-s (which does in fact look different from the f); even if you don't, post-processing (dictionary lookups to see if f or s is valid at a point) can clear up many cases, and for those it doesn't, well, you're going to have to check and fix the OCR anyway.
(This is not theory; Distributed Proofreaders (http://www.pgdp.net/ [pgdp.net]) has and uses such a post-processor.
And even so... (Score:3, Insightful)
I don't understand why the text HAS to be selectable... That's cooler, but it shouldn't need to be a requirement.
Re:And even so... (Score:3, Insightful)
Re:And even so... (Score:2)
Re:Typeface ? (Score:3, Funny)
Re:Typeface ? (Score:2)
Perhaps but I don't think the actual letters have. I suspect that the document quality is too poor to make out the characters. Smudges, tears and faded characters probably have more to do with it than the language.
Re:Typeface ? (Score:2)
Re:Typeface ? (Score:2)
Not really.
"But what Plutarch can this age produce, to immortalize a life so noble? May some excellent historian at length be found, some writer not unworthy of his subject; but may his employment be
long deferred!"
It's not American, but that's from a book published in 1808 in Britain, edited from text written in the 17th century. The grammar may be a lit
Re:Typeface ? (Score:5, Informative)
1. Even with the same exact font (blocks of type) being used, one letter 'A' and the next letter 'A' could look different enough to confuse an OCR program, due to blotchy ink or blotchy paper, like so: 2. Also, the spacing between letters was not as uniform, which would con fuse an OCR pro gram into B reaking words at in con vein ientplaces.
3. And, as the other pofter mentioned, theref the ditterent ftyle ot fymbolf they ufed to ufe.
See... (Score:3, Interesting)
Now, where is the open source OCR software that they can use to read the old wonky typefaces?
Newspaperarchive.com (Score:4, Interesting)
Re:Newspaperarchive.com (Score:2)
As a side note. I have been collecting a newspaper every year on my children's birthdays and will put this together for them in a scrapbook when they are about 20 so that they can see what was relevant in the areas they were living in at the time.
Re:Newspaperarchive.com (Score:3, Informative)
Yes, but you charge for it. This will be free. If I were you, I'd start looking for a new business model... or start donating to Disney's lobbying fund.
Re:Newspaperarchive.com (Score:2)
AT THE HOSPITAL Mlltlrad 115 Clav Mrs Ronald 1 Mrs Cora 1020 and Tom Bradli'v Hamilton ucra admlltrd yo'lerdav lo the Chilli colhe hospital AT WASSMANN RESIDENCE Mr pnd frccl of this n'y iveie iliuner yupsls lasl evening of Mr intl Mrs R O of Ulica lliE occasion
There is a very nice picture I could download if I wanted to, though.
Re:Newspaperarchive.com (Score:2)
You know it's coming (Score:5, Funny)
Howdy, pardner! To read about that scalliwag Black Bart's shootout with Arizona Jack last week, you'll need to pay two bits per article or buy a subscription for a gold dollar or its equivalent in salt pork or live chickens.
Not entirely accurate (Score:3, Informative)
From 1877-1986, the Post offers the full page scans of the articles as they appeared in the newspaper. Begining in 1987, the full text versions of articles (without photos) are available.
Oxford University tried something similar (Score:5, Informative)
Half the fun of old papers is... (Score:5, Insightful)
These, to me, were always half the fun whenever I perused old microfiche in the library.
There is a bar in NYC called McSorley's, which has been in continuous existance since 1846 or so. They have framed newspaper articles on the wall from over a hundred years ago, 130 year old pictures, political campaign buttons from McKinley's run. Talk about a neat experience.
Actually seeing the old print would mean more to me. I rather hope that they serve images of the old papers, not just the computer-read text. But hey, that's just me.
Re:Half the fun of old papers is... (Score:2)
Perhaps later on an image archive would be useful, but untill they can get several terrabytes of bandwidth for free and image->text (on the fly) systems are perfected text is probably a better idea.
Grep! The only way to search 100 years of data for a misspelled word so you can poke fun at the foolish writer.
Re:Half the fun of old papers is... (Score:2)
This was for real.
The newspapers need to step up (Score:3, Insightful)
Re:The newspapers need to step up (Score:2)
Re:The newspapers need to step up (Score:2)
Only to someone who's forgetting history. Some of the issues will be surprisingly relevant; some will only matter to those who study history. It doesn't mean that once something is a hundred years old, it doesn't matter; the roots of current events are in matters more then a century old, often many centuries old.
copyright insanity (Score:5, Insightful)
and copyright restrictions are in force on papers published after 1923
in case anyone was still left who thought copyright laws were reasonable....
Re:copyright insanity (Score:3, Insightful)
The point is that this is a perfect illustration of why the current copyright length is insane. It's something you can use to explain it to your neighbors, and they might get it. It's even something you might be able to use to explain to your legislator in terms they can understand ("hey, look, long copyrights even get in the way of this perfectly reasonable government proje
This is old news (Score:5, Interesting)
To most Americans, the period from 1790 to 1915 is kind of a mystery except for Gettysburg and the Ford Theater.
There was tremendous growth in the number of newspapers during that period, starting at a handful in 1790 to thousands in the 1920's. They fell on hard times with the advent of radio.
During that time, everyone with a spare nickel and a desire to publish something put out their own rag. They would trade stories, publish letters to each other, have flame wars, etc. I think it must have looked a lot like the blogosphere, with a bit more latency.
The more things change, the more they stay the same. Sometimes, we need to see the old news to recall that.
Re:This is old news (Score:2)
What an interesting example of history repeating itself. Here we have a 19th century implementation of Usenet. With the LoC(Library of Congress, that is) and the Gutenberg Project(which has a sizeable but not LoC-sized collection already), we wil
Re:This is old news (Score:2)
Exactly. I include myself in that. I got very little history in high school apart from the early history of our country. I got very little recent history (post civil war.) It's the one part of my high school experience that I consider lacking. I took a History of Western Civ. since 1600 at one of the Universities I attended and while very interesting, it focused mainly on European history.
Re:This is old news (Score:2)
Want earlier papers? (Score:2)
Presumably papers after 1923 will be added one year at a time as the copyright expires? Or will the mouse protection league keep them locked away for ever?*
*On a related note a BBC radio broadcast about a hitch hiking trip had a comment from a Fat Woman in her slightly derranged middle age who was on her way to Disney World in Florida. She said that America would be a much better place if Disney ran it
Re:Want earlier papers? (Score:4, Interesting)
The Mickey Mouse Protection Act,(aka Sonny Bono Copyright Term Extension Act [slashdot.org]) tacked on an immideate and retroactive 20 years to copyright length. So, don't look for anything to be entering the public domain until 1/1/2019. And that's not even considering the likelyhood of Congress extending the length of copyrights again.
Kind of a bummer... (Score:3, Interesting)
I'm sure OCR technology will advance quickly enough to allow the scanning of these newspapers.
SWEET! (Score:2)
My first time in the paper: Front page of Times Union on Feburary 19th/20th, 1989.
Please Simplify (Score:2)
How many Volkswagen Beetles would be needed to contain this?
Why not pass it through project Gutenburg? (Score:3, Insightful)
Re:Why not pass it through project Gutenburg? (Score:2, Interesting)
I've done over a thousand pages since it's started... It's gotten really easy for me to pump out pages, and I've been turned on to alot of different information that I'd normally not expose myself too... It's quite enriching -- so you should try it if you got time!
Time Travel (Score:2)
can't scan? (Score:2, Insightful)
Bollocks. Even if they are trying to OCR this stuff, it's critical that the original page bitmaps remain available, anyway.
I'm amazed they still have these archives. One of my favourite people, Nicholson Baker [wikipedia.org] has made a personal crusade [gwi.net], written books [salon.com] on the subject, and put enormous amounts of his own cash, into preserving newspapers that government archives are hellbent on destroying. In particular he attacks two
Re:can't scan? (Score:2)
Check out Dec. 7, 1941 some time (Score:2, Interesting)
Re:Check out Dec. 7, 1941 some time (Score:4, Funny)
Already been done for journals (Score:2, Informative)
process be similar for for newspapers. But, newspapers are printed on lower quality paper and
possibly lower quality printing technology.
Making of America (MOA)
http://cdl.library.cornell.edu/moa/ [cornell.edu] (Cornell U)
http://www.hti.umich.edu/m/moagrp/ [umich.edu] (U Michigan)
National Geographic online (Score:2)
Re:National Geographic online (Score:2)
http://www.nationalgeographic.com/cdrom/
Re:National Geographic online (Score:2)
Re:National Geographic online (Score:2)
If you're willing to scan them, Distributed Proofreaders (http://www.pgdp.net/ [pgdp.net]) is willing to correct the OCR and even have people assemble them into HTML, provided you're willing to let Project Gutenberg (http://www.gutenberg.org/ [gutenberg.org]) post them.
Goverment and the american history. (Score:2)
Re:Goverment and the american history. (Score:5, Insightful)
The LoC would have their reputation destroyed among the librarian and researcher communities if they were caught doing that; and they would be, because hard-core researchers would notice any significant changes in the text and go back to the microfilm and original text copies.
Librarians tend to be among the strongest anti-censorship groups in America. There's never been any insinuation that the Library of Congress was having its strings pulled by the forces in power. I trust the Library of Congress to be a neutral provider of information much more then, say, the Washington Post or the Encyclopedia Britannica.
I can see a lot of places (libraries primary example) that will no longer carry or supply this type of information, because the government will supply it to us.
Most libraries are part of government. Why should you trust your home-town library more than the Library of Congress?
I can't wait to read the old ads (Score:2, Interesting)
Re:I can't wait to read the old ads (Score:2, Interesting)
Disney's fault (Score:5, Interesting)
Lawyers and legal researchers (Score:2)
What were the jury members saying after the trial? Who were the witnesses and what was their standing in the community? How did the decedent's estate fare where the bastards claimed that they were not bastards?
Aside from the births and deaths, the property records will be very valuable.
Many of these documents are available in microform, but the actual value of the documents will be increased exponentially where the full text is searchable. At present the vast majority a
Connecticut Offers Something Similar (Score:2)
They have archives for the NY Times, Hartford Courant, LA Times, Wall Street journal, Washington Post, etc. While the archives don't go back too far (twenty years for some papers, six for the NY Times) it is nice to see governments offering citizens access to this information free of charge. I use it quite frequently, and with hope they can get funding for the historical New York Times service (whic
you don't have to go to local library or LOC (Score:5, Informative)
Yes its not online. we don't have the staff or money to put it online, pesently, but we are trying to put as much of our records online right now.
Anyway, you can check out the one I work for, and if you Live in Mississippi, please come by and check us out. We are open 6 days a week and are totally free.
http://www.mdah.state.ms.us/ [state.ms.us]
Finally (Score:4, Funny)
Re:Google (Score:5, Insightful)
Obituaries and marriage announcements, for one this. This stuff will be a gold mine for genealogists.
Re:Google (Score:5, Interesting)
I work as a research assistant, which involves a great deal of time going through libraries and copying old journal articles (and I get paid, too, can you believe that?)
Eight or nine months ago I was looking stuff up for my professor's book on the history of the death penalty in the United States, and she had me track down an article from the Hattiesburg (Miss.) American on an outlaw named John Long, who was hanged in Mississippi in 1870. No library in New England archives the Hattiesburg American--not even Harvard or the Athenaeum--so in the end I had to call the Hattiesburg Public Library and ask the librarian to make me a photocopy of that article.
(We had a hard time understanding each other--I had to spell out the name "John Long" because my Boston accent confused her. I had the same problem in South Carolina when I asked the gas station attendant what town I was in. It was Summerton, which she pronounced something like "Suhhhn't'n"--eventually she had to point to it on a map.)
Believe me, this project could save me a lot of backache and eyestrain. Looking through six months of the New York Times from 1899 on microfilm because some footnoter wasn't more specific than "late 1899" is no joke.
Re:Google (Score:5, Insightful)
That excerpt strongly implies the use of OCR, in which case the search engines probably won't require a substantial amount of time to index the archive.
On a related note, many historically memorable events occurred during the timeframe mentioned. These include the American Civil War, the Titanic disaster, and many others.
Re:Google (Score:2)
(courtesy The Onion [theonion.com], in case you haven't seen "Our Dumb Century")
Re:Google (Score:3, Insightful)
WWI!
Isn't it amazing that reporting on WWII is still under copyright?
Re:Google (Score:3)
I believe a lot of old films have already been lost, because tracking the current copyright holder is too expensive or simply cannot be done, but without their permission it is illegal to copy the old & decaying prints onto new media.
Re:Google (Score:5, Insightful)
Re:Google (Score:2)
I'm temped to mod you funny, but sadly I think you're serious. Obviously nothing important happened between the late 1800s and 1920, we should probably just ignore it all. I guarantee those 30M pages are more significant than half of google's 8B. Unless you think a person's blog with pictures of their cat and a review of the latest Dashboard Confessional album is important.
Re:Google (Score:4, Insightful)
What they named news at their time is what we call history right now.
Re:Google (Score:2)
Re:Google (Score:2)
Words. They had moved on from heiroglyphics and runes by then.
What Did They Write About In the 19th Century? (Score:3, Informative)
"
Presumably, everything you missed by not taking history.
In that timespan, the U.S. expanded to the Pacific; fought wars with Mexico and Spain; participated in World War One; prompted the formation of the League of Nations; built the world's largest railway network; invented the telegraph, telephone, electric light, and the airplane; developed mass production and the auto industry; produced inumera
Re:Google (Score:5, Insightful)
In general, the most interesting stuff is often the stuff which was the least interesting when the newspaper was published, such as advertisments, expressions and figures-of-speech in the articles, opinion pieces, the style of reporting, the biases.
All these little things that generally convey the atmosphere and mindset of an age. It's easy to find out facts, like the construction date of a factory. It's more difficult to find out what people were thinking about the new factory.
Re:Thank you (C)opyrights law (Score:3, Funny)
Not if someone patents the act of reading historical articles about black Monday!
Re:Thank you (C)opyrights law (Score:4, Informative)
Nope; everything from 1909 to 1922 is only in the public domain because it was grandfathered in in the Sonny Bono Copyright Extension Act. Newspapers that were published in 1929 will be in the public domain in until 1929+95 years. So in 2025 you'll be able to read about Black Monday.
Re:Scan it (Score:3, Informative)
Feist [cornell.edu] says that just effort doesn't a copyright make; it requires creative input.
Project guttenberg has their small print because of editing
Reread the small print. It's not a copyright license, it's a trademark license. If you remove the Project Gutenberg trademark from the etext, you can do whatever you want with it. (Assuming it's not one of the rare ones that's still under copyright, but the author gave the right post it.)
Re:some old newspapers are available online (Score:2)
Also the Christian Science Monitor and the Wall Street Journal. Boston Globe, Chicago Tribune, and Atlanta Journal-Constitution in a few months assuming they can get off their butts. All those materials are scanned from microfilm, split up, OCRed, put through some stuff I can't talk about, then shoveled into Proquest's database. You can search for words in a dat