Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
Technology

Computers Paraphrase English 212

Posted by michael from the me-talk-pretty-one-day dept.
AhaIndia submits a link to a story discussing computerized paraphrasing of English news articles. This technology, destined to eventually replace most reporters with very small shell scripts, is thankfully still in its infancy.
This discussion has been archived. No new comments can be posted.

Computers Paraphrase English More

Computers Paraphrase English

Comments Filter:

  • hmm... soudns familiar... (Score:4, Funny)

    by Dorothy 86 ( 677356 ) * on Friday December 26, 2003 @01:46PM (#7813642) Homepage
    This technology, destined to eventually replace most reporters with very small shell scripts

    This shirt? [thinkgeek.com]

  • Automated slashdot? (Score:3, Funny)

    by TwistedSquare ( 650445 ) on Friday December 26, 2003 @01:46PM (#7813644) Homepage
    This technology... thankfully still in its infancy.

    So one day instead of complaining against michael and co., everyone will be moaning about someone else's code - seems more appropriate for a nerd site somehow ;)

    • At least it can use a spellchecker. And it can probably catch dupes on occasion, too, with some work. I don't know what you'd be complaining about.
    • "This technology is thankfully still in its infancy."

      I think Michael misspelled 'unfortunately'. But what am I saying...god forbid we have a day when scripts take over Slashdot. Of course, they'd probably program them to dupe and put in random M$-bashing statements.

    • I don't see why you'd need any fancy AI or genetic algorithms to mimic the slashdot submitters. Most of them just copy+paste the first two or three sentences of the article, without adding anything. That could easily be replaced by a perl script in about 20 minutes.
  • Google news [google.com] already uses a similar technique to decide what to put in the summary beneath the headline, it does not paraphrase but it does actually extract a summary.

    Also if you have Microsoft Word [microsoft.com] lying about there is a feature called Auto-summary which is suprisingly good, amost as effective as going through a document yourself looking for the main points.

    • Re:Interesting use of Technology (Score:5, Informative)

      by Tim C ( 15259 ) on Friday December 26, 2003 @01:51PM (#7813666)
      I've provided search engine functionality to a few sites using Verity's K2 product, which provides a similar piece of functionality. If you (programmatically) ask it to return a summary of each hit, what you get is what it considers to be representative of the document as a whole, not merely the first few lines, or a paragraph, or whatever. It actually works pretty well, but then it should, as (a couple of years ago) it cost almost as much as my house...
    • Mac OS X users can select text and choose 'Summarize' from the Services menu in any Cocoa or Services-enabled Carbon application. Summarization is also available to any application programatically [apple.com] through the Find By Content API.
    • Hey, I went and played w/ this feature of word. Here is the summary of the article. hmmm... maybe if we set up an auto summary more people would RTFS?

      Anyway, here it is:
      Now, computers can play along

      Computers can't do nearly that well at paraphrasing. Now, using several methods, including statistical techniques borrowed from gene analysis, two researchers have created a program that can automatically generate paraphrases of English sentences.
      The program gathers text from online news services on specifi

  • fox_news.sh (Score:5, Funny)

    by sinclair44 ( 728189 ) on Friday December 26, 2003 @01:47PM (#7813648) Homepage
    #!/bin/sh curl $1 > paraphrase > slant -patriotic -stupid > fox_news_story.txt

    • Re:fox_news.sh (Score:5, Funny)

      by drakaan ( 688386 ) on Friday December 26, 2003 @02:00PM (#7813746) Homepage Journal
      perl makestory.pl -slant "liberal dem party-line" -severity "raving" -subject "Cheney Halliburton motives"

      Fair is fair ;)

      • Re:fox_news.sh (Score:4, Funny)

        by niom ( 638987 ) on Friday December 26, 2003 @02:43PM (#7813959)
        Fair is fair ;)

        Except when immediately followed by "and balanced".

  • Very small shell scripts (Score:5, Funny)

    by bunnyman ( 121652 ) on Friday December 26, 2003 @01:47PM (#7813650)
    Yes, but until it can post duplicate articles with slightly different phrases, it will never replace CowboyNeal!

  • But still.... (Score:5, Insightful)

    by AgBullet ( 624575 ) on Friday December 26, 2003 @01:50PM (#7813664) Journal
    won't you need someone to write the stuff to be paraphrased in the first place?? explain to me how that replaces reporters with small shell scripts.
    • I think the joke was directed at "reporters" like those found on /. They would be first in line to be replaced as their entire job seems to be to collect interesting articles and repost. Heck, the shell script would probably get the whole dup problem figured out too ;o)

    • Re:But still.... (Score:4, Insightful)

      by EvilTwinSkippy ( 112490 ) <yoda@nOSpAM.etoyoc.com> on Friday December 26, 2003 @02:07PM (#7813793) Homepage Journal
      There are reporters? Crap, every other article in my local fishwrap is Rueters, the other half is AP. There are one or two articles for local color, generally homicides or documenting yet more ways our local government is a) corrupt, b) inept, and/or c) playing partisen politics with/against the state goverment.

      By the time it's printed in the "News" its usually pretty old.

      • No Crap.

        It used to be that a reporter was there, got the facts and then got the feeling too. But I guess with all the information these days, there's no way people could do everything. Still, it's just going to seem unhuman when one day a news story is a few lines of XML:

        And the "reporter" is just a software program that turns that into a readable "story". Then you can choose how you want the news displayed, with various schema. Like if you're in a good mood, you can put in the happy schema and it doe
      • documenting yet more ways our local government is... inept

        They don't have to try very hard. I remember a couple of years ago the local news had a shot of a police car that had run into a postal truck. No comment necessary, just ten seconds worth of footage. That image is with me every time I vote.

    • ...not nec a problem to be solved by the code. Which BTW probably are a leetle more complex than small shell scripts, and see a good textbook like Jurafsky and Martin (pub 2000) for why.

      Re journalistic integrity - There's the possibility that a single entity could issue the release to the wire services, they could relase it in some kind of 'compiled' form (where it's just the syntax/semantic relations.) (How this could be different from how releases are issued now is a good question, but I guess there'd

    • Just imagine all /. users replaced by very small shell scripts and you've got it. I'm suspicious the that process was begun long ago.

  • The Ultimate Tool For Plagiarism (Score:5, Interesting)

    by popo ( 107611 ) on Friday December 26, 2003 @01:51PM (#7813672) Homepage

    All someone has to do now is marry this technology with a term-paper database, and "Hello Original Work!"

    The question will then become, how many different unique "paraphrases" can the system ultimately generate?

    • Actually you can use topic maps to decompose a body of work into individual statements and then use a set or randomly generated "flavors" to re-constitute the facts into an original work. The rules about what goes where are pretty cut and dry.

      More stuff to help people avoid shitwork, only for humanity to discover our purpose in life IS to do shitwork.

    • This isn't necessarily the big problem it appears. I've heard of many college professors and high school teachers using automated plagiarism detectors in the news, and that strikes me as stupid, as well. I mean, if a student has to write a paper on _The Bell Jar_, I'm sure he can find one online. But in most classes, you expect some level of familiarity with the students, on part of the teacher. If a kid who sleeps in every class and who's comments tend to be off topic or stupid turns in a paper worthy of T

      • Re:The Ultimate Tool For Plagiarism (Score:4, Interesting)

        by SurgeonGeneral ( 212572 ) on Friday December 26, 2003 @03:33PM (#7814224) Journal
        Yes, we've all heard the arguments against cheating.

        Especially the, 'you're only cheating yourself' one.

        Its irrelevant because this will not affect the way we cheat so much as the way we learn and the way we write. Think about it beyond your personal experience in high school.

        1. On the micro scale, an autosummerize feature like this will allow someone to take another's essay and put their facts into their own words. But I dont see how this makes any difference to the cheater other than saving him an hour. To see this tech as a problem on this level is to ignore the future.

        2. On the medium scale, it will allow someone to take multiple papers, extrapolate all the facts and their sources and then string them together again with their own interpretation. This will allow the learner to come up with a new argument and possibly a fresh insight based on the available information. In this case, it saves the learner a few hours of reading, though he has to do the same amount of thinking and logical reasoning. Is it a shame that the person doesnt have to waste time reading irrelevant information? Still, looking at it on this level is not thinking very deep.

        I take history in university and the essays we have to write are done by data mining books. Lots of books. We have to read large amounts of material in as short a time as possible. We have to find out what is important and what is relevant. Am I really learning how to analyze facts? I dont think so. I am learning how to write university papers and theorize based on incomplete information. I am learning how to make a lot of wasted time look like a lot of work.

        3. The macro scale. What if every book ever written was replicated in full electronically and available for parsing. What if I could extrapolate every fact from every source even remotely relevant to a topic. I'm right back to where I was before : hours and hours of reading. Yet, my argument will be more solid and my information more complete then it ever could be using the outdated method of data mining: looking in the indexes of books. In this case, what am I learning? I am learning how to think. I am learning how to spot holes, inconsistancies, fallacies, and etc. In this case the technology has eliminated cheating altogether because there is no single source to copy from. And if I want to understand how all these facts are related to each other I either have to think about it or read an other authors interpretation of it. (thus I could still cheat in the classical sense)

        4. But lets look at it on one more level, the very tiniest level and the most futuristic. A well constructed paragraph or sentence cant be parsed down, and wouldnt make sense if it was. The facts contained in a paragraph only become important in relation to one another. So in the end, it could just change the way we write. Enough with this puffed up crap, enough with padding your papers - either state whats important or nothing at all. A well constructed essay in the future will be one that cant be "autosummerized" without losing all its intelligability.
        • Your points all are valid, I think, and actually very interesting, but I don't think this means there's no such thing as cheating. If you plagiarize, the issue is not the text you copied, but the ideas. So yeah, perhaps this system would allow us to actually crack down on plagiarists, as well, by detecting copied ideas, even if restated. But I suspect we might find that a lot of papers aren't really as original as the authors thought.
    • And, if this technology is sufficient to write good term papers based on online information, what is the point of learning to write term papers? Certainly any students who have access to such technology will have no use for doing it themselves after school, when the technology will be more advanced and more money will be available for it.

      At that point, teachers ought to be teaching students how to get such software to produce the effect they want on the audience. For that matter, they could try teaching an
      • if this technology is sufficient to write good term papers based on online information, what is the point of learning to write term papers ... teachers ought to be teaching students how to get such software to produce the effect they want on the audience.

        Since it's so comfortable one wonders why the baby ever leaves the womb.

  • Just like the T-shirt says [livejournal.com]

  • Dupe (Score:5, Informative)

    by greenhide ( 597777 ) <jordanslashdotNO@SPAMcvilleweekly.com> on Friday December 26, 2003 @01:55PM (#7813707)
    Unfortunately, there isn't yet a way to use computers to detect dupes [slashdot.org].

    Or Is there?!? [google.com]

  • School Reports (Score:4, Insightful)

    by gregfortune ( 313889 ) on Friday December 26, 2003 @01:56PM (#7813709)
    So, will there be difference between paraphrasing and copying now in an educational setting? Seems like this could make a report pretty easy...

    1) Brainstorm some key points/ideas
    2) Have this program data mine for relavent articles online
    3) Feed sections of each article into the program and have a finished paper

    Granted, the tech isn't quite that powerful yet and probably wouldn't do a whole paper, but it sure looks like it could supply several paragraphs of material per page...

    • Re:School Reports (Score:2, Interesting)

      by roninmagus ( 721889 )
      I do very much hope so; as a computer science major who hhaaatteess general studies classes, I hope very much that the English/History classes which so graciously waste my programming time with useless writings go down the drain. Of course, my website [daveandrews.org] is entirely such useless writings, so I stand trumped.

      However, I did meet my girlfriend and hopefully future wife in Sophomore English at MTSU. Go figure.

  • Rethink English ! (Score:4, Informative)

    by Thinkit3 ( 671998 ) * on Friday December 26, 2003 @01:58PM (#7813728)
    Lojban is among the more interesting newer languages. It can be parsed just like c! Esperanto is somewhat interesting. English will be regarded in the future as a curious artifact--it was swept along with the technology revolution simply because ASCII didn't include accents and extra marks on letters. Eventually we'll get away from vocalization all together and have purely numerical, written laguages.

    Right now, trying to work with English in computers deals way more with the strangeness of the language than the more interesting issues of cognition that lie underneath.
    • English will be regarded in the future as a curious artifact

      One man's informative is another man's troll... Esperanto was interesting and look where it got. Nowhere. People will speak in what's easiest. English is becoming a de facto standard that will continue to be the most spoken language in the world. People won't use odd designed languages because it will be harder than current languages, which got where they are today though iterative refinement to be the best suited language for us to communi

    • Right now, trying to work with English in computers deals way more with the strangeness of the language than the more interesting issues of cognition that lie underneath.

      That's true. Computer languages that don't stick close to "regular" human expression are very popular [cloud9.net] and growing quickly. Languages that resemble written English [python.org] are dwindling rapidly.

      After all, code is meant to be written, not read [ioccc.org], and programmers should strive to write such that their work can't be understood [unsw.edu.au] by anyone not an expe

  • Fake literature (Score:4, Funny)

    by MAPA3M ( 718897 ) on Friday December 26, 2003 @01:59PM (#7813734)
    Isn't this the way those trashy love novels are written?

    • Not quite, but very similar [amazon.com]. That is by far the most stunningly dumb book I came across in my stint working at a bookstore.
    • Isn't this the way those trashy love novels are written?

      Wow, it must take a hugh shell script to turn "Mary wanted the big strong muscle man to solve all of her problems. Henry, the big strong muscle man, was horny. They had unprotected sex. Everything was perfect from then on."

      Imagine what that could to to a Hello World program.

      LK

  • Or games... (Score:3, Funny)

    by A55M0NKEY ( 554964 ) on Friday December 26, 2003 @01:59PM (#7813736) Homepage Journal
    Someone set up us the bomb!
  • ...most reporters with very small shell scripts...

    I know I heard this phrase (loosely) before, but does someone know the name of the reference?
    • ...most reporters with very small shell scripts...

      I know I heard this phrase (loosely) before, but does someone know the name of the reference?


      At ThinkGeek [thinkgeek.com] perhaps?
      Or one of myriads of signatures quoting this?

      Regards,
      --
      *Art

  • Replace reporters?? (Score:2, Insightful)

    by dyj ( 590807 )
    How is this going to replace reporters? Reporters don't just paraphrase other reports. They actually are supposed to search for stories (hopefully factual!) on their own.
    • I think someone just wanted Journalists to know what it feels like to be a tech in this day and age. What they can't get a computer or a trained chimp to do, they will find some guy in another country who will do it cheaper.

      We know we will be in trouble when every commentary article begines with "I am thinking that..."

      • Until you can get computers to drink on the job, get paid by businesses to write advertisement for them (a la Enron, CART, etc), and fall for any buncomb that someone says in a serious voice, then they won't exactly 'replace' journalists.

  • Something similiar existed on the Amiga (Score:4, Interesting)

    by Serk ( 17156 ) * on Friday December 26, 2003 @02:03PM (#7813768) Homepage
    Back in the late 1980's I had a word processor for my Amiga that had a function whereby it would do a global search and replace of every Xth word (User settable) with a synonym from the built in Theasarus... Very handy for those term papers I so hated in high school...

    I'm assuming this (Of course I didn't RTFA) is far more advanced than what we had back then, but the idea for this has been around for quite a while at least...
  • AhaIndia submits story discussing paraphrasing of articles. This technology, destined to replace reporters shell, is still in its infancy. Huh, perhaps we'll still need humans after all . . .

  • Do you know what reporters DO? (Score:5, Insightful)

    by DavidinAla ( 639952 ) on Friday December 26, 2003 @02:08PM (#7813802)
    For you to say that this technology will someday replace reporters makes me think that you're clueless about what reporters do. Do you realize that the biggest parts of a reporter's job are gathering facts and making judgments about 1) which stories are worth reporting, 2) which are the relevant facts about a story and 3) who's lying and who's telling the truth about a story? The actual writing that you see is many times almost incidental to most of what a reporter does. You might not like the judgments that a reporter makes (and I could agree with that in many cases), but software can't go out into the world and talk to people and use judgment and intuition to find information to write about.

    As an ex-reporter and editor, I find it laughable that anyone might think this technology will replace reporters. It's sort of like suggesting that machines that can read source code and interpret it can somehow figure out what new software people want and then write it. Both possibilities are equally insane.
    • Hey, don't troll this stuff out quite yet - sure it's future ware right now, but think ahead, and ... more to the point, read some about it. There's more to language and computational linguistics than you might think. Just because your (former) line of work stands to be partially replaced doesn't mean that the technology is insane.

      to wit, there are attributes of register, tone, and modality that can be applied not just to individual sentences, but to entire pieces of text that may be able to indicate a

      • Maybe you're not clear about the difference between a reporter and an editor.

        It's theoretically possible that an editor could be replaced in some instances by software, but not the reporter. The reporter doesn't have anything to start with -- no sentences for software to analyze. A reporter normally starts with some vague thing like a source in the city clerk's office telling him that some bogus expenditures are being put into the sanitation department budget for next year, but nobody really knows what's
    • The fact is that this technology is just replicating what people are already doing. The technology won't replace many reporters. It will replace the cut-n-paste people who have already replaced the reporters. Most "news organizations" are nothing more than AP chop-shops right now. You think there's a lot of fact-checking and analysis going on? I wish that were true, but it's just not.
    • DavidinAla asked:
      Do you know what reporters DO?
      This is slashdot.

      This is Michael, Hemos, and Taco we're talking about. What kind of dumbass question is that?!

      Of *COURSE* they don't know. Heh, even avoiding dupes, spellcheck and fact checking are alien concepts...

    • "which stories are worth reporting"

      With this technology, ALL of the stories could be reported.

      "which are the relevant facts about a story"

      odd, I myself get very pissed about reporters who don't give ALL the facts. If you mean summarizing, that is EXACTLY what this is supposed to do.

      "who's lying and who's telling the truth about a story"

      That's for the reader to decide. A reporter who makes judgements concerning what they are reporting and expresses their view of the subject is a bad one. At least in
      • I'm sorry, but you're SO ignorant about the way the process works that I can't begin to correct all of your misunderstandings. If you really and truly believe that it's even possible to give readers ALL of the available information every single day, you're completely unaware of how much information is out there.

        Do you want to report what is on the menu at every restaurant in town every day? What about an attendance list of who made it to school at every school in town? What about the results of every medic
  • HeySubcontinent's story linkage analyzes the automatic stegoplagarization of documents written in the language derived from Britain. Expected to displace at some point journalists, these hacks presently bash with the force of a small child. Good.

  • Someone must research a story . . . (Score:5, Interesting)

    by kfg ( 145172 ) on Friday December 26, 2003 @02:11PM (#7813812)
    conduct interviews and generate original copy. These people are called reporters.

    The people who take this copy off the wire and paraphrase it for publication in the local paper are called copy writers.

    This software will reduce the number of copy writers needed, not reporters.

    This is certainly an issue to the copy writers and their families, but overall it's really just a blue collar worker being replaced by a robot issue.

    The idea of a 'style dial' I find a bit more disturbing.

    KFG

  • Generation isn't that easy (Score:5, Insightful)

    by Ezubaric ( 464724 ) on Friday December 26, 2003 @02:13PM (#7813820) Homepage
    The poster incorrectly assumes that this could be used to replace reporters. The problem is that computers have a difficult time generating new text. The methods that computers use to evaulate text (as any user of grammar-check would realize) aren't that great.

    In fact, most language models cannot generate even a large portion of English text. Those that do have a good range rarely have good accuracy, because there are many things that we "just don't say that way." This is why when you're talking to a non-native speaker, you often cannot explain why something they said was wrong. This is because there is no real grammar rule against speaking in a given way.

    So if we rule out syntax-based models, that just leaves statistical-based models. I worked in a NLP lab during the summer of 2002, and my prof there said that syntax and statistics are like the two sides of the force. Statistics are quick and easy but are seductive. They corrupt you and leave you unable to really think about the language itself. You only think in terms of bigrams and HMMs.

    So even though these systems are doing well, they are mostly statistical. Thus, it's hard to get incremental improvement. You have to have larger corpora, and larger corpora usually have more errors, thus defeating any advantage you might get by capturing more aspects of a language.

    In my opinion, only with well-developed language models that can effectively generate NL can we get anywhere. Which is what Barzilay is working on, but it's still a long, long, long way off.

  • The article, summarized by MacOS X (Score:5, Interesting)

    by sakusha ( 441986 ) on Friday December 26, 2003 @02:19PM (#7813856)
    MacOS X has a summarization feature implemented in the Services menu. I decided to summarize the CNet article just to see what I got, and because I like the idea of summarizing an article about summarizing.
    In the famous sketch from the TV show "Monty Python's Flying Circus," the actor John Cleese had many ways of saying a parrot was dead, among them, "This parrot is no more," "He's expired and gone to meet his maker," and "His metabolic processes are now history."

    ...The program gathers text from online news services on specific subjects, learns the characteristic patterns of sentences in these groupings and then uses those patterns to create new sentences that give equivalent information in different words.
    The researchers, Regina Barzilay, an assistant professor in the department of electrical engineering and computer science at the Massachusetts Institute of Technology, and Lillian Lee, an associate professor of computer science at Cornell University, said that while the program would not yield paraphrases as zany as those in the Monty Python sketch, it is fairly adept at rewording the flat cadences of news service prose.

  • Hardly news... (Score:4, Informative)

    by JayJay.br ( 206867 ) <100jayto@g m a i l.com> on Friday December 26, 2003 @02:22PM (#7813867)
    This article [slashdot.org] posted before already tells us all this, the paper that originated it [mit.edu] was mentioned in the comments, and this one is another of a series of papers by this researcher [mit.edu].

    OK, nothing else to see here, move on to the next redundant post (Is that paraphrasing 'dupe'?)
  • ...that explains cowboyneal
  • For your convenience, here's the link to the original article that requires registration [nytimes.com].

  • Bring on the Machines (Score:5, Interesting)

    by DumbSwede ( 521261 ) <slashdotbin@hotmail.com> on Friday December 26, 2003 @02:39PM (#7813936) Homepage Journal
    I don't think many people read the article. While Michael suggest this could replace reporters, it is not about summarizing a whole article, but merely paraphrasing individual sentences and elements. This would be useful for checking for plagiarism where one author has merely line by line paraphrased another. Another useful area is in language translation, where the paraphrasing may make the translation more understandable. I don't think todays translation programs allow you to say the the same thing two or three times, but repeat it back differently (paraphrase) if not understood by your listener the first time.

    Of course the time will come when machines summarize articles, and I believe I have seen where this has already been tried with mixed success. It would be kind of neat to see /. use both a summary engine and a paraphrase engine on submitted articles. Then we could have 3 article descriptions: the posters description; a machine summary of the same article; and a machine paraphrase of the original posters summary.

  • Paraphrase (Score:2, Insightful)

    by JediDan ( 214076 )
    Would be nice to be able to summarize + paraphrase large articles and documents. Not all of us have the necessary time to read 20+ page documents.

    It won't replace original works, but it could help reduce a lot of extraneous data on the web :)
  • Slashdot needs to implement another new editorial policy: if you have nothing intelligent or really funny/biting to say, don't! An interesting topic with a another half-assed presentation.

    Obviously this is a developing field. The best models seem to use phrases from the original text, anyway the Mac OSX example above shows that it is useful to users willing to take it with a massive grain of salt, even if we are not into full computational sentience yet.

    When it works even a little better it will replace all those awful grade school teachers who assign paraphrasing as a homework assignment. The reporters who might have been replaced by it will have already lost their jobs, except for the ones in AhaIndia of course who will paraphrase for the rest of us, usually at a marginally better level than the machine.

    The research is interesting - and I'd like to understand Barzilay's notation is that APL or calculus of statement? - in the paper (pdf) [jhu.edu] I found on google. Also see the papers on her site [mit.edu].

    Of course structured text is easier, and news stories are known to have most of the meat in the beginning, but this is great stuff.

    One interesting older system is ThoughtTreasure [signiform.com] which was built to understand a story and answer questions about it. The author also did work on news analysis ("NewsForms") too. There are tools out there, I've been making a survey myself too. If anyone has information about practical NLP tools for real world tasks please post.

  • It's unlikely to catch on... (Score:3, Insightful)

    by ChunKing ( 513714 ) on Friday December 26, 2003 @02:56PM (#7814023)

    The main problem is that languages, especially English, are so idiomatic that mechanical translators will be a too much of a disadvantage - take the Babelfish [altavista.com] translator for instance.

    Furthermore, the English language is so flexible that just about any word can arbitrarily substitute for anything else - for instance, take 'bad' meaning 'good'.

    It would be impossible to program a machine to be able to understand the full spectrum of idiomatic phrases but the future may lie in employing neural net technologies so that computers can do some limited learning. [cornell.edu]

  • Columbia News Blaster (Score:3, Informative)

    by Richard Allen ( 213475 ) on Friday December 26, 2003 @03:04PM (#7814077)
    I believe this was covered in a related Slashdot before regarding to this site: http://www1.cs.columbia.edu/nlp/newsblaster/

    Here is a quote from their site:
    Columbia Newsblaster is a system to automatically track the day's news. There are no human editors involved -- everything you see on the main page is generated automatically, drawing on the sources listed on the left side of the screen.

    Every night, the system crawls a series of Web sites, downloads articles, groups them together into "clusters" about the same topic, and summarizes each cluster. The end result is a Web page that gives you a sense of what the major stories of the day are, so you don't have to visit the pages of dozens of publications.

    Newsblaster is an academic project from the Natural Language Processing group at Columbia University's Department of Computer Science. It is designed to demonstrate the Group's technologies for multidocument summarization, clustering, and text categorization, among others. It is funded under DARPA TIDES and KDD and has been operational online since September 2001.

    Current and future enhancements include international perspectives, multilingual capability, and tracking events across days.
    • Yeah, I saw a demo of this, and was pretty impressed, though eventually I decided that paraphrasing wasn't nearly as interesting as simply identifying the big news... so I went back to Google News. Now I just read CNN, the New York Times (print), and Slashdot, and I figure that between the three, everything's covered.
  • Silliness aside about an Apple 2 being able to gather the news for us and feed it, the thing about wordsmithery is that there is a certain amount of creativity that needs to go into it. Otherwise you have the literary equivalent of the Backstreet Boys and such. Not a good mix.

  • I'm a sexist, so what? (Score:3, Funny)

    by Lord Kano ( 13027 ) on Friday December 26, 2003 @03:24PM (#7814183) Homepage Journal
    From the article:
    The researchers, Regina Barzilay, an assistant professor in the department of electrical engineering and computer science at the Massachusetts Institute of Technology, and Lillian Lee, an associate professor of computer science at Cornell University, said that while the program would not yield paraphrases as zany as those in the Monty Python sketch, it is fairly adept at rewording the flat cadences of news service prose.

    Two women came up with this! Why doesn't it surprise me in the least that women are officially researching ways to automate the process of saying the exact same thing in an infinite number of different ways?

    LK
  • Now, correct me if I am wrong, but hasn't Hollywood beem using this system for some time now? If a movie isn't a direct rip of something that was made in the past, then it takes familiar characters and tosses them in a blender with a dash of CG effects and frappes until smooth.

    Television uses this system, too. The formula there seems to also involve borrowing a successful British TV show's concept, just to keep things a little fresher.
  • Comment removed based on user account deletion
  • Slashdot has been using this system to generate its articles for a while now. Obviously it's still loaded with bugs.

    -

  • Wake me up when... (Score:3, Funny)

    by Megane ( 129182 ) on Friday December 26, 2003 @09:35PM (#7815523)
    ...when we can replace upper level management with small shell scripts.
  • Boy, I'm glad that computers don't have their (hands?) in reporting news; it'd be terrible to get rid of all that slant in the media this way and that. I mean who wants fair, equitable stories?! You read the NYT to ra ra for the Bleeding Heart shit, or if you're a heartless republican the Journal is for you. Now how would they sell if they just told the facts as they were and left interpretation up to the readers?!

    Well, at least Slashdot will always be biased, thank god for that.
  • ... (in the Services menu) to summarize the referenced article:

    The program gathers text from online news services on specific subjects, learns the characteristic patterns of sentences in these groupings and then uses those patterns to create new sentences that give equivalent information in different words.

    The researchers, Regina Barzilay, an assistant professor in the department of electrical engineering and computer science at the Massachusetts Institute of Technology, and Lillian Lee, an associate prof

Slashdot Top Deals

Every cloud has a silver lining; you should have sold it, and bought titanium.

Close