Forgot your password?
typodupeerror
Software Technology

Software Finds Plagiarism In Research 111

Posted by CmdrTaco
from the grant-revoked dept.
shmG writes "Researchers from the Virginia Bioinformatics Institute have created a seek-and-destroy program — for plagiarism. Called ET Blast, it's designed to find plagiarism in scientific papers. It does a full-text analysis, and then looks for similar publications in several databases. 'We have better literature,' Garner said. 'There are abstracts and full papers, and a database called Crisp, where you compare stuff to every grant the NIH gets. It's compared to any research that's been funded.'"
This discussion has been archived. No new comments can be posted.

Software Finds Plagiarism In Research

Comments Filter:
  • What about ... (Score:4, Interesting)

    by gstoddart (321705) on Wednesday October 27, 2010 @10:40AM (#34037268) Homepage

    What about academic "recycling".

    I remember being told a long time ago that some researchers will basically make several permutations of the same paper to submit to a bunch of different places. It's essentially the same paper, with nothing new in it, but if you can get several places to publish it, you can pad out your publications list.

    • Re:What about ... (Score:5, Insightful)

      by notgm (1069012) on Wednesday October 27, 2010 @10:42AM (#34037292)

      if you resubmit your own work, it's not plagiarism.

      • if you resubmit your own work, it's not plagiarism.

        Correct! It's amazing to see how many people don't understand this point, but it's correct: you can't plagiarize yourself, because plagiarism is the act of passing somebody else's work off as being yours.

        I hate it when researchers report the same work in many different papers, but although it is a violation of research reporting standards, and in some cases a violation of an intellectual property contract... it's not plagiarism.

        • by Travelsonic (870859) on Wednesday October 27, 2010 @10:58AM (#34037528) Journal
          In High School, they tried to cram the concept of "self plagiarism" down our throats - what a crock of shit... you can NOT by DEFINITION plagiarize YOUR OWN WORKS. Recycling may be lazy, may violate other ethics, but to call it plagiarism is, IMO, very intellectually dishonest of these institutions.
          • Re: (Score:3, Informative)

            by Anonymous Coward

            I actually ran into this in grad school. When writing a tech related paper, I referenced one of my past papers on the same subject as a source. My professor made it clear I had to cite myself to avoid "self-plagiarism". I thought it quite possibly the stupidest thing I had ever heard in my life, and it was coming from a celebrated PhD at a major New England university.

            • The reason for the rise of the concept of "self-plagiarism" is these types of automated plagiarism detectors. If I have written a lot of papers that are in their database and can lift sections out of a previous paper I wrote without citing it as a source, these programs are going to generate a lot of false positives.
            • by pentalive (449155)
              It does help others find your previous work.
          • RECYCLOPS will make you recycle!
        • by Lucky75 (1265142)
          Tell that to my university where I got accused of academic dishonesty for reusing one paragraph again in a course that I failed. Utterly ridiculous.

          "Okay, I give myself permission to copy work from myself....there.....now it isn't plagiarism."
          • by Lucky75 (1265142)
            To follow my last post, I usually like to use the following argument: If I'm asked what the answer to 1+1 is, I'm going to answer '2'. I'm not going to say that the answer is '3' next time just to make my answer different.
            • To follow my last post, I usually like to use the following argument: If I'm asked what the answer to 1+1 is, I'm going to answer '2'. I'm not going to say that the answer is '3' next time just to make my answer different.

              You're just not being creative enough. You can come up with a different answer, for example "1+1 is 1.999..." or "1+1 is 1, for sufficiently large values of 1" etc.

      • by tmosley (996283)
        I'd hope not. For most of the papers and grants coming out of our lab, we use the same introduction, and many of the procedures are the same, so many sections are just cut and pasted with a few words changed here or there to fit the particular experiment or project we were working on. It would be a major pain in the ass if we had to start rewriting that crap every time.
        • by gstoddart (321705)

          I'd hope not. For most of the papers and grants coming out of our lab, we use the same introduction, and many of the procedures are the same, so many sections are just cut and pasted with a few words changed here or there to fit the particular experiment or project we were working on.

          Oh, I get that you may do a series of experiments all with some commonality. That is fine.

          I'm specifically talking about people who essentially recycle the same paper several times with no material changes to any of the resear

      • if you resubmit your own work, it's not plagiarism.

        It is, however, fraud in most cases, since most scientific journals require that papers submitted to them be research that is unpublished and not currently submitted for publication elsewhere.

      • Because, because beyond certain point of recycling, it's just dishonesty.
      • Re: (Score:2, Informative)

        by robotkid (681905)

        if you resubmit your own work, it's not plagiarism.

        Let me clarify the issue for those not accustomed to the rules of scientific publishing.

        There IS a thing as self-plagarism, and it's not necessarily a minor offense. At it's core, if you submit essentially the same work to multiple venues with the intent to pass each off as an independent body of work when they are not, then there is intent to deceive and that is an ethical breach of conduct. Worst case scenario, the author list and abstract has been changed just enough that it leads others to believe th

      • Most publications are group work. Maybe the first author wrote the entire work without input, using only the results of others. And maybe every other author made significant changes or critiques. Those words can't be reused – unless they include every previous author in the new list. Reuse an introduction a few times and the author list is going to get pretty long. Anyway, it is copyright violation to use previously published phrases and images in a publication for a different publisher. That is

    • by notgm (1069012)

      rewriting your own articles isn't classified as stealing.

      • Nor is plagiarism - plagiarism is fraud. The idea that it is more than that, IMO of course, is built on a fallacy driven need to drill into the heads of minds [like myself] about the seriousness of it. I can't help, however, in thinking that they have gone overboard and instead of effectively teaching us away, they seem more bent on SCARING THE EVERLOVING SHIT out of us and that, IMO, does NOT help in understanding ANY material/concept/etc.
    • by PvtVoid (1252388)

      I remember being told a long time ago that some researchers will basically make several permutations of the same paper to submit to a bunch of different places. It's essentially the same paper, with nothing new in it, but if you can get several places to publish it, you can pad out your publications list.

      So what? You can't plagiarize yourself. Researchers put out multiple, nearly identical papers all the time, especially those published in conference proceedings. (For example, this guy [stanford.edu] just go elected vice president of the American Physical Society.) It's also very common to recycle review material from one paper you have written to use in another.

      This is entirely distinct from university academic misconduct policies which require papers and so forth submitted in fulfillment of course requirements to

    • When you publish in a scientific journal you hand over the copyright to the work. Therefore if you publish those results again, without citing the previous publication, you can be sued for copyright violations by the original publishing journal. I've heard of authors being banned from a specific journal for such behavior. It is not "Plaigarism" in that you are not taking credit for someone elses work, but it is academic dishonesty of the sort that can ruin a career.

      For example, my advisor gave a (Revi
    • In the dejavu subsite [vt.edu], most publications share an author, so yes, it does "recycling" / self-plagiarism.

    • by Phoghat (1288088)
      It's not plagiarism it's bullshit
  • Would be nice to widen it to IP & Copyright infringement.

  • by mlts (1038732) * on Wednesday October 27, 2010 @10:42AM (#34037300)

    This sounds almost exactly like turnitin.com where when one uploads a paper to it, it searches almost anything it can get ahold of and will list any text in any academic journal that is copied verbatim.

    • Re: (Score:1, Informative)

      by Anonymous Coward

      Those cunts @ turnitin archive *YOUR* paper for eternity (without payment and without any course for redress) to achieve network effects and enhance their service.

    • by robotkid (681905)

      This sounds almost exactly like turnitin.com where when one uploads a paper to it, it searches almost anything it can get ahold of and will list any text in any academic journal that is copied verbatim.

      An apt analogy. Imagine the following scenario: you are simultaneously enrolled in a two classes that both require a lengthy essay which constitutes a large portion of your final grade. You find the two assignments to have similar enough parameters and decide to submit the same essay to both teachers without any prior approval for the double-dipping, thus making it appear you have spent more effort than you actually have. You are only "plagarizing yourself", so no harm, right?

      Doubtful.

      Self-plagari

      • Re: (Score:2, Interesting)

        by redbeard55 (1002526)

        There is no harm you have done the required work. Just because you can use your work in more than one place doesn't harm anyone. Assertions to the otherwise are ridiculous.

        • by arkenian (1560563)
          Not true. A college credit is awarded based on the amount of work you've done in a given subject area. If they're giving you two credits for one credit of work, that's just as wrong as if I, as a government contractor, bill my time twice because a paper I wrote applies to two projects. Its not me using the work on both projects that's wrong, its having my customer pay me twice for the same work. This is about making it harder to cheapen the degree.
    • If you want to know the difference between this and turnitin, you'd have to read the article, it specifically mentions a few differences...

  • Even better if it will show papers that are suspiciously similar to pharmaceutical companies advertising literature.

  • Since researchers constantly plagiarize their own work in order to get their paper count up, there are going to be some very red faces....
    • Re: (Score:3, Funny)

      by noidentity (188756)

      Since researchers constantly plagiarize their own work

      Is this where the author of something passes it off as his own? I agree, that's a terrible thing.

  • I wonder, how is the false positive / false negative rate? I mean, places like turnitin.com for example shows this problem quite well with regards to how even quotes - cited and all - raise some flags.
  • by Zontar_Thing_From_Ve (949321) on Wednesday October 27, 2010 @10:52AM (#34037432)
    I can't blame the submitter for this one. The article itself uses the term "search and destroy" early on, yet says absolutely nothing about destroying anything.
  • by Anonymous Coward

    They found a research paper on hydrogen stole 2 thirds from an existing paper on water.

  • by onionman (975962) on Wednesday October 27, 2010 @10:55AM (#34037484)

    I once had an English teacher who said, "If you have more than five consecutive words matching a source, without a citation then it's plagiarism." Perhaps that's how freshman writing assignments are graded, but it's silly when applied to scientific papers. Pick up any math paper on number theory, and you're bound to find the sentence "Let p be an odd prime number." without citation, but that would hardly qualify as plagiarism. Yet, syntactic matching appears to be exactly what this program is doing.

    What constitutes "plagiarism" in a scientific paper is very different from plagiarism in journalism or English literature. In scientific writing, it is expected that authors will use the same flat, impersonal style and repeat definitions and the results of others to save the reader the time of having to look them up. So, simple pattern matching between science papers will result in a great many false positives. In science (and math) writing what matters is the new result which the author is claiming. It seems to me that it would be nearly impossible for a computer program to detect the distinction.

    • > I once had an English teacher who said, "If you have more than five consecutive words matching a source, without a citation then it's plagiarism." Perhaps that's how freshman writing assignments are graded, but it's silly when applied to scientific papers.

      No. Just... no. It is not "silly," it is insulting, in either freshman english lit or scientific papers. Any teacher who defines plagiarism that way has a lot more to learn than he has to teach.

      • > I once had an English teacher who said, "If you have more than five consecutive words matching a source, without a citation then it's plagiarism." Perhaps that's how freshman writing assignments are graded, but it's silly when applied to scientific papers.

        No. Just... no. It is not "silly," it is insulting, in either freshman english lit or scientific papers. Any teacher who defines plagiarism that way has a lot more to learn than he has to teach.

        Perhaps so, but I could see where such a rule could come from, and it could instill a discipline of making sure things are properly cited. Without any other context, obviously the rule is rubbish, but I could see it as an excellent rule to live by when taking freshman courses in writing/composition.

        • by Oxford_Comma_Lover (1679530) on Wednesday October 27, 2010 @11:39AM (#34038162)

          > Perhaps so, but I could see where such a rule could come from, and it could instill a discipline of making sure things are properly cited. Without any other context, obviously the rule is rubbish, but I could see it as an excellent rule to live by when taking freshman courses in writing/composition.

          But that's half the problem. The rule may come from a desire to instill discipline, but it's just a bad rule, because it teaches that plagiarism of ideas isn't plagiarism at all, and that stringing five words together in a way that's been used before is, and that rewriting something in your own words makes it no longer plagiarism.

          Demand students live by a childish rule, and you will at best be someone they have to ignore as they try to actually learn things.

          • by jvkjvk (102057)

            because it teaches that plagiarism of ideas isn't plagiarism at all, and that stringing five words together in a way that's been used before is, and that rewriting something in your own words makes it no longer plagiarism.

            While I agree with your general premise about childish rules... Just no.

            Plagarism is taking someone elses words and claiming them as your own.

            You seem to be infected by the IP bug.

            Fortunately for the rest of us, one cannot plagarize ideas. Reformulating a concept in your own words does not count as plagarism, nor should it.

            Regards.

            • by AlecC (512609)

              There is a grayer area than that. If I rewrite your book, with a paragraph-by-paragraph correspondence, the same plot, the same characters with names and appearances slightly changed, it is still changed. A book callel Earl of the Rings, about a hibbit from the Shaw taking a broach to be destroyed in Mt Gloom would probably be plagiarism (unless it changed enough to become parody).

            • You seem to be infected by the IP bug.

              Fortunately for the rest of us, one cannot plagarize ideas. Reformulating a concept in your own words does not count as plagarism, nor should it.

              You seem to be infected by a different sort of IP bug.

              Plagiarism is not the same thing as copyright infringement (though it's not uncommon for the same act to involve elements of both). One can plagiarize public domain sources. One can plagiarize ideas.

              Plagiarism is what happens when a writer presents other people's work (their words or their ideas) as his own, without giving due credit to the source. Pretending that you thought of something when you're actually just copying another author's reasoning is intellectual dishonesty, and squarely within the realm of plagiarism.

              If you copy someone's words verbatim, there is an added obligation to specifically identify the copied passage by blockquoting, using quotation marks, or otherwise clearly setting off the passage from the rest of your writing. If you're just paraphrasing, there's no obligation to use quotation marks (that would be silly) but there remains a need to properly name your source (through footnotes or other means). Rewriting someone else's work in your own words is otherwise still very much plagiarism.

    • by pz (113803) on Wednesday October 27, 2010 @11:16AM (#34037754) Journal

      Furthremore, when a scientist has spent a number of years on a long-term research plan, the condensed versions of what he is studying become so well rehearsed that it gets memorized. I have stock phrases that I use when I want to describe this or that aspect of my work because, after giving dozens of presentations about it, they are the ones that work best. They are the most highly polished and refined. They communicate the idea well. And so, they often get trotted out with every manuscript or grant application. My students and post-docs learn to use the same phrasing because, flatly, it works.

      None of the instances of those phrases or full sentences require attribution because they are all from the same motherspring of thought. We are the writers. And, as you might imagine, this might well produce a raft of false positives to a system that blindly compares text.

      • It makes a feature vector of the text in its entirety, then computes the "distance" between any two vectors. This distance is computed in some large dimensional space I believe not necessarily using a Euclidean metric. If the "distance" is below some threshold the papers are suspect Id imagine.
    • by careysub (976506)

      ... Yet, syntactic matching appears to be exactly what this program is doing.

      What constitutes "plagiarism" in a scientific paper is very different from plagiarism in journalism or English literature. In scientific writing, it is expected that authors will use the same flat, impersonal style and repeat definitions and the results of others to save the reader the time of having to look them up. So, simple pattern matching between science papers will result in a great many false positives. In science (and math) writing what matters is the new result which the author is claiming. It seems to me that it would be nearly impossible for a computer program to detect the distinction.

      Hours of speculation and typing can save one minute of reading TFA. From the article:

      "Unlike other plagiarism detectors, it does not use phrases or similar words to check for copying. Helio Text actually looks at the entirety of the text."

      So no, it does not. It uses instead some sort of similarity metric computed from analyzing the entire text. This is possibly similar to the text distance metrics used in vector space search engine models (see: en.wikipedia.org/wiki/Vector_space_model ). They will be publi

      • That makes sense. So it constructs a feature vector for each text and computes their distances relative to eachother. If there is a distance below some threshold then the papers are suspect.
      • by onionman (975962)

        ... Yet, syntactic matching appears to be exactly what this program is doing.

        What constitutes "plagiarism" in a scientific paper is very different from plagiarism in journalism or English literature. In scientific writing, it is expected that authors will use the same flat, impersonal style and repeat definitions and the results of others to save the reader the time of having to look them up. So, simple pattern matching between science papers will result in a great many false positives. In science (and math) writing what matters is the new result which the author is claiming. It seems to me that it would be nearly impossible for a computer program to detect the distinction.

        Hours of speculation and typing can save one minute of reading TFA. From the article:

        "Unlike other plagiarism detectors, it does not use phrases or similar words to check for copying. Helio Text actually looks at the entirety of the text."

        So no, it does not. It uses instead some sort of similarity metric computed from analyzing the entire text. This is possibly similar to the text distance metrics used in vector space search engine models (see: en.wikipedia.org/wiki/Vector_space_model ). They will be publishing a paper online in PLoS ONE.

        I did RTFA. However, there is no code, no algorithm description, no indication whatsoever in TFA describing exactly how their program operates. From the vague references in TFA it appears that this is nothing more than a glorified, article+abstract-wide, pattern matcher. Perhaps it is a little more clever and uses something similar to Google's page ranking algorithm via applying distance metrics to textual spaces. However, that is also a form of syntactic analysis rather than a context analysis. Barrin

    • by sribe (304414)

      ...you're bound to find the sentence "Let p be an odd prime number."

      Actually, I kind of doubt you see that exact phrase very often. Although, you're certainly more likely to see it than "Let p be an even prime number."

      • by sribe (304414)

        Replying to myself, yes, I know about 2 ;-)

        • by Lehk228 (705449)
          well it's better to use constants than magic numbers anyways, though you should use a more descriptive name than p
      • by NoSig (1919688)
        Actually it is quite common in papers that deal with primes in the first place, though the phrase is more often just "Let p be an odd prime" rather than "let p be an odd prime number".
        • by sribe (304414)

          Actually it is quite common in papers that deal with primes in the first place, though the phrase is more often just "Let p be an odd prime" rather than "let p be an odd prime number".

          OK. I didn't remember it phrased that way from any number theory, but that was decades ago for me. Seems to me a bit obtuse compared to calling out the exception that is being excluded. But if it's done that, it's done that way, regardless of my opinions...

          Regarding your point on phrasing, yeah, just google the two. Yours wins 169,000 to 0.

    • There are approximately 120,000 words in the English language. Most high school students only know probably a third of that at most. So 40,000. Its not hard to imagine across 1000 documents that there would be a pretty high chance 5 words would match in a sequence especially since there exists grammar as well as common ways of expressing things such as "had been taken to heart".
    • by tbischel (862773)

      "Pick up any math paper on number theory, and you're bound to find the sentence 'Let p be an odd prime number.' without citation, but that would hardly qualify as plagiarism."

      I wonder how often you see specifically an odd prime number... since two is the only even prime, its really the oddest of the bunch.

      • by chad_r (79875)

        I wonder how often you see specifically an odd prime number... since two is the only even prime, its really the oddest of the bunch.

        The answer is:

        "About 48,200 results (0.53 seconds)"

    • by cpm99352 (939350)
      Once upon a time, there *BAM*


      What a stupid rule.
    • by canajin56 (660655)

      "Syntactic matching appears to be exactly what this program is doing". At least you openly admit that you are only assuming you know how the fuck it works. Given that they are working in Bioinformatics, and that it's called "ET BLAST" I'm going to go out on a limb and say that it works similar to how BLAST works. When you computing the similarity matrix for a protein (or DNA), well, you could just put those two amino acid sequences (or basepair) side-by-side and count up where they match. Only, some am

  • I poked around the site, and found the page describing some JSON APIs and things, but no links to code or developer pages.

    So where's the code?

    Hmm, okay, that's weird. The project is run by the Virginia Bioinformatics Institute, but the disclaimer [vt.edu] says:

    This software and data are provided to enhance knowledge and encourage progress in the scientific community and are to be used only for research and educational purposes. Any reproduction or use for commercial purpose is prohibited without the prior express written permission of the University of Texas Southwestern Medical Center.

    So they don't hold copyright to it? Or they didn't write it? Hmmmm....

    • They probably are using some code owned by that institute to save time writing it themselves.
  • Even though recycling is not plagiarism, I would love to see this tool being used to create some sort of recycling ranking for individual academics and colleges. There is a not-so-fine line between exploring different aspects of a subject and simply recycling for the purpose of maximizing presence. The former is necessary for the pursuit of research. The later is just f* dishonesty (and a costly one for society since it is typically used for securing research moolah.)
    • Its considered unethical by the majority of scientists to recycle papers unless there is a significant update from one to the next, i.e., methods changed, or additional steps are taken which improve the results. It is not considered unethical to have your paper resubmitted to a different conference or journal if it was rejected from another however.
      • Its considered unethical by the majority of scientists to recycle papers unless there is a significant update from one to the next, i.e., methods changed, or additional steps are taken which improve the results. It is not considered unethical to have your paper resubmitted to a different conference or journal if it was rejected from another however.

        I know, I was referring to the former. In fact, referring a paper to different conferences (say within the same year), that I would *not* consider it recycling.

  • by bobdotorg (598873) on Wednesday October 27, 2010 @11:31AM (#34038002)

    ... can it find dupes on Slashdot?

  • How is that news? I've seen a few universities using systems like that for a few years now...
  • The first study I read in Nature ten years ago placed it about 1-2% in European/North American Journals. A more recent study doubled that figure. Pilot tests in Asia find the number well into double digits.

    No one has fully stated the cause for the increase. I am guessing its better software and nearly all papers are in electronic databases now. A more pessimistic explanation would be that as the "Internet Generation" enters the scientific workforce, their sloppy IP habits migrate into research papers.
  • http://www.crossref.org/crosscheck.html [crossref.org]

    They already create DOIs for their published work and now can check the works before publishing.

  • In fingerprint analysis, the computer spits out a possible match. It's up to the human to determine whether or not that match is valid. It's the same with this stuff.

  • How does this text comparison work? Is it intelligent enough to weight the different sections differently?

    Very often, much of the introductory and methodology sections may be recycled or adapted from previous publications and only the results and conclusions are scientifically novel.

    • by dxk3355 (987361)
      I just saw a presentation that described the two basic formulas that these programs use. The important one is to measure Damerau–Levenshtein distance. This can be combined with fuzzy string searching or other algorithms to determine a percentage.
      • by dxk3355 (987361)
        Dice's coefficient is another important one. You can use it to across words, sentences, and paragraphs to determine a similarity measure.
    • by canajin56 (660655)
      It doesn't rate different sections differently. But, it works by going, for each sentence, find the most similar sentence and compute the score. The score for matching sentences will be somewhat high. The score for dissimilar ones will be quite negative. So, even if it's 50% recycled stuff, and 50% all new stuff, the negative score from the 50% of the sentences that are new will dominate the total score, and it's going to end up pretty low overall. But yeah, it's not a plagiarism detection algorithm, i
  • Unfortunately, during the beta stage the program came across this certain Spielberg movie and a Metallica song and offed itself. Too bad, it seemed to be a pretty handy piece of software.
  • This is a really great tool, actually. For scientific, the time between gathering notes/ideas/data and writing them down can be significant. Even an academic mini-thesis might have 200+ citations. By the time you write the paper it's hard to remember which of your (handwritten) notes are original. I've always wanted a tool that could double check for me.
  • The article points to this link [vt.edu] for the search engine. I did a search with a small paragraph copied from a paper and found too many results with different scores (it doesn't explain what these scores mean). It didn't tell anything decisively if the text is copied from any source, which is expected from a plagiarism tool.

    Secondly, the About [vt.edu] page doesn't talk plagiarism at all. What it says is: "eTBLAST is a unique search engine for searching biomedical literature. Our service is very different from PubMed
  • "Ouch."

  • Didn't I read this same article last week?
  • by Tomji (142759)

    My English prof back in 2000 had this software already.
    However, my final paper was "borrowing" quiet heavily and he didn't find out. Maybe this version works better? :)

You've been Berkeley'ed!

Working...