Academic Publishers Turn To AI Software To Catch Bad Scientists Doctoring Data (theregister.com) 39
Shady scientists trying to publish bad research may want to think twice as academic publishers are increasingly using AI software to automatically spot signs of data tampering. The Register: Duplications of images, where the same picture of a cluster of cells, for example, is copied, flipped, rotated, shifted, or cropped is, unfortunately, quite common. In cases where the errors aren't accidental, the doctored images are created to look as if the researchers have more data and conducted more experiments then they really did. Image duplication was the top reason papers were retracted for the American Association for Cancer Research (AACR) over 2016 to 2020, according to Daniel Evanko, the company's Director of Journal Operations and Systems. Having to retract a paper damages the authors and the publishers' reputation. It shows that the quality of work from the researchers was poor, and the editor's peer review process missed mistakes.
To prevent embarrassment for both parties, academic publishers like AACR have turned to AI software to detect image duplication before a paper is published in a journal. The AACR started trialling Proofig, an image-checking programme developed by a startup going by the same name as their product based in Israel. Evanko presented results from the pilot study to show how Proofig impacted AACR's operations at the International Congress on Peer Review and Scientific Publication conference held in Chicago this week. AACR publishes ten research journals and reviews over 13,000 submissions every year. From January 2021 to May 2022, officials used Proofig to screen 1,367 manuscripts that had been provisionally accepted for publication and contacted authors in 208 cases after reviewing image duplicates flagged by the software. In most cases, the duplication is a sloppy error that can be fixed easily. Scientists may have accidentally got their results mixed up and the issue is often resolved by resubmitting new data. On rare occasions, however, the dodgy images highlighted by the software are a sign of foul play.
To prevent embarrassment for both parties, academic publishers like AACR have turned to AI software to detect image duplication before a paper is published in a journal. The AACR started trialling Proofig, an image-checking programme developed by a startup going by the same name as their product based in Israel. Evanko presented results from the pilot study to show how Proofig impacted AACR's operations at the International Congress on Peer Review and Scientific Publication conference held in Chicago this week. AACR publishes ten research journals and reviews over 13,000 submissions every year. From January 2021 to May 2022, officials used Proofig to screen 1,367 manuscripts that had been provisionally accepted for publication and contacted authors in 208 cases after reviewing image duplicates flagged by the software. In most cases, the duplication is a sloppy error that can be fixed easily. Scientists may have accidentally got their results mixed up and the issue is often resolved by resubmitting new data. On rare occasions, however, the dodgy images highlighted by the software are a sign of foul play.
How did we get to here? (Score:3)
Re:How did we get to here? (Score:4, Insightful)
Lots of data gets falsified due to pressure from Big Pharma, for one.
Re: (Score:1)
There's somewhat of a difference between pressure to produce real results, and pressure to cheat. But I do agree they often result in the same thing: cheating.
Not just pressure from Big Pharma (Score:2)
If our political leaders lie and che
Re: (Score:2)
It's the result of PPP: "publish or parish" pressure. And cross-checking others' work is not glamorous and doesn't pay well. Checks and balances are necessary in a good many endeavors and need to somehow be funded.
Re TFA: "To Catch Bad Scientists Doctoring Data"
It's not necessary to put "bad" in the title, there's rarely a good scientist who doctors data. (A military spy on "your side" is about the only situation I can think of.)
Re:How did we get to here? (Score:4, Interesting)
Perhaps we need to take a step back & think how/why we got to the point where falsifying data, & therefore casting doubt on what is known about the world, has become a common practice?
You mean how science became linked to money? Simple, businesses are always pushing to make more money. With US colleges becoming for-profit businesses (despite claims to the contrary) thanks to a major political party being against further education and cutting funding as much as possible, it's a problem in US colleges too.
A system designed to reward greed will push people to do unsavory things or be replaced by someone who will. Fixing this will be a lifelong endeavor as "the system" is highly lopsided in favor of moneyed parties.
Re: (Score:3)
It's like everything else in a capitalistic society. Eventually, money becomes the main driver of everything. In the drive to make more money, all ethics and morals either slowly or quickly erode, as people do whatever they can to gain wealth, which, in a capitalistic society, also means gaining power. Nothing is immune from it. Not even science.
Re: (Score:1, Flamebait)
Because scientists in other systems are perfect right? You know, right now in liberal arts departments in universities there is an ideology that teaches that you can ignore data and reality if it doesn't align with your political ideology. And this is a distinctly left wing ideology. At the top, university presidents are judged by their ability to build new buildings and little else. A step down, bureaucracy is growing at a rapid rate and consuming all the extra funds universities are taking in. Below
Re: (Score:2)
The problem here is that you're thinking it's limited to capitalistic societies. It's literally every where&when. Sometimes the push is stronger than others, and sometimes there emphasis is on alternative currencies, like political connections, but it's always present. Even when cowry shells are the currency. (I seem to recall that Sparta was able to avoid the problem by making the currency large chunks of iron, that nobody could carry around. Their alternative currencies were slaves and political
Re: (Score:2)
And it's corruption all the way down.
Re: (Score:2)
The artificial competition currently being pursued by the institutions & publishers doesn't make any more money
Re: (Score:2)
If you think the slaves weren't ground down, I don't know what your model is. The competition is just as strong in other societies, though the accounting may be a bit different. In some societies they kill lots of people. Rome was like that. It's not always clear from the histories just how things were done, but certainly the push to be on top was strong enough to kill large numbers of pretenders to the throne. To me this doesn't imply weak pressure lower down, it rather implies a desperate striving to
Re: (Score:2)
Re: (Score:2)
Actually the slaves I was referring to were the Spartan slaves I had mentioned earlier. You could, however, also check into the "gentle" methods used by Rome and Carthage. My expamles were chosen because they were before capitalism had evolved, so they could not have been contaminated by it.
And if you were talking about pre-agricultural revolution, then I would agree that the people commonly worked fewer hours. The difference the industrial revolution made is that nominally free people worked more intens
Re: (Score:2)
Re: (Score:2)
Haha. Chinese are among the worst offenders.
Re: (Score:3)
Re: How did we get to here? (Score:2)
IMPOSSIBLE (Score:1, Funny)
This is not possible. Scientist would never lie or mislead the public. I have heard this rule applies from AGW to Covid so I know it must be true... Trust the science!!!!
Re: (Score:2)
Adversarial Network (Score:3)
What a Twist (Score:3)
Having to retract a paper damages the authors and the publishers' reputation. It shows that the quality of work from the researchers was poor, and the editor's peer review process missed mistakes.
To prevent embarrassment for both parties, academic publishers like AACR have turned to AI software to detect image duplication before a paper is published in a journal.
When I reached this part, I had a sudden twinge, as if I were reading an article about pepper spray and stun guns, and how they can be acquired and analysed effectively by serial killers and rapists to hone their craft so everything goes smoothly on "game night."
How much embarrassment do we want to prevent for crooked researchers?
Re: (Score:2)
While not as colorful as your take, this was pretty much my immediate thought at well: what they're actually doing here, whether they intend to or not, is training people who falsify results to do a better job of it while ensuring there is no personal or professional consequence for doing so.
When someone is found to have falsified their data and submitted it for publication, the result should be to broadcast the fact that they did so as far and wide as possible, so that their entire body of work can be revi
So what we need are.... (Score:1)
People that are better with photoshop....
Re: (Score:2)
Having worked in academia, Photoshop and InDesign is sadly the prime way of making figures, statistics and layouts for some. Slack channels are full of grad students and faculty asking how to improve the little data they have to extrapolate the hypothesis' underpinning.
It's primarily government funding that drives this too, basically you get funding on the basis of having already been close to proving your hypothesis.
Re: (Score:2)
Retraction is a given, but publish or perish? (Score:2)
The problem is that people skew results to make it either very good or extremely good.
Science needs us to publish negative results, and admit when approaches fail.
You hypochondriac nerds are in for a shock (Score:1)
Just wait until someone turns that engine on the covid data. The heads of the local branch covidians will pop.
Re: (Score:2)
Just wait until someone turns that engine on the covid data. The heads of the local branch covidians will pop.
The covid data was badly gathered and badly analyzed, it wasn't fake. So this AI won't detect anything there. Sorry to burst your bubble.
Checklist that doesn't require AI (Score:3)
Results are likely doctored if:
- The research is paid for by a commercial interest / drug company
- The research (methods and assumptions) is not made publicly available
- The raw data is not made publicly available
- The research conclusions align with a political agenda
- The research or devices related to it, are patented by the researchers or university
All others: 50% chance
But good luck finding one of those others.
Re: (Score:3)
- The raw data is not made publicly available
So all medical data then? Because the medical field never releases the raw data from their studies. I am sure they claim this is about patient privacy. Most medical studies have a very small number of patients and you can't anonymize such small sample sets. However, I am also sure that if a statistician dug into that data, many many drugs (with large side effects and questionable effeminacy) would be pulled off the market very quickly. Doctors are not scientists and aren't given the statistical traini
Re: (Score:3)
Medical raw data can be de-identified, at least to the degree that it would require serious forensic analysis to figure out who is who.
This is similar to the 100,000 Genomes project https://www.genomicsengland.co... [genomicsengland.co.uk] Sure, if you get that raw DNA data, you could upload it to a site like GedMatch and figure out who a particular raw data sample belonged to. But it would take a lot of effort, beyond what a typical marketer would want to spend. If somebody is after YOUR specific identity and are willing to work
Re: (Score:2)
Also, if the sample size is so small that it can't be effectively de-identified, then you know that the results are little better than random. And yes, I believe there are many drugs on the market that should not be, because they are statistically no better than a placebo, but may have serious side effects.
Running fast while standing still... (Score:2)
Running fast while standing still...the Red Queen's Race [wikipedia.org] with AI.
AI detects doctored data, then new AI doctors data better, and then the AI checks the doctored data better....ad infinitum. Perhaps it will spur increases in CPU and processor power. :-)
JoshK.
Progress? (Score:3)
Ok, so I may have falsified the data backing up the algorithm efficacy, but if I had gotten the damn thing to work I wouldn’t have needed to.
Re: (Score:2)
"put your goals in as a measurable objective"
In a more idealistic time, a high school teacher explained to me how science is done.
"A scientist develops an idea, a theory, about some basic science matter. If he thinks it's worth exploring he begins experiments. The purpose of the experiments is to show that the theory is WRONG. After many failures to prove his idea WRONG, the scientist reaches out to others- maybe they can prove it WRONG. Finally, if nobody can prove the theory WRONG, the results are publish
Re: (Score:2)
High school teacher obviously never worked as a professional scientist. However, he did attend a church known as the The Church of Latter Day Scientists where he was inculcated in some scientolic religion.
No scientist invents a theory in the hope of proving it wrong. More like he dares others to prove him wrong, confident that he's right. Furthermore, many sciences don't have experiments, only observations.
Re: (Score:2)
No scientist invents a theory in the hope of proving it wrong. More like he dares others to prove him wrong, confident that he's right. Furthermore, many sciences don't have experiments, only observations.
Semantics. The only successful scientists are ones that have tried to prove themselves wrong and failed to do so with the math, or through observation and experiment. Sure, I bet none of them wanted to be able to disprove their hunch, but to achieve success it has to actually work and thus fail to be disproven. When they failed to do it themselves and it was predictive that’s when they started daring people to prove them wrong. Scientists that want to be right and did not try to think about how th