I remembered the account of developers committing bugs so that reviewers were forced to find something instead of just giving the all clear.
Maybe there should be a group of researchers that submit papers that seem believable with known issues to avoid scientific journals stagnating. Maybe the names and institutions of the group submitting papers should be hidden so that more reputable/famous people and institutions get the same level of scrutiny as everyone else.
In terms of function though, the "hairy arm" is the complete opposite though: it's not to make sure reviewers are paying attention, it's to distract meddling reviewers from the changes you don't want them to veto.
> A feature added for no other reason than to draw management attention and be removed, thus avoiding unnecessary changes in other aspects of the product.
Seems clear to me: "change the system so that publications in big-name journals are not important for career advancement." There are many fields that don't have this requirement, or anything like it, and the suggestion is that academia join their ranks.
The "alternative solution" to having a thing that wacks people on the head every Tuesday is...not having it.
Businesses have money. Governments have votes. What measurable quantity do you propose for scientific work? Votes of colleagues? It's even more corrupt.
Well, simply, for senior researchers to stop making big-time publications so high-up on the list to hire people. I know quite a few senior researchers, and they do have some say in it...
A more benign example was the college professor who started each lecture saying he was going to say one wrong thing.
(it would keep the students alert)
or maybe Van Halen's "CRC check", where their venue contract would ask for a bunch of things set up in advance, including a bowl of M&Ms with the brown ones removed.
(it was a test of diligence, that they read the contract and knew the requirements)
Years before this current trend of LLM generated papers, there was a tool IIRC by some MIT students which would generate plausible looking papers which when actually read were obviously garbage. The title and abstract would have the sort of words you would expect of a research paper, but the resulting sentences would be nonsense.
The idea being that if the submission got published, the review process was obviously a total joke. I think they even had some examples that had actually been published.
You want to solve reproducibility? Apply forensic scrutiny to every publication?
That's a great idea, but who is going to pay for it?
Every scientist knows that it's more than possible to publish deliberately fraudulent work. Every non-scientist thinks that detecting this fraud is easy.
> Every non-scientist thinks that detecting this fraud is easy.
Eh, this working engineer doesn't think so. Freeform fraud detection is a hard problem in general, and when you're talking about (apparent) cutting-edge research, the pool of people who might spot specific details is tiny.
A low probability of detection does suggest that the penalty should be relatively high to have a deterrent effect.
Agreed. I probably won't catch a bug that the tests wouldn't also catch unless it's obvious. I'm looking for things like:
-Is this duplicating something
else?
-Should this be part of this
object?
-if I come back in a year, will
it take me 10 minutes or an
hour to understand it?
-is this necessary?
-should I approve this but add a
task to the tracking board to
figure out if this should be
done in the service instead of
the client?
-etc
If my understanding of the timeline is right, Dias already had a retraction when this was sent in. I don’t think reputation is to blame here, the reviewers just weren’t up to the task.
It might be worth pointing out that reviewers are only on board to offer their opinion with the material given at face value. After all they are probably not in a position to experimentally verify whether the results are reproducible.
The editor always has the final word on whether a manuscript gets accepted.
Weird, I was peer reviewing a manuscript earlier today and actually thought of something similar. The manuscript needs some work to improve its English and I wondered if some of the broken English passages were a test to see what nonsense a referee like me would put up with. I don't think it is a test but the thought did occur to me.
Maybe there should be a group of researchers that submit papers that seem believable with known issues to avoid scientific journals stagnating. Maybe the names and institutions of the group submitting papers should be hidden so that more reputable/famous people and institutions get the same level of scrutiny as everyone else.