Animals make pretty terrible predictive models for treatment research, but what else is there?
It’s mid-winter and I’m just warming my hands by the fire at a house party when the conversation turns to mice.
One of the attendees works in mouse research, but he’s been disillusioned by years of translational failure. Everyone knows mouse research is completely pointless, he says. The results hardly ever translate to humans.
Sorry, what? Doesn’t your work involve mouse studies? I ask. Yes, he says. And almost everything we do is completely useless.
His words have been nibbling around the edges of my mind for the past six months. Surely, more than 115 million animals per year, billions of dollars, and entire careers can’t have been wasted on a faulty model? I decided to find out whether there was any truth in what he had said.
Preclinical animal research is “not science, it’s crap shooting”, says Dr John Pippin, speaking by phone from Washington DC. “It has never been validated as an accurate method for studying human diseases.”
Dr Pippin is a cardiologist and clearly a dog lover. At first, I can barely hear him over the ruckus in the background; his two excitable pet dogs are summarily relocated to the next room (“They’re buddies. Maybe they will calm down now”).
Earlier in his career, Dr Pippin conducted research using dogs. “I believed in it,” he says. “I thought it was a regrettable, but necessary step to take in order to advance human medicine. And I was wrong.”
His research, conducted in the late-1980s, examined the role of radioactive substances in identifying myocardial infarction.
His team would open up a dog’s chest, tie off a coronary artery, and induce a heart attack. They would then inject a radioactive material and conduct medical imaging. Once the dog was euthanised, they’d remove its heart and study the tissue under a microscope.
“What we learned was irrelevant to human medicine,” says Dr Pippin. “It was even irrelevant for dog medicine.”
After this unpleasant epiphany, Dr Pippin conducted a review that cemented his belief in the fallibility of animal studies. All his subsequent research has been animal-free.
Dr Pippin now advocates against the use of animal models as part of the Physicians Committee for Responsible Medicine, a US-based not-for-profit health organisation backed by more than 12,000 physicians.
Animal models are a cornerstone of biomedical research and are used, almost universally, to make decisions about whether to launch human trials or not.
But in reality, only a very slim proportion of drugs that work in animal models succeed when tested in humans.
This low strike rate is no secret. Tthe National Institute of Health in the US estimates that only 5% of clinical trials mirror the animal research findings.1
The failure of animal models to predict winners makes drug development a slow and expensive exercise. For every new drug brought to market, around $2.6 billion is poured into research and development, preclinical and clinical trials, with the process often taking between 10 and 15 years.
The poor predictive value of animal research plagues almost every field of medicine.
In cancer research, where most animal models poorly mimic the complexity of the disease, the translation rate is about 8%.2
Despite decades of investment in animal research for stroke, and more than 100 clinical trials, not a single neuroprotective treatment for humans has emerged.3 In other words, animal models appear to have had zero predictive value in the field of stroke research.
In another example, all 90 HIV vaccines that passed animal testing with flying colours have bombed in human trials. Researchers who have spent decades testing HIV vaccines of non-human primate models have also come up empty handed.4
More than 120 drugs for Alzheimer’s disease, 150 drugs for inflammatory diseases, and 27 drugs for traumatic brain injury have failed clinical trials after showing benefit in animals.
Around a dozen experimental treatments for amyotrophic lateral sclerosis have flopped in human clinical trials over the past decade. The survival benefits of the single efficacious treatment were marginal.5
A growing body of literature has called out the delusional optimism around the predictive value of animal models.
A 2004 paper in the BMJ questioned the value of animal research for human medicine.6
The worth of animal studies was often considered axiomatic or self-evident by clinicians and the public, yet little evidence was available to support this view, British medical sociologist Pandora Pound wrote.
In 2006, a notable systematic review in JAMA of 76 highly cited animal studies on preventive or therapeutic interventions found that only one-third translated at the level of human randomised trials, and only around 10% of interventions were approved for use in patients. 7
A year later, in 2007, a paper in the BMJ found concordance between animal and human studies in only half of the six drugs and procedures studied.8
“In other words, the animal experiments were no more likely than a flip of the coin to predict whether those interventions would benefit humans,” US neurologist Dr Aysha Akhtar wrote in the Cambridge Quarterly of Healthcare Ethics.4
Alien physiology
Sometimes it’s just a square-peg round-hole problem. No matter how hard researchers try to replicate human disease processes, the animal model is just too different or too simplistic.
For example, artificial injuries in animals do not reproduce the complex pathologies that cause strokes in humans, such as atherosclerosis.
Similarly, human tumour cells cultured and inserted under the skin of immunocompromised mice cannot fully recapitulate the genetics and histology of human tumours, although a recent study published in PLOS Genetics in January found that mouse models mirrored human breast cancer much more closely than previously thought.
Genetically engineered mice that carry the same gene mutations as human cancer patients do slightly better than traditional mouse models, and are projected to lift the translation rate of research in this area to around 50%.9
In other areas of research, decades went by before someone bothered to check whether the widely used animal model actually bore any resemblance to the human disease being studied.
Perhaps the most egregious example of this is the 2013 revelation that mice make frightfully bad models for human trauma, burns and sepsis.10
Genes expressed in injured and ill mice were only elevated about half the time than in humans with the corresponding condition. When plotted on a four-quadrant scatter plot, the data showed a random distribution – like a pair of butterfly wings – with no patterns and no similarities whatsoever.
“When I read the paper, I was stunned by just how bad the mouse data are,” Dr Mitchell Fink, a sepsis expert at the University of California, told The New York Times.11 “It’s really amazing — no correlation at all.”
The finding illuminates the hitherto confusing failure of around 150 clinical trials for sepsis treatment, many of which gained funding on the basis of mouse trials.
There’s always been some evidence that mice are as good as aliens in this field; it takes a million times the amount of E. coli in the bloodstream to kill a mouse compared with a human.
Another example is tuberculosis drug discovery, which hasn’t struck gold since the 1970s.12 Mice are easy to infect, they don’t spread the disease by coughing, and they make affordable, easy test subjects.
But mice can only display the active form of TB, and do not develop the granulomas seen in human latent TB.
To understand latent TB, and develop a therapeutic arsenal against it, some scientists are swapping mice for non-human primates, such as crab-eating macaques, with variable success.
Animal models with genetic uniformity make better experimental subjects, but it’s impractical to inbreed primates because the generation time is so long.
Chinese researchers believe cloning could be the way forward.
Two healthy cynomolgus macaques were cloned using somatic cell nuclear transfer for the first time in January, but the finicky and expensive process would be difficult to scale up.
Could humans be to blame?
The recent failure of a major TB vaccine might have cast further doubt on the usefulness of animal models, if an investigation by the BMJ hadn’t implicated the researchers instead.
Despite the MVA85A vaccine being tested in mice, guinea pigs, cattle, and monkeys, it still failed miserably in a clinical trial involving nearly 2800 infants in South Africa.
The Bill & Melinda Gates Foundation, a major supporter of MVA85A, pulled funding from human trials for TB vaccines as a result of this apparent contradiction between animal and human studies.13
But the BMJ investigation, published in January, found that a trial in 16 rhesus macaque monkeys provided strong evidence against the efficacy of the MVA85A vaccine, and was ignored.
The monkey trial raised the possibility that MVA85A was actually impairing the effectiveness of the BCG vaccine, which it was designed to boost. The study concluded 18 months before the start of human trials in 2009.
Instead of coolly reviewing the evidence, scientists were picking and choosing between animal studies, and the BMJ questioned whether the human trial should have proceeded at all.
“The animal studies, when looked at with unblinkered eyes, actually weren’t nearly as promising as they were in the minds of the team involved in the clinical trial,” says Malcolm Macleod, a professor of neurology and translational neuroscience at the University of Edinburgh.
Scientists can sometimes struggle to be impartial and rigorous when assessing the animal evidence for exciting new treatments, he says. “We allow ourselves to be seduced by the idea that we might find something brilliant.”
The MVA85A debacle is not an isolated case; it is an example of systemic problems in animal research, says Professor Kimmelman, a medical ethicist and associate professor at McGill University in Canada.
Like many animal studies, the macaque study – in which just six animals were injected with MVA85A – was neither large enough nor persuasive enough to stall the roll-out of human trials.
Animal studies are frequently poorly designed and underpowered. Methodologies designed to reduce bias in experimentation (such as blinding, randomisation, doing calculations to determine the number of animals needed, registering the protocol, and using statistical models to minimise experimental noise) are often overlooked by animal researchers.
A further source of bias is the tendency of scientists to bury negative findings and invest energy in publishing positive results. This publication bias accounts for about one-third of the efficacy reported by animal models.14
The major choking point for clinical trials is phase II, where drugs are first tested for effectiveness in humans. If animal studies on efficacy were done with the same scrupulousness as safety studies, fewer drugs would die at this point in development, Professor Kimmelman says.
At the moment, there is no way of knowing whether there is a fundamental translation problem in animal research, or simply a crisis in quality.
“We cannot really say how high the translation really is from animals to humans because the quality of reporting is insufficient,” says Merel Ritskes-Hoitinga, a professor in evidence-based laboratory animal science at Radboud University in The Netherlands.
New protocols for animal studies, such as the UK’s ARRIVE guidelines for reporting in-vivo experiments, are expected to lift experimental design quality.
Animal research may also benefit from something akin to the Cochrane collaboration, says Professor Ritskes-Hoitinga. “Systematic reviews of animal studies should become commonplace,” she says.
Systematic reviews for human studies outnumber those for animal studies 10 to one.15
There’s a strong belief medical research that multiple animal models collectively have greater predictive value than one – but is this true?
“I have wanted to answer that question for a long time, and I’ve been stymied,” says Professor Kimmelman. “I can’t get access to the animal evidence that is used to support drug-development programs. Most of it sits inside filing cabinets or it is reported so poorly that it is impossible to even interpret it.”
Professor Ritskes-Hoitinga says researchers should choose the animal that best mimics the human disease state, and then use that to produce high-quality studies.
If animal research becomes less susceptible to bias in the future, the translation rate should increase. “Because you really get a grip on all the factors that play a role and you can interpret them reliably and that’s not possible at the moment,” says Professor Ritskes-Hoitinga.
If the translation rates don’t budge, even with all the necessary adjustments to protocol, we will have to radically rethink our approach and probably stop investing money in animal research altogether, she says.
The Dutch seem quite unattached to the animal model paradigm; two years ago, the Minister for Agriculture, Martijn van Dam, pledged to make the Netherlands world leader in animal-free innovations in 2025.
A Dutch national committee, which is overseeing the transition, envisions the phasing out of all animal procedures for regulatory safety testing of chemicals, food ingredients, pesticides and veterinary medicines.16
The committee proposed using a combination of in vitro, in silico and ex-vivo test methods, supplemented by, and corroborated by, historical data.
“This paradigm shift would deliver at least an equivalent, but very likely a more reliable, risk assessment,” the committee said.
The idea of replacing animal models with alternatives (such as stem cells, human organs grown in the lab, 3D-printed human tissue, or computer modelling) make some people highly uncomfortable.
“The information that researchers get out of animal studies about safety, including understanding how a drug is absorbed, distributed, metabolised, and excreted, is still absolutely vital,” says Ri Scarborough, a manager at Monash University’s Cancer Research Program who wrote an article on translational failure in animal research for The Conversation last year.
“If someone asked me to be part of a first-in-human trial of a drug that had only ever been tested on tissue in a laboratory, I would say, ‘no way’. Without knowing about a drug’s effects on another living animal – preferably a few other mammal species – I simply wouldn’t do it.”
For some, the answer is more exploratory preclinical animal research, not less.
Industry’s obsession with “picking winners” and pushing drugs into clinical trials before the underlying mechanism of action has been fully investigated is one reason that drug trials fail at such a high rate, says Professor Geoff McCaughan, the head of the liver injury and cancer program at the Centenary Institute in Sydney.
Animal research is just one step in the information feedback loop that strengthens our understanding of human disease, and inspires new drugs and diagnostics, he says.
In his view, well-designed test tube experiments and animal studies can demonstrate causation, something that human observational studies can’t do. Animal models aren’t perfect, but they do allow us to pick apart biological mechanisms and replicate many aspects of human disease, he says.
When evaluated for its overall contribution to medical knowledge, animal research for stroke suddenly looks less disappointing, according to Professor Ulrich Dirnagl, a Berlin-based stroke researcher.
While no new therapeutics have been created in recent years, animal models have strong predictive value for human stroke pathophysiology – and have yielded clinically relevant knowledge on everything from statin use during stroke to thrombolysis to neuropsychiatric complications.17
The same can be said for many areas of medicine.
Look no further than the list of Nobel prizes to find the contributions that animals have made: the discovery of sulphonamides, penicillin, the yellow fever vaccine, polio vaccine, the cellular origin of retroviral oncogenes, the HIV-AIDS virus all depended on the use of animal models.
Even in cancer, where translation from mice to humans has proved almost impossible, there is still room for creativity. A relatively new branch of animal research is using pet dogs and cats with naturally occurring cancers to trial drugs.
The advantage of using pets is that their tumours have similar causes and biological mechanisms as human tumours, says Claire Cannon, a veterinary oncologist at The University of Melbourne. “The tumours look the same under the microscope, they often metastasise the same way, they have the same sorts of gene abnormalities that are seen in human cancers.”
So far, preclinical research on pet dogs has produced at least two successful drugs for human oncology, sunitinib (Sutent) for renal cell carcinoma, and L-MTP-PE (mifamurtide) for pediatric osteosarcoma.
So where does this leave us?
Everyone can agree that animal models are underwhelming predictive tools for drug development. But how you choose to interpret that information depends on your perspective.
On the one hand, even a 5% success rate looks good when there’s currently no viable alternative.
And anyone who is familiar with the difficult process of scientific research knows that big breakthroughs don’t occur at a predictable rate.
Career-defining discoveries are extremely rare, unexpected, and often serendipitous. It’s a culture of inquisitiveness, and patience with imperfect models, that makes these developments possible, says Professor McCaughan.
For others, these “eureka” moments do not owe anything to animal research. That animal models overlap with human biology once in a blue moon can be put down to chance. “Even a blind squirrel will sometimes find an acorn,” quips Dr Pippin.
Animal models are intrinsically lacking relevance to human medicine, so it makes no sense to talk about how to improve their performance, he says.
“How much correlation would we need to see if we were using grape Jell-o to predict what happens in humans? There are not the elements necessary to make a prediction.”
It’s like an optical illusion trying to jump between these two world views. Both sides claim to have the weight of scientific literature on their side, and both make persuasive arguments –yet their viewpoints are incompatible.
I suppose this is what it feels like to be living in the middle of a paradigm shift.
Just like history settled the argument between Ptolemaic astronomers and Copernicus, so too will time settle the debate between those who ridicule and those who respect the role of animal models in translational medicine.
References:
1. https://ncats.nih.gov/ntu/about
2. https://www.ncbi.nlm.nih.gov/pubmed/24489990
3. http://www.bmj.com/content/348/bmj.g3387
4. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4594046/
5. https://www.nature.com/news/preclinical-research-make-mouse-studies-work-1.14913
6. http://www.bmj.com/content/328/7438/514.full
7. https://www.ncbi.nlm.nih.gov/pubmed/17032985
8. http://www.bmj.com/content/334/7586/197
10. http://www.pnas.org/content/110/9/3507.abstract
13. http://www.bmj.com/content/360/bmj.j5845/submit-a-rapid-response
14. http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1000344
