Doctors still smarter than the average machine

As GPs may have suspected, the diagnostic skills of doctors still far surpass those of computers

---

 

Many patients use apps and websites to arrive at their own diagnosis before stepping foot in a GP clinic. But, as GPs may have suspected, the diagnostic skills of doctors far surpass those of computers – and a recent study proves it.

Research published in JAMA Internal Medicine showed doctors were more than twice as accurate as web-based systems when it came to diagnosis.

Websites and apps (known as “symptom checkers”) picked the correct diagnosis in a dismal 34% of cases, whereas physicians got it right 72% of the time.

“In what we believe to be the first direct comparison of diagnostic accuracy, physicians vastly outperformed computer algorithms,” the authors said.

A total of 234 internal medicine, family practice and pediatrics physicians solved 45 clinical vignettes during the study.

These vignettes were presented to physicians through an online platform and included the patient’s symptoms and medical history but had no physical examination or test findings.

In a previous study, these same scenarios were used to test 23 websites and apps.

These included symptom checkers produced by the American Academy of Pediatrics, England’s National Health Service and the Dutch College of General Practitioners, among others.

Apps placed the correct diagnosis in their top three only half the time (51%), whereas physicians included the right answer in 85% of cases.

Curiously, doctors were more likely to choose the correct diagnosis for more severe, less common ailments. Symptom checkers were the opposite, having less trouble with low-acuity, common vignettes.

Speaking with TMR, Dr Nathan Pinskier, chair of the RACGP Expert Committee – eHealth and Practice Systems, said this study confirmed previous academic work in this space.

“It is very hard for machines to replicate human thinking. Humans are much more agile,” he said.

But Dr Pinskier said certain areas of diagnostics, such as pathology and radiology, could benefit in the near future from the superior pattern recognition that computer algorithms could provide.

“But when you start to have a multitude of factors that you need combine [to make a diagnosis] there is very little evidence that suggests at the moment that computers are more sophisticated,” Dr Pinskier said.

“We obviously don’t get everything right every time, and that’s demonstrated in the paper. We make mistakes, but I think we are much better at thinking on our feet.”

However, Dr Pinskier could see computers augmenting the work of GPs by helping doctors to structure their thinking and check for errors.

“There is no question about that,” he said.

JAMA Internal Medicine 2016, 10 October