A tick of approval by the National Office of Audit for the MHR is being misread as pass mark on "value for money", despite there being no measurements of its effectiveness
If you’re seeing headlines about the National Office of Audit’s (NOA) review of the My Health Record (MHR) and are comforted a little by commentary that this controversial project is somehow effective and on track to save the healthcare system billions of dollars, don’t be. Not yet, anyway.
The NOA spent more than $600,000 measuring the effectiveness of the “implementation” of the MHR by the Australian Digital Health Agency (ADHA), not the actual effectiveness of the electronic record. It’s like giving NASA a tick of approval for how they built a rocket to get to the moon, but failing to check whether it had any chance of actually getting there.
The MHR is an extremely expensive, complex, fund-sucking, attention-distracting venture, and by not providing any meaningful reporting on how much traction it is actually getting in the healthcare community, and how effective it is so far and is likely to be into the future, we are wasting our time and money. What is the government thinking here?
The most misleading thing the NOA report does (perhaps not deliberately) is to claim that because it assesses the “implementation” as effective, therefore the MHR is on track to save our healthcare system $14.6 billion by 2027. They assume this because this is an estimate provided by the ADHA from a while back. It is ignoring entirely whether the MHR is actually taking hold in the manner it has been promised to.
Read anything you can find on the MHR in terms of progress and performance – the ADHA website, the 2018- 19 ADHA annual report, senate estimates, et cetera – and there is no meaningful report which describes a series of benchmarks for savings, with a firm starting point, an estimate of where we should be up to, and relative points into the future.
The annual report of the ADHA is the most telling document. There are no tables of past present and future on benchmarking of performance. There are only randomly thrown out “big net” numbers, which are meaningless because they are given no context against which we make a measurement of whether the MHR is progressing in the manner originally promised, or even with toned-down expectations as the project has moved forward.
Of course, there is the 90% opted-in number. But the reality is this number is is pretty meaningless in the context of efficient healthcare, so far at least. Firstly, most people don’t care if they’re opted in or not, at least 10% -which is a lot – did care and opted out, and we don’t know how many of the 90% are now meaningfully engaged in managing their records. You can assume not very many.
Then there is the point that more than 85% of our healthcare system costs are with the chronically ill, who are mostly over 65. So why have we mixed the whole population into this project?
The ADHA annual report is replete with large net figures being quoted which we guess are supposed to indicate progress – 1 billion clinical documents uploaded, this many thousand pharmacists registered – without any reference to a point of meaning to these numbers to measure progress.
Let’s take 1 billion clinical documents uploaded as an example. What does that mean to anyone? OK, it’s a lot. But what’s in the documents, who uses them and to what effect and how accessible are they? And, is a billion a lot or a little in the context of our healthcare data flow? The figure means nothing by itself. It’s just a lot more than zero. But then a gazillion is a lot more than one billion (thankyou Dr Evil).
The ADHA is not reporting meaningful GP engagement – either because it won’t, or because it can’t. We know GPs are generally upset with the whole project because its “opt out” disregarded the whole concept of patient consent, and they are busy enough without filling in more reports with an unknown return on investment for their daily work cycle.
The ADHA does tell us how many GP practices are registered, and even attempts to tell us that this many GPs are actively using the MHR. But if you dig into the detail of how this is measured, registration and most activity has been driven by ePIP, not necessarily meaningful GP engagement. There isn’t any meaningful assessment of ‘active engagement’. And the activity reporting is not high anyway. The same goes for patient engagement. There is no meaningful measurement being reported of how patients are using the record.
The ADHA report is a pitch based on isolated one-off anecdotes and net big isolated numbers that are never referenced to a point of progress.
Why aren’t we being told now how many patients use the MHR, what demographics they represent, whether their control of their record is being trusted enough by doctors to use it meaningfully, and so on.
The audit office’s pronouncement on the MHR is near useless. It says the MHR has been implemented to a reasonable standard. It then makes an assumption that we therefore would expect the forecasts of savings of the MHR originally projected to be realised. How does that follow? There is no connection between one and the other in terms of progress and return on investment.
There are some great anecdotes in the ADHA annual report of saving lives and of helping in times of natural disaster. But they aren’t meaningful measurements painting a picture, they are one-off stories painting a picture.
Why doesn’t the ADHA attempt any such figures and why are we paying more than $260 million for the ADHA each year without any meaningful measurement of ongoing return for our investment. At this point of time the MHR is still pie-in-the-sky stuff. The effectiveness of the MHR is still a mystery, $3 billion into its investment. It just shouldn’t be.
It isn’t the audit office’s fault that it’s report is misleading. Much of the media has interpreted the audit as one of checking if the overall project was on track. The office does say that the longer term clinical benefits of the MHR haven’t been evaluated in the report. Which begs the question why it would assume the project is on track to save $15 billion?
We aren’t saying the MHR project is a white elephant yet. We are asking that if it is so expensive and important, why do we refuse to measure it properly and publish those findings?
We are also saying that there are a lot of factors closing in on the project and the ADHA which should put a lot of pressure on the government to audit the project properly.
As an example, one of the most useful parts of the MHR in the last year has been a function called Medicines View. This was a neat piece of programming by the MHR people to extract medicines tables from a couple of reliable sources (such as the chemists dispensing system) and put it in a format at the top of the MHR which was reasonably reliable. Great. Up-to-date meds info is very important.
But eScripts are coming, and fast. Now that we’ve legislated for eScripts only, and based on how effective and fast this technology rolled out in the US, we can expect within a few years that up to 90% of our scripts will be held on our smartphones (or on those of our family or carers if we are older and not able to go digital). That means the smartphone has the most up-to-date and meaningful meds record.
Then consider what technology such as the web-sharing data standard FHIR is doing overseas now and is likely to do for important distributed information that a patient will encounter as they visit a GP, hospital, pharmacist or allied health professional. This technology clearly points a path to where a patient’s phone or other device is going to be the most important personal medical record holder (much like it’s the centre of your banking now). As you visit each of these professionals their databases will talk to your phone. Your phone will only have meaningful and recent data being sent to it.
The MHR is old data architecture now and the government knows it. It reaches out to distributed databases, takes the data and mashes it all in the middle on your MHR without any organisation. That’s dangerous for a whole lot of reasons, not the least of which is we are holding everyone’s health data in one place and it will be hacked one day.
The ADHA has taken that security issue seriously. It is spending more than any other government department on security. But it’s money that it wouldn’t need to spend if our personal health record is going to be held on our phones by our phone provider – where security is far superior.
These are just a few of the issues that the MHR and the ADHA are facing.
Either there is a reluctance to report just how effective the MHR is in terms of doctor and patient engagement against the $15 billion number, or the ADHA isn’t able to do it. I think the agency is able to get this data now, which means it is very worried where it is heading, and it should be. It has cost a fortune in time, money and attention around what we should be doing in the realm of digital health.
With apologies now to our mainstream media, I think we have a “right to know”.
And if the ADHA honestly doesn’t know, can we put someone in charge who does?
In Friday’s newsletter we do an in-depth look at the ADHA’s key strategy pillars and where we are, and aren’t, getting a return on investment for the healthcare system
