New Zealand's $16 Billion Public Health System Runs on a Single Excel Sheet (theregister.com) 74
The Register's Simon Sharwood reports: The body that runs New Zealand's public health system uses a single Excel spreadsheet as the primary source of data to consolidate and manage its finances, which aren't in great shape perhaps due to the sheet's shortcomings. The spreadsheet-using agency is Health New Zealand (HNZ) which was established in 2022 to replace 20 district health boards in the expectation it would be more cost-effective and deliver more consistent services. The org has a budget of $NZ28 billion ($16 billion) and advised lawmakers it would stay within it for FY 23.24.
That prediction was incorrect and HNZ blew its budget, leading to a review of its finances that last week delivered a damming report [PDF] that found the org lost "control of the critical levers that drive financial outcomes" and had an "inability to identify and respond to the disconnect between expenditure and revenue." The Deloitte-penned report also found an Excel spreadsheet was the "primary data file used by HNZ to manage its financial performance" and was used for "consolidation, journals, business-critical reporting, and analysis."
The report also noted five big problems with the sheet used at HNZ:
- Financial information was often 'hard-coded,' making it difficult to trace to the source or have updated data flow through.
- Errors such as incorrectly releasing accruals or double-up releases were not picked up until following periods.
- Changes to prior periods and FTE errors in district financial reporting Excel submissions, would not flow through to consolidated file.
- The spreadsheet can be easy to manipulate information as there is limited tracking to source information where information is not flowing directly from accounting systems.
- It is highly prone to human error, such as accidental typing of a number or omission of a zero.
Relying on the spreadsheet also meant Health NZ moved slowly: The report found "monthly financial reporting usually took 12-15 days to consolidate and five days to analyze."
That prediction was incorrect and HNZ blew its budget, leading to a review of its finances that last week delivered a damming report [PDF] that found the org lost "control of the critical levers that drive financial outcomes" and had an "inability to identify and respond to the disconnect between expenditure and revenue." The Deloitte-penned report also found an Excel spreadsheet was the "primary data file used by HNZ to manage its financial performance" and was used for "consolidation, journals, business-critical reporting, and analysis."
The report also noted five big problems with the sheet used at HNZ:
- Financial information was often 'hard-coded,' making it difficult to trace to the source or have updated data flow through.
- Errors such as incorrectly releasing accruals or double-up releases were not picked up until following periods.
- Changes to prior periods and FTE errors in district financial reporting Excel submissions, would not flow through to consolidated file.
- The spreadsheet can be easy to manipulate information as there is limited tracking to source information where information is not flowing directly from accounting systems.
- It is highly prone to human error, such as accidental typing of a number or omission of a zero.
Relying on the spreadsheet also meant Health NZ moved slowly: The report found "monthly financial reporting usually took 12-15 days to consolidate and five days to analyze."
One hell of a spreadsheet (Score:5, Insightful)
That must be one hell of a spreadsheet.
I admire the chutzpah of whoever put it together.
Re:One hell of a spreadsheet (Score:5, Interesting)
Whole lot of ERP consulting later, you find out this sort of "master" data source is VERY common. Maybe its not always Excel, but often there's some weird master data that someone in Finance and maybe someone in Operations has to manually fuck with to make things make sense.
What's cool is when it's something else really wild like Lotus 123 (but the DOS version), SuperCalc, Multiplan, some random shit someone came up with in a dBase-like software written in ALGOL-60 that runs on PDP-15 and is in fact the reason that they still have a fucking PDP-15 and it actually just write the data to a TSV which is thing transferred over a serial port through a pearl and then to their ancient Sony NEWS system and then gets imported into SAP. You try to take the TSV file make it into a CSV and import it into SAP, but then nothing works and eventually 113 million dollars later, you just leave the PDP-15 and Sony NEWS in place but buy 4 spares from Yahoo! Auctions JP. (I wish this last part was actually a joke, but sadly it is not.)
Re: (Score:2)
Access is better than Excel, that's granted. But not THAT better.
Re: (Score:2)
depends upon scale, doesn't it? but i agree, no upper level program ever replaces the efficiency and effectiveness of good code, however, it's all a matter of appropriateness just saying
Re: (Score:2)
Access is better than Excel, that's granted. But not THAT better.
The Gateway (2000) service database ran on Access. I can still hear my manager screaming, "DATABASE IS DOWN! EVERYBODY OUT!" when it would lock up several times a day due to multiple people trying to write to the same record within the same few seconds. What a robust database system that is.
Re: (Score:2)
Quick and easy to develop in and had that neat trick of being able to convert over to using a MSSQL server with a few clicks which made it almost workable for multiuser purposes
Re: One hell of a spreadsheet (Score:2)
I thought corporate America ran on COBOL on a heavy IBM z9 dinosaur??
Re: One hell of a spreadsheet (Score:2)
That's much more robust than an Excel spreadsheet.
Re: (Score:3)
it can be, remember Mel https://users.cs.utah.edu/~elb... [utah.edu]
Re: (Score:3)
In more recent years, others have dissected the Story of Mel in painstaking detail, which surprisingly enough does not detract from the story, but makes it richer, warts and all.
https://melsloop.com/docs/the-story-of-mel/pages/mels-hack-the-missing-bits
https://www.e-basteln.de/computing/rpc4000/storyofmel/
Re: (Score:2)
zSeries machines are works of engineering art. Ever see RAIM? https://www.ibm.com/community/... [ibm.com]
There's a reason why these machines are expensive but have years of uptime.
Re:One hell of a spreadsheet (Score:5, Interesting)
Hate to tell you, but whatever company you work for, the "real" corporate ledger is probably in Excel.
Actually you'll find that this is less likely for many companies. As soon as you have numbers that start having more than a couple of zeros after them you quickly find yourself in a situation where you need to report externally. Reporting externally comes often with audit requirements. The government tolerates bullshit such a department not being able to balance its income and expenditure, but shareholders don't.
You'll find excel bullshit in 4 categories:
- Governments - where reporting is not followed up on.
- Medium private companies - who outgrew whatever software they were using when it was small but don't have share holders telling them to keep proper books.
- Startups - because tech bros know better.
- One of my friends see only companies with ledgers like this, but he's a forensic accountant and gets called in when they file for bankruptcy. That said technically this category normally falls into one of the two above, so it's not really 4 categories.
Re: (Score:2)
Re: (Score:2)
Open the books line by line - invoice by invoice (Score:4, Insightful)
In general for governments around the world:
The government should open their financial books, line by line, invoice by invoice, expense by expense so that forensic accounts could go in and identify all of the spending which is not directly linked to a specific item in
- the constitution or the founding document
- laws passed by parliament / congress
- an executive order
- a regulation from a regulatory authority
- a treaty ratified by the government. Excluding, for the USA, agreements signed by a president, not ratified by the legislature - in other words an un-ratified treaty having no legal basis
Each line item or parent line item would need citation of what law, constitution clause, executive order, regulation, treaty clause, etc. that line item is legally authorized for.
N.B. There are laws in other countries, Sarbanes Oxley, which require corporate executives to personally sign that they have reviewed the financial numbers and places the executives at personal legal liability of the numbers are falsified. (Enron scandal).
Re: Open the books line by line - invoice by invoi (Score:1)
All the government Ministeries in New Zealand report their spending on a monthly basis... These contribute the things like the Half Year Financial Update.... Most governments around the world don't do it that frequently....
But the current government has a slashed taxes. And fired huge numbers of public servants.... including putting large numbers of the IT staff on the chopping block... you know the ones who would be responsible for replacing this spreadsheet...
Re: Open the books line by line - invoice by invoi (Score:1)
Oh.... and all that data is uploaded monthly to Treasurery, and the spending is tracked against the Appropriations...
That upload is not json, not CSV, not a .wsdl endpoint.... but is itself an excel spread sheet....
Re: (Score:2)
In general for governments around the world:
The government should open their financial books, line by line, invoice by invoice, expense by expense so that forensic accounts could go in and identify all of the spending which is not directly linked to a specific item in
- the constitution or the founding document - laws passed by parliament / congress - an executive order - a regulation from a regulatory authority - a treaty ratified by the government.
Sounds good. We might need to bring in someone with business expertise for that. The effort should be very public. Should go over well, am I right?!
Re: (Score:2)
N.B. There are laws in other countries, Sarbanes Oxley...
Sarbanes Oxley is a US law.
Re: (Score:2)
MESS (Score:2)
Re: (Score:1)
I hate converting spreadsheet apps into a real system because users get so used to ad-hoc fudges and additions they can't handle the rigidity of a real system.
Most of such rigidity is necessary for chain of custody tracking etc., but they just can't resist the "fudgit urge" for quick fixes. Eventually they get used to it and even appreciate how it keeps things in the right lane, but they are grumpy during the learning curve.
Re: (Score:3)
I hate converting spreadsheet apps into a real system because users get so used to ad-hoc fudges and additions they can't handle the rigidity of a real system.
You don't realise it, but you just described why people use Excel and why it's a VERY GOOD tool for what it does.
Most of such rigidity is necessary for chain of custody tracking etc., but they just can't resist the "fudgit urge" for quick fixes. Eventually they get used to it and even appreciate how it keeps things in the right lane, but they are grumpy during the learning curve.
That rigidity is needed from a corporate POV, but it hinders the actual work of good managers that use that flexibility to, well, manage their department: adapting to changing conditions and requirements by creating new models and workflows, without the need to build a full software toolchain.
These people complain because you're taking away useful and powerful tools that they rely upon for their
Re: (Score:1)
Usually one is called in to automate such because mistakes reach unacceptable levels, not because they love hangin' out with we cheerful coders.
That's awesome (Score:3)
One? That's pretty streamlined (Score:4, Funny)
...our HR runs on 3 spreadsheets.
How the mighty have fallen (Score:2)
financial outcomes? Healthcare needs to not profit (Score:1)
financial outcomes? Healthcare needs to not profit. and no some CEO should not get an bonus to eat up the profit
Re: (Score:2)
Re: (Score:2)
Anything worth doing is worth doing for money.
Profit is good. It inspires people to do better.
Article is hard to understand/believe (Score:5, Insightful)
So, the claim is HNZ uses a single Excel spreadsheet as its "primary" source of data, with the unspoken intimation that there are no other sources of data. Of course, this appears to be false, as the article later refers to "an estimated 6,000 applications and 100 digital networks" (in a disparaging tone), but that then implies that there is indeed a bunch of other data.
So, what is the truth? Maybe HNZ actually has no appreciable data. That's possible but sort of hard to believe. Another somewhat easy to understand idea is that the underlying data has been boiled up into an "executive"-level spreadsheet. It's not unusual for executive decision makers to want a simplified, highly abstracted representation of the data.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
This! What I am getting out of this is that an excel table is used to summaries the source of truth from diverse applications which is really common for decision making, but is not ultimately the control of underlying data.
Re: Article is hard to understand/believe (Score:2)
It wasn't that long ago global companies kept their books entirely on paper, which was actually pretty reliable and auditable. Spreadsheets are basically a better, electronic way to do that.
It's like they are complaining they used a spreadsheet rather than some complicated, proprietary accounting software with millions of lines of unauditable code in t
That's okay. I do everything with sticky notes (Score:3, Funny)
Coded by color and prioriitized by which side of the monitor bezel they are affixed to.
Deutsche Bank (Score:2)
Re: (Score:2)
I replaced a big spread sheet for an energy company with a Java backend on a single old SUN workstation (before it got moved to a bigger one), frontend was in VB as it needed to run on "meta framed" NT desktops. Communication was plain old vanilla CORBA.
We replaced a "system" that caused about 600million losses each year over several years in a row.
But some some dragon will jump up and claim: "write once run everywhere" is a myth ... where is he?
At that time, Excel did not even have "big decimals" (floating
Many companies are like this (Score:2)
Problem isn't the spreadsheet (Score:2)
This is such a bad take.
NZ healthcare is struggling is because the government wantonly slashed its budget, knowing that it wouldn't then be able to meet its commitments and service levels. That same government then demanded that things look and appear a certain way. Is it any surprise then, that they got watermelon metrics?
The truth is this: if you reduce a budget without understanding the impact, or removing the waste first, any org
Re: (Score:2)
Other things this government has done and subsequently blamed on others
Cancelled the “three waters” centralised management and budget plan for fresh, storm and foul water, pushing back the cost onto local councils. Despite these councils largely professing to hate “three waters”, they made no allowance for this, so were left with no budget and cancelled infrastructure replacement.
Tried to tear up the treaty governing the Crown relationship with Maori natives.
Cancelled the ferry rep
Re: (Score:2)
You could have done so much better. Perfect opportunity to damn "capitalism" and conservatives and all the other shibboleths you're supposed to pillory.
Slacker.
*taps the sign* (Score:2)
Yes, have seen this (Score:2)
SOP in critical agencies (Score:3)
Lots of other businesses did and still do the same. I had a decent career translating those spreadsheets into web/database based multi-user apps.
Deloitte Is Utter Dog Shit (Score:2)
Everything looks like a nail (Score:1)
Bargain! $179 for an eternal license (Score:3)
and get Word for free!
And if it does the job, then what is the problem?
Usually systems to manage government information cost millions.
import to database (Score:2)
I used to work at a company that did something similar. The problem is that a lot of data has to be manually entered in somehow, and a spreadsheet was the easiest way early on. We developers wrote code that was able to directly read in the Excel spreadsheet data and work from it when we got an updated file. Inconvenient all around, but manageable at the time.
What you really should have is a CMS (content management system) with loads of input forms that let people type in or otherwise import that data to a d
AI Reverse Engineer? (Score:2)
I postulate that Excel would be as "easy" as a VB4 program...??? Not a troll, am I missing something?
Lotus 1-2-3 (Score:2)
They should've used Lotus 1-2-3.
Re: (Score:1)
I dunno. I kinda want a spreadsheet that can count past 3.
not sure why excel is getting the bad wrap here (Score:2)
from the sounds of it, the data it depends on has errors that lead to cascades of further errors... when you work with garbage data, you get garbage out... any tool that relies on data from multiple sources where people are involved is prone to these errors. It sounds like they have a process issue, not a technology issue.
Auditing, and data verification would resolve these issues.
Deloitte's aim is sales driven (Score:2)
My opinion, based on an very bad experience with them long ago, is that Deloitte's true aim is to nurture a forever dependency on any they deal with to ensure future revenue streams for Deloitte. Again, just my opinion and I've personal reasons to dislike them intensely. My view may be contaminated by that.
All of that in an Excel spreadsheet?! (Score:2)
Huh (Score:1)
The body that runs New Zealand's public health system uses a single Excel spreadsheet as the primary source of data to consolidate and manage its finances, which aren't in great shape perhaps due to the sheet's shortcomings.
So unlike the robust financial health enjoyed by public health systems elsewhere, lol
Well, I'm sure it's nothing that a 100 zillion spent on some new system that never launches can't improve!
Double trouble (Score:2)
The UK system is 10 times as fucked. Probably runs on 2 spreadsheets and some macros.
Re: (Score:2)
The UK system is 10 times as fucked. Probably runs on 2 spreadsheets and some macros.
Even better: Palantir [palantir.com] has all the data.
This isn't the worst idea. (Score:2)
No joke.
If you look how Excel actually is used, it's basically a platform for user-built data+calc-procedures. Very much what SQL was intended to be in the 70ies.
If the Sheet isn't too bloated, this might even be feasible.
The key problem I would have would be the proprietary nature of the software. If someone can migrate that sheet to a small FOSS DB, then the new NZ healthcare system is ready to run for a century.
Re: (Score:2)
Re: This isn't the worst idea. (Score:2)
Very much what SQL was intended to be in the 70ies.
I never used a 70s version of it. But by the 90s, SQL supported multiple users and roles for those users. In effect, collecting data from various organizations into their own tables and records. And securing those entries from either malicious or fat-fingered screwing around. And manipulating/consolidating that data using algorithms developed and placed under configuration control by centralized accounting departments.
I have yet to run across anything that could remotely be considered a spreadsheet that di
Excel ?!? Ehm.... (Score:2)
Restriction brings complication (Score:2)
I See Hammers Everywhere (Score:2)
I've seen worse, like a portfolio of around $750B in private equity funds tracked and reported through dozens of Excel spreadsheets.
As the saying goes
"if the only tool you have is a hammer, everything looks like a nail"
Re: I See Hammers Everywhere (Score:2)
"if the only tool you have is a hammer, everything looks like a thumb"
FTFY
5.223 million people (Score:2)
NZ has a little over 5 million people. While some government people may keep a summary of the nation's financials in Excel, you can't exactly track 5M+ rows in Excel. And that's assuming one line per person. Sure, you could spread the data across several worksheets, but if you've ever tried it you quickly find it doesn't work well and gets unusable quickly.
Secondly, New Zealand's average life expectancy is 82.25 years according to https://www.worldometers.info/... [worldometers.info]. Compared to below and it seems like they
I don't know about source info... (Score:2)