[I just posted this on Eusprig – but I suspect it is too long to hold the interest in a list post]
I think there is a total chasm between
a. academic researchers whose main spreadsheet experience is the classic ‘student grades’ thing and
b. business spreadsheet jockeys who are in spreadsheets all day everyday.
group a think several hundred formulas is big, group b think several thousand is small.
group a think most commercial spreadsheets have material errors, group b rarely see any error effect.
a think b are over confident, b think a are inexperienced.
Within Eusprig I think we need to find a way to reconcile and explain these two completely opposed views of apparently the same thing. Otherwise neither side will ever gain any credibility from the other.
Personally I don’t believe many commercial spreadsheets have material errors, because most commercial spreadsheets are immaterial. They are a small piece of a bigger effort.
Yes I have seen spreadsheets wrong by millions, and 10+ % or whatever you want to call materiality. But did it change anything? no, not ever.
In a billion dollar, multi year, deal evaluation model, a multi million formula error can be dwarfed by inflation or interest rate assumptions. But whatever, if the price comes in at 1 billion and the client only wants to pay 900 million, then the whole analysis, errors and all, is largely irrelevant. Now the question is ‘are we prepared to take the risk that we can deliver this and survive for 900m?’ or slightly more cynically ‘will they ever tie cost overruns back to me and take back my bonus?’
In my experience spreadsheets are normally one of many inputs to important decisions, any inputs out of tune with the majority are either reviewed for credibility or rejected.
So I agree that most spreadsheets have defects, and I agree that very few lead to an erroneous outcome. And I agree that this is the Human element of spreadsheet interaction, ignored in much academic research. I also believe that the big issue is wasted time and effort, around ineffective spreadsheet use, not error impact.
Maybe we need some more holistic research that covers the whole person/spreadsheet system (in a commercial setting) rather than the spreadsheet in isolation.
I would highlight that in my experience when a spreadsheet changes hands (for holiday cover, job role change or whatever) there is a huge spike in wasted time and risk of nonsense outputs, and external support requests.
What’s is your experience? have you also found that the complete information system that includes these potentially erroneous spreadsheets is usually somewhat self healing? (and self learning – ‘x in reporting is useless, I now ignore everything they send me’)