A few weeks ago, at a talk I was giving at a Finding Petroleum conference (link), I quipped that the Oil and Gas industry has been run on spreadsheets for over 30 years. Someone in the audience joked back during the questions afterwards that I wasn’t quite right, it was actually Powerpoint! They had a point, but here’s the reason I think spreadsheets have been the reason for the 20 years of progress between 1980 and 2000 and why they are not the right tool for the next revolution*.
Here is a chart of the FTSE 100 index between 1978 and 2017.
There was a period of about 20 years of near exponential growth between 1980 and 2000. I think there is a strong case to be made that this was thanks to the arrival of the spreadsheet. The first spreadsheet, VisiCalc, was released on the Apple platform in 1979. The first version of Excel came out in 1985 but it wasn’t until the release of Windows 3.1 in 1992 that things really took off.
So why are they responsible for this growth?
Because now we had a tool that allowed us to do much more complex analysis of things. Everyone could build their own model of the world in a spreadsheet and optimise it. Goal Seek let us “solve for X”. Now we could model the past and use it to predict the future – hooray, and off we merrily went. We got really good at planning and offline analysis and developed a centralised Command and Control approach:
- We better modelled and understood what was happening
- Data was sent back for offline analysis & understanding
- Then we sent the instructions back
That was great, but I believe this way of working came to an end with the Dotcom crash in March 2000, 19 years ago today as I write this. The spreadsheets had got too big and our faith in the models we built was misplaced. It’s easy to make errors in formulas but is is very difficult to audit a spreadsheet someone else has built. Complex spreadsheets make it look like we know what’s going on, and the person with the most convincing argument (best spreadsheet) at the time wins. But it doesn’t mean the answer on the spreadsheet is what will actually happen!
Every model comes with implicit assumptions and what is not in the model is just as important as what is.
The world has changed. There is now a distrust of centralised decision making and a rebellion against command and control.
Spreadsheets are a great tool and will always be around, but I think we need to change our thinking in order to advance again. We need to move away from the old command and control style.
We must recognise that we don’t actually know the future and we can’t define exactly what we want/need up front.
We must recognise that we don’t actually know the future and we can’t define exactly what we want or need up front. We have to take small steps, get knowledge, fail fast and learn quickly.
Oh, and why did I pick 2008? Well that was the financial crisis caused by a lot of people doing stuff based on models (most probably on spreadsheets) that they didn’t actually understand.
——
* This is based on a presentation I wrote with Gareth Davies in 2017 and presented at the Digital Energy forum in Aberdeen on 14-March (link).