A Warning to Consultants about Models

statistical_models.jpg


Due to the complexity of sustainability and social responsibility, we use a lot of models. That said, as management consultants and project / programme managers we use a lot of models as well. One of the key lessons learned though is that "colouring by numbers" does not guarantee success. It is important to realize the limitations and constraints of models, and the fact that they should always be re-evaluated, particularly when new information comes to light. This is important for all of us no matter what side of the debates we are on.

How Reliable are Models?

As a "methodologist", one of my favourite phrases is “all models are wrong… some models are useful,” by George Box, an industrial statistician.

 
This does not even get into the debate of how models may be interpreted differently by other languages, cultures and educational / academic / professional / trade disciplines.

This ties into the philosopher Alfred Korzybski's quote ‘the map is not the territory’, every model can only ever be a model, a necessary simplification of the world, and one that should never be mistaken for the real thing.

Why do we use Models?

Some people like prose, some like models, some like other things to describe and understand things. I like models personally. As Jay Forrester points out below, we organize our understanding of the universe in models to help us understand and interact with the real system or real world.

How should we Continuously Evaluate Models?

However, using the scientific method, we must continuously re-evaluate and test our models.

From the 'Institute & Faculty of Actuaries' I appreciate the following lesson learned for dealing with models (and methods) in the real world:

Al models are deliberate simplifications of the real world. Attempts to demonstrate a model’s correctness can be expected to fail, or apparently to succeed because of test limitations, such as insufficient data...

When a model fails, a variety of narratives may be offered to describe what went wrong. There may be disagreements between experts about the causes of any crisis, depending on who knew, or could have known, about model limitations. Possible elements include:

  • A new risk emerged from nowhere and there is nothing anyone could have done to anticipate it - sometimes called a “black swan”.
  • The models had unknown weaknesses, which could have been revealed by more thorough testing.
  • Model users were well acquainted with model weaknesses, but these were not communicated to senior management accountable for the business
  • Everyone knew about the model weaknesses but they continued to take excessive risks regardless. (Maggid, 2017)

Conclusion

The lesson learned out of this is to not believe completely in models, but to continuously test them out. They aid in communication, awareness and understanding, but they may not provide a complete picture of the real world and all of the possible permutations.

References

  • Keynes, J.M. (1961) The General Theory of Employment, Interest and Money. London: Macmillan & Co., p. viii.
  • Maggid, Joshua. (April 16, 2017). All Models are Wrong. Actuary Info. http://actuary-info.blogspot.com/2017/04/all-models-are-wrong.html
  • Raworth, Kate. (2017). Doughnut Economics: Seven Ways to Think Like a 21st-Century Economist . Chelsea Green Publishing. Kindle Edition. 
  • Whitla, Steve. “How Meaningful Is Your Management Model Diagram? • Meaning Guide.” Meaning Guide, 6 Dec. 2016, meaning.guide/index.php/2016/12/06/how-meaningful-is-your-management-model-diagram/#2x2 matrix.

About the Author - Peter Milsom, FCMC, GPM Global

Peter Milsom helps organizations understand what sustainability means to them, to maximize their benefits and value while minimizing costs and threats. He does this by helping organizations at all levels from the c-suite to the front lines understand what all of the elements of sustainability are, and helping them prioritize and focus on those areas that make the most sense. His area of focus in the sustainability space is change delivery or projects. Peter has also worked on several international standards, and is currently the Convenor of ISO 21502 Guidance on Project Management, and is the Secretary of the ISO 31000 Implementation Guidance Handbook. Peter is currently a Board Member of CMC-Ontario.

He is a contributing author: Sustainable Project Management - The GPM Reference Guide, 2nd Edition; The GPM P5 Standard for Sustainability in Project Management v1.5.

A version of the article was first published here.

cNotes_Banner_.png