A User’s Guide to Expert Advice

All your knowledge is about the past and all your decisions are about the future.

Ian Wilson

The Mayor of my small city owns a short string of dry cleaners. He sort of galumphs around town like a latter-day Bullwinkle. He’s a small-town avatar of Alfred E. Neuman.

His job – like that of all leaders – is to mobilize resources and get the right things done. But in our complex world, it’s often difficult for a leader to know what the right things are. Decisions must be made in realms in which community leaders have no experience or expertise. Thus, they must rely on the advice of experts for guidance.

For a decade or so, I was the recognized technical expert in a field fraught with technical challenges and political minefields, where decisions sometimes involved hundreds of millions of dollars. I later led a multi-million dollar enterprise, where I had to rely on the expertise of others. Having both provided expert advice and used others’ expertise to make important decisions, let me share a few lessons I’ve learned.

Experts advise, leaders decide. This is the most important lesson I’ve learned! Simple, isn’t it – but packed with meaning. First and foremost, a leader needs to define victory – what is the desired outcome. We all want to make data-informed decisions, but that means that the leader needs to lay out the context for the decision. If not, the expert’s advice may not only be misdirected but might lead to unintended consequences. In my experience, describing one or more desirable end-states and asking how to achieve them gives better results than effectively limiting the expert’s response.

Let me use a climate change example. South Florida has seen rising sea levels, and increasing numbers of King Tides and flooding of low-lying areas. Area leaders are united in wanting to limit the impacts of flooding on their communities. One leader might go to his experts and ask how to prevent roads from flooding. Experts might answer that roadways should be elevated. And – voila! – it works, except that now the flooding is in residents’ yards and houses. A better approach might be to ask how to limit flooding so that it did not impact either transportation or disrupt people’s lives, leading to changes in land use and better water shed management.

Another example. The US approach to the pandemic has been to “flatten the curve,” i.e., victory was defined as no Covid-19 deaths due to lack of appropriate medical care. The Swedish approach has focused on protecting the most vulnerable while not intruding too much on daily life. The rising tide of deaths of despair (not to mention medical procedures delayed too long), the economic upheaval and the social unrest we in the US are already experiencing may indicate that Swedish leaders were better at defining victory. No matter which approach ends up with a better end-state, this highlights the importance of carefully defining victory.

One other point to remember. Decisions, particularly public policy decisions must transcend domain expertise by considering other factors. It’s not enough to follow the science. Legal, economic (resulting unemployment) and social impacts (e.g., potential domestic violence, increased deaths of despair) and squishy things like values and the public’s expectations must also be considered. Thus, for decisions with broad societal ramifications, a leader needs advice from a number of different disciplines. The leader has to balance their different perspectives and try to craft a decision that comes as close as possible to the desired outcome. Ultimately, that’s why it’s lonely at the top – leaders often have to implement decisions in an uncertain environment.

Experts are human – usually. Man is a pattern-seeking animal. One of the keys to our survival as a species is that we see patterns (for example, of potential danger) and act on them (e.g., run like hell in the opposite direction). An expert’s advice is most often based on the pattern the expert perceives. Experts’ expertise is thus simply the sum of their experiences, i.e., what they have seen and what they have learned. Sometimes we take that to mean that an expert has to be some hoary old fart who’s been around for years. While it does take time to develop expertise, it’s really more a matter of how much has been learned rather than the time spent learning it (i.e., it’s better to have learned from thirty different experiences, than to have experienced the same thing thirty times).

In my case, I was relatively young (and in a young field) but had seen – and caused! – a lot of failures and had worked hard to learn their causes and prevention. In my case, that bred a great deal of humility – I recognized that I probably knew more than most others, but also recognized how little I knew compared to all that there was to know. Leaders should beware of experts who think that they know it all. They’re likely to introduce cognitive biases into their advice (e.g., cherrypicking data; ignoring facts that don’t fit their preconceptions).

Just as dangerous is the expert who recognizes the uncertainties within a given situation (i.e., can’t find a pattern) but defaults to some other basis for advising leaders (e.g., an unvalidated computer model; use and abuse of models will be the subject of a later post). Too often, these situations result in a sort of “Groupthink” – where experts cluster around a single concept that they might individually not support so strongly.

I’ve learned one other useful lesson: experts, like the rest of us, have biases that may not match those of the decision-maker. I’m sure many of you have been in the situation where a consultant has recommended a course of action that would benefit him, but might or might not achieve victory. The expert may recommend a very conservative course of action so that there is little danger of the expert being proven wrong in his field – the expert can claim success whether or not he’s pointed to the best path. If the expert is part of an entrenched bureaucracy, she may tilt her advice so that it benefits her organization.

Hedgehogs and foxes. Philip Tetlock popularized a concept that dates back to the Greek poet-warrior Archilocus – there are those that know lots of little things (foxes), and those that know one big thing (hedgehogs). This applies to experts as well. Each type has its strengths and weaknesses. Foxes are more likely to foresee potential unintended consequences of a proposed action than hedgehogs, and to be collaborative. However, hedgehogs’ advice may reflect their deeper understanding within a given situation, and thus be superior for bounded problems. Conversely, hedgehogs often are guilty of “epistemic trespassing,” believing that their expertise in one discipline makes their opinion of great value in another.

In my experience, experts from a specialized organization tend to be hedgehogs; broader organizations provide the broader range of experiences needed for foxes. The decision-maker needs to remember that while life is not stovepiped, bureaucracies are – the best advice comes from a competitive intellectual market, involving both foxes and hedgehogs. In the words of Dr. Li Wenliang, who tried to warn the Chinese government of the dangers of Covid-19, “There should be more than one voice in a healthy society,” i.e., an effective decision should have broad input.

To use – or not to use. Let me briefly close with an echo of something I touched on above. The decision-maker is responsible – and accountable – for decisions made, not the expert. Saying that “I’m just following the Science” is a copout and an abdication of responsibility. In my career, I’ve tried to determine whether to follow expert advice based on three factors:

• My trust in the expert or group of experts. This entails factors such as their inherent biases, their track record, their confidence in their conclusions and their consideration of potential unintended consequences.
• The inherent quality of the advice. This entails factors such as how well the current situation seems to match the experts’ assumptions and the experts’ appropriate consideration of uncertainties.
• The fit to the decision. The experts may give me great advice but it’s up to me to determine whether it actually will lead to victory as I’ve defined it.

All leaders eventually are faced with decisions which transcend their own experience. The increasing complexity of our communities, and the unprecedented challenges they face, require that the leaders of our communities receive expert advice. But those leaders must recognize that they are the ones who will make the decisions, and who will be held accountable for their results. I hope that these extracts from my own experience will help leaders better utilize experts and their expertise, and to make better decisions based on expert advice.

========

A brief postscript. This week, we surpassed 100,000 deaths from the coronavirus in the US. On Memorial Day, we also honored those whose lives were sacrificed in service to their country. We can best honor those whose lives were lost from the virus by learning – and acting on – the lessons their loss can teach us. We need to do this with eyes not shaded by party or prejudice, and with a clear intent to not walk down this path again.