Solving crimes requires both analysis and deduction, as does the decision-making process. Inductive is gathering evidence and analyzing it. Deductive is developing a hypothesis and then testing it with evidence.
“It is a capital mistake to theorize before having data. Insensibly one begins to trust facts to suit theories, instead of theories to suit facts.” – Sherlock Holmes
Sherlock formulated theories based on observable facts. This is consistent with the “big data analytics” approach so popular today. Accumulate a body of data and then look for relationships and patterns. But does that not leave one open to error? It has been said that sufficiently tortured data will confess to anything. And academic research begins with a hypothesis and then accumulates and analyzes data to test it. This is to prevent “fishing expeditions” where the researcher first finds correlations and then formulates a theory based on them.
In large databases one can frequently find correlations, but that may not establish causation or in the case of Sherlock “who done it.” Today’s data scientists use artificial intelligence and machine learning to find explanations for things. But do they explain reality or only identify data relationships? If A is correlated with B that may provide little guidance, since both may be caused by C. And if causation is being sought did A cause B or did B cause A? . Meaningless correlations have subjected many to ridicule for failing to provide any rationale for why they might exist. For example, it has been found that executive compensation levels are correlated with company performance, suggesting that paying more might get you more. But equally plausible is that successful organizations can afford to pay more without Board or shareholder rebellions. So what is the truth? Of what value is the correlation in guiding strategy?
Holmes concluded that Dr. Watson should consider acquiring a physician’s practice by using evidence gained through observation. He assumed that the worn steps on the stoop of a Physician’s office suggest an active practice that perhaps Dr. Watson should acquire. But that data alone could be misleading. If there were other tenants in the building they could have generated much of the traffic. And perhaps the wear and tear on the stairs was caused by dragging bodies out after failed treatment. So in order to correctly diagnose Sherlock should have used multiple regression analysis to identify all of the contributing factors and their relative impact on the result. Anyone having done multiple regression analysis lacking a computer can attest to the reality that it would take hours, if not days, to come up with such an explanatory model. And of course computers were not available at the time.
So were his instantaneous conclusions just good guesses or the result of extraordinary intuition? Was he the master of deduction? This is a bigger mystery than he was ever asked to solve.
Robert J. Greene | Reward $ystems, Inc. 