1763: Bayes’ Theorem

Bayes’ theorem is a fundamental concept in probability theory.  It was formulated in 1763 by the English statistician and Presbyterian minister, Reverend Thomas Bayes.

History of Bayes Theorem

Thomas Bayes was born in London in 1702 and studied at the University of Edinburgh.  During his time at Edinburgh he was exposed to some of the leading mathematicians and philosophers of his time.  He was elected as a Fellow of the Royal Society where he may have served as a mediator of intellectual debates. He later returned to London to become a minister, but he continued to pursue an interest in mathematics, specifically in probability theory. 

Bayes wrote his theorem in order to address the question of how to revise beliefs in the light of new evidence. However, more interestingly it appears that he likely wrote it as a mathematical means to defend Christianity and to combat an argument by David Hume in his 1748 essay Of Miracles, from his book An Enquiry Concerning Human Understanding. In this essay, Hume made the case for dismissing miracles, such as the resurrection of Christ, on the grounds of probability. In effect he argued that the probability for miracles (a violation of the laws of nature) was much more improbable than the probability that miracle was accurately reported. While there is no absolute or direct evidence that Bayes sole motivation to compose his work was to refute Hume’s essay, there is extremely good circumstantial evidence he did at least in part, given the details surrounding the events of his later life and the eventual publication of his work.

Whatever the real motivation for his work may have been, Bayes’ work was published two years after his death, when his friend Richard Price brought it to the attention of the Royal Society and read on December 23, 1763. It was published to following year both in the Philosophical Transactions, the journal of the Royal Society, and as an offshoot. The now famous essay was titled An Essay towards solving a problem in the Doctrine of Chances.  It should be noted that in 1767, Prince published a book titled Four Dissertations, where he explicitly took on the work of Hume and challenged his probabilistic arguments in Of Miracles. He used Bayes results in an attempt to show that Hume failed to recognize the significance of multiple independent witnesses to a miracle, and that the accumulation of even imperfect evidence could overcome the statistical improbability of an event.

As things sometimes happen in the history of science, the theorem initially was largely forgotten, until it was independently rediscovered by the brilliant French mathematician Pierre-Simon LaPlace in 1774. The theorem is used to describe the conditional probability of an event.  Conditional probability tells us the probability of a hypothesis if some event has happened. 

The Goal of Getting Closer and Closer to the Truth

Bayes’ Theorem involves beginning with an educated guess, called a prior probability, and then revising your prediction when new evidence comes in.  As the new evidence is considered the probability of the event is updated give you the posterior probability.  Bayes’ Theorem provides a useful way of thinking by approximation, getting closer and closer to the truth as we accumulate new and relevant evidence.  This is an important point to consider because we are always working with incomplete information in nearly all situations.

A Bayesian way of thinking requires us to constantly update our probabilities as new evidence becomes available to us.  This revision does not happen just once but can continually happen. We may never know the truth with 100% certainty, for example we can never be 100% certain the sun will rise tomorrow.  But with Bayesian thinking we can be 99.999999% sure which tells us we’re getting very close to the truth and gives us a high degree of confidence in the proposition. Bayes theorem helped to revolutionize probability theory by introducing the idea of conditional probability – probability conditioned by evidence. If you have an extraordinary hypothesis, it should require extraordinary evidence to convince you that it’s true.

Practical Uses of Bayes Theorem

Bayes Theorem has relevance in any avenue of life because it is a form of probabilistic thinking.  If you think about it, everything you and happens to you in life is probabilistic in nature.  The theory’s flexibility and versatility provide the ability to make both life and business decisions under conditions of uncertainty.  Here are a few examples of Bayesian theory used in the real word. In biology it is used for medical diagnosis, genetics, and the spread of infectious diseases.  In computer science it is used in speech recognition, search engine algorithms, spam filtering, and weather forecasting.  Its practical examples are almost limitless.  Ultimately, it is a learning process, with more observations and evidence leading to better certainty.  Lets take a look at one interesting application of Bayes theorem in a real word setting. 

The theorem was used to crack the Nazi Enigma code during WWII.  The Enigma code was an encryption machine that the Germans used to send secure messages.  Its effectiveness was that its cipher system was changed daily.  Alan Turing, the brilliant British mathematician, used Bayes Theorem to break down an almost infinite number of translations based on messages that were most likely to be transmitted.  For example, messages from German U-boats were most likely to transmit messages containing information about the weather or allied shipping.  The strong priors thus greatly reduced the possible translations to be deciphered and sped up to time to crack the code.  Eventually he and his staff invented a machine known as The Bombe, which ultimately cracked the German Enigma Code. The use of Bayes’ theorem in cracking the Enigma code was a monumental breakthrough for the Allies, as it provided them access to critical information about German military operations. It provided a significant strategic advantage in the war effort and played a key role in their eventual victory.

Bayes’ theorem continues to impact statistics and society to this day. In recognition of Bayes’ contribution to the development of probability theory, the Bayesian Analysis journal was established in 2006 as a peer-reviewed academic journal dedicated to Bayesian statistics. Additionally, they Thomas Bayes Award is awarded every two years by the Royal Statistical Society to recognize outstanding contributions to the field of Bayesian statistics. The continuing relevance of Bayes’ theorem is a testament to the enduring legacy of Thomas Bayes and his contribution to the field of probability theory.

Continue reading more about the exciting history of science!