Tag Archives: statistics

John Graunt and the Birth of Medical Statistics

6 Replies

Dr John Carmody of the Department of Physiology at the University of Sydney, recently appeared on the ABC Radio National program, Occams Razor, speaking about John Graunt, a man many years ahead of his time. For those of you preferring the written to the auditory format, he has kindly provided his talk as a guest post for the Mule.

We become blind to what is familiar.

So dependent is modern medicine on accurate measurement that patients and doctors alike accept the fact without surprise or question, perhaps believing that it is inevitable. Yet the importance of numbers of any sort in medicine, let alone precise ones, is a concept that is little over 350 years old. In physiology, the most basic of medical sciences, this dates only from 1628 when William Harvey published his great book on the circulation, a discovery which he formulated and proved through numerical argument.

Then in London, in 1662, 350 years ago this year, John Graunt published a booklet which we can now understand was the beginning of medical statistics, of epidemiology, of medical demography. In the manner of those times he gave it the formidable title of Natural and political observations, mentioned in a following Index, and made upon the Bills of Mortality, to which he added the supplementary description, “With reference to the Government, Religion, Trade, Growth, Air, Diseases and the several changes of the said City”. His work was, therefore, far wider than establishing a new medical discipline. He was arguing for the necessary interaction of medicine, good government and sensible policy—indeed, perhaps for the discipline of quantitative economics, as well. We can realize how original Graunt’s work was when we remember that the only previous English census was the compilation of the “Domesday Book” in 1086 and that the first official census was not taken until 1801.

Graunt’s genius was to recognize—as none of his contemporaries had done—the immense importance of what we would now call a “database” which had existed in London for about 60 years. These were the so-called “Bills of Mortality” which the administrative clerks of the Church of England parishes in London had been obliged to keep scrupulously since James I became king in 1603. In fact, when James granted a charter to the Company of Parish Clerks in 1611, he legally obliged the members to be far more diligent in their recording than before his accession to the throne. These Bills recorded the christenings and the burials, parish by parish, each week. As well, the burials were accompanied by what Graunt called the “diseases and casualties” which brought about those deaths. He drew on the records of about 97 parishes within the city walls and 16 outside them and in a typical year he would have to deal with 20,000-25,000 burials and supposed causes of death.

He was very concerned with the reliability of those diagnoses which were rarely professionally reported. As he wrote, “When anyone dies, then, either by tolling, or ringing of a Bell, or by bespeaking of a Grave of the Sexton, the same is known to the Searchers corresponding with the said Sexton. The Searchers hereupon (who are ancient matrons, sworn to their office), repair to the place where the dead Corps lies, and by view of the same, and by other enquiries, they examine by what disease or casualty the corps died. Hereupon, they make their report to the Parish-Clerk.” Graunt keenly recognized the flaws in such a system and acknowledged that “I have heard some candid physicians complain of the darkness, which they themselves were in hereupon”. He also saw the possibility of corruption, the temptation, as he put it, for “the old-women searchers after the mist of a cup of ale and the bribe of a two-groat fee” to report, say, “Consumption” instead of the more shaming “infection of the spermatick parts”. In fact, he was convinced that syphilis, or the “French pox” was substantially under-reported.

Nevertheless, he decided that the incidence of such problems probably had changed little over the period which he was examining, so errors of those kinds were likely to be fairly consistent. “The ignorance of the Searchers is no impediment to the keeping of sufficient and usefull Accompts”. However, he saw other potential flaws in his data. Whereas corpses had to be disposed of for obvious reasons of health and amenity, and therefore burials provided a pretty reliable index of deaths, christenings did not reliably count births. This was because Catholics and Puritans, in particular, were reluctant to have their offspring baptized into a faith which they opposed. Furthermore, from 1649, when Charles I was executed, until 1660, when his son was restored to the Throne, the government of England was dominated by the Puritans, so many people were more confident to flout Anglican authority. Graunt was therefore obliged to make some corrections to his figures. Then, in attempting to make comparisons of births, deaths and diseases between London and the country, he had to deal with population disparities and calculate per capita rates in the absence of any census information. Another source of error, which was especially nettlesome during outbreaks of plague, was under-reporting of that disease—either because the affected households simply threw bodies into the streets, or because the “Searchers” were unwilling to inspect the bodies closely for fear of contracting the disease themselves. This meant, as Graunt recognized, that plague deaths were under-reported and the counts attributed to other causes were inflated.

Not content with simply aggregating and analyzing his data, Graunt drew up a synoptic list of 106 points in what he called his “Index”, several of which were recommendations for social and health policy.

He asserted, for example, that it would be “better to maintain all Beggars at the publick charge, though earning nothing, then to let them beg about the streets; and that employing them without discretion, may do more harm, than good”. He also found that “not one in two thousand are murthered in London”—a statistical finding which could be considered the birth of serious criminology. Even more importantly, he found that “the Rickets is a new disease, both as to name, and thing”. That diagnosis, he realized, did not appear at all in the Bills until 1634 and even then there were only 14 cases in that year; but by 1658 there were 476 cases. He seriously considered the possibility that previously it had been misdiagnosed but used his data to disprove that hypothesis. This is a remarkable reflection of the approach of William Harvey who had also used numbers to falsify arguments against his concept of the circulation of the blood.

Three years later, at the end of 1665, Graunt published, London’s dreadful visitation, or, A collection of all the Bills of Mortality for this present year, in which he applied the same analytical techniques to the demographic consequences of the “Great Plague of London”. Even today it is amazing and chilling reading: week by week, parish by parish, it documents the relentless surge of that awful disease from its first real appearance in May when 28 cases were recorded. Thereafter, the fatalities increased horrifyingly: about 340 in June; 4400 in July; 13,000 in August; 32,300 in September; 13,300 in October; 4,100 in November and 1,060 in December—a recorded total for that year of 68,600 deaths. And remember: in his earlier book, Graunt had decided that plague was, in such circumstances, seriously under-reported.

Its effects can be put into perspective by this contrast. For example, in the week from 29 August to 5 September, the Bills of Mortality reported 6,988 deaths from plague out of 8,252 burials recorded in the London parishes for that week, and in those 7 days a mere 167 christenings were recorded. Altogether, there were 9,967 christenings in that year and 97,306 burials—an almost 10-fold difference compared with the more usual disparity of less than two-fold and, according to Graunt’s estimates, those burials represented more than 22% of the population of London.

This catastrophic effect on the population of the capital could hardly be replenished by the usual birthrate because even in the first part of 1665 the christenings had been only 57% of the number of burials. In his earlier book, though, Graunt had found that there was substantial nett loss of population from the country to London. The result was that by 1675 the population of the capital was back to pre-plague levels.

In 1663, between the publication of Graunt’s extraordinary books, he had been elected as a Fellow of the Royal Society of London, though this seems not to have been an entirely straightforward matter. By profession, this genius was a haberdasher, whereas, according to the first history of the Royal Society, its membership was comprised principally of “gentlemen, free, and unconfin’d”. That self-congratulatory but diplomatic history which Thomas Sprat published in 1667, only 6 years after King Charles II had joined the society, says of Graunt’s election, “it is the recommendation which the King himself was pleased to make” adding that “his Majesty gave this particular charge to His Society, that if they found any more such Tradesmen, they should be sure to admit them all, without any more ado”. Those last words suggest to me that the “Gentlemen” of the Society required a little Royal “persuasion” which, the King seemed to be hinting, he did not wish to exert a second time.

Graunt was moderately active in the affairs of the Royal Society for a few years, but in the late 1660s he fell onto hard financial times, principally, I think, on account of his conversion to Catholicism. Certainly, this required him to relinquish his military commission as a Major and doubtless had adverse effects on his professional activities. He was eventually bankrupted and died in 1674.

His fading fame was not the only thing which then disappeared. So did some important records of the Worshipful Company of Parish Clerks. In his History of London, William Maitland noted, in 1739, that he had access to the Bills of Mortality only from 1664, stating that the Company “were of the opinion that the same was lent to Graunt…..but by some accident never returned”. He was neither the first nor the last scholar to forget to return borrowed materials to their owners. Nevertheless, the world of medicine remains forever in his debt. Graunt taught doctors that, for all of the importance of their focus on each individual patient, they must also shift their attention to understand what is happening to the whole population and to do so with the aid of the best possible statistics. The world is also in debt to King James, not only for the Bible which he commissioned, but for his insistence that the Parish Clerks should keep those good statistics. It is an unusual example of a beneficial combination of science and religion.

Benford’s Law

24 Replies

Here is a quick quiz. If you visit the Wikipedia page List of countries by GDP, you will find three lists ranking the countries of the world in terms of their Gross Domestic Product (GDP), each list corresponding to a different source of the data. If you pick the list according to the CIA (let’s face it, the CIA just sounds more exciting than the IMF or the World Bank), you should have a list of figures (denominated in US dollars) for 216 countries. Ignore the fact that the European Union is in the list along with the individual counties, and think about the first digit of each of the GDP values. What proportion of the data points start with 1? How about 2? Or 3 through to 9?

If you think they would all be about the same, you have not come across Benford’s Law. In fact, far more of the national GDP figures start with 1 than any other digit and fewer start with 9 than any other digit. The columns in the chart below shows the distribution of the leading digits (I will explain the dots and bars in a moment).

Distribution of leading digits of GDP for 216 countries (in US$)

This phenomenon is not unique to GDP. Indeed a 1937 paper described a similar pattern of leading digit frequencies across a baffling array of measurements, including areas of rivers, street addresses of “American men of Science” and numbers appearing in front-page newspaper stories. The paper was titled “The Law of Anomalous Numbers” and was written by Frank Benford, who thereby gave his name to the phenomenon.

Benford’s Law of Anomalous Numbers states that that for many datasets, the proportion of data points with leading digit n will be approximated by

log10(n+1) – log10(n).

So, around 30.1% of the data should start with a 1, while only around 4.6% should start with a 9. The horizontal lines in the chart above show these theoretical proportions. It would appear that the GDP data features more leading 2s and fewer leading 3s than Benford’s Law would predict, but it is a relatively small sample of data, so some variation from the theoretical distribution should be expected.

As a variation of the usual tests of Benford’s Law, I thought I would choose a rather modern data set to test it on: Twitter follower numbers. Fortunately, there is an R package perfectly suited to this task: twitteR. With twitteR installed, I looked at all of the twitter users who follow @stubbornmule and recorded how many users follow each of them. With only a relatively small follower base, this gave me a set of 342 data points which follows Benford’s Law remarkably well.

;

Distribution of leading digits of follower counts

As a measure of how well the data follows Benford’s Law, I have adopted the approach described by Rachel Fewster in her excellent paper A Simple Explanation of Benford’s Law. For the statistically-minded, this involves defining a chi-squared statistic which measures “badness” of Benford fit. This statistic provides a “p value” which you can think of as the probability that Benford’s Law could produce a distribution that looks like your data set. The follower-count for @stubbornmule is a very high 0.97, which shows a very good fit to the law. By way of contrast, if those 342 data points had a uniform distribution of leading digits, the p value would be less than 10-15, which would be a convincing violation of Benford’s Law.

Since so many data sets do follow Benford’s Law, this kind of statistical analysis has been used to detect fraud. If you were a budding Enron-style accountant set on falsifying your company’s accounts, you may not be aware of Benford’s Law. As a result, you may end up inventing too many figures starting with 9 and not enough starting with 1. Exactly this style of analysis is described in the 2004 paper The Effective Use of Benford’s Law to Assist in Detecting Fraud in Accounting Data by Durtshi, Hillison and Pacini.

By this point, you are probably asking one question: why does it work? It is an excellent question, and a surprisingly difficult and somewhat controversial one. At current count, an online bibliography of papers on Benford Law lists 657 papers on the subject. For me, the best explanation is Fewster’s “simple explanation” which is based her “Law of the Stripey Hat”. However simple it may be, it warrants a blog post of its own, so I will be keeping you in suspense a little longer. In the process, I will also explain some circumstances in which you should not expect Benford’s Law to hold (as an example, think about phone numbers in a telephone book).

In the meantime, having gone to the trouble of adapting Fewster’s R Code to produce charts testing how closely twitter follower counts fit Benford’s Law, I feel I should share a few more examples. My personal twitter account, @seancarmody, has more followers than @stubbornmule and the pattern of leading digits in my followers’ follower counts also provides a good illustration of Benford’s Law.

One of my twitter friends, @stilgherrian, has even more followers than I do and so provides an even larger data set.

Even though the bars seem to follow the Benford pattern quite well here, the p value is a rather low 5.5%. This reflects the fact that the larger the sample, the closer the fit should be to the theoretical frequencies if the data set really follows Benford’s Law. This result appears to be largely due to more leading 1s than expected and fewer leading 2s. To get a better idea of what is happening to the follower counts of stilgherrian’s followers, below is a density* histogram of the follower counts on a log10 scale.

There are a few things we can glean from this chart. First, the spike at zero represents accounts with only a single follower, accounting around 1% of stilgherrian’s followers (since we are working on a log scale, the followers with no followers of their own do not appear on the chart at all). Most of the data is in the range 2 (accounts with 100 followers) to 3 (accounts with 1000 followers). Between 3 and 4 (10,000 followers), the distribution falls of rapidly. This suggests that the deviation from Benford’s Law is due to a fair number users with a follower count in the 1000-1999 range (I am one of those myself), but a shortage in the 2000-2999 range. Beyond that, the number of data points becomes too small to have much of an effect.

Histogram of follower counts of @stilgherrian’s followers

Of course, the point of this analysis is not to suggest that there is anything particularly meaningful about the follower counts of twitter users, but to highlight the fact that even the most peculiar of data sets found “in nature” is likely to yield to the power of Benford’s Law.

* A density histogram scales the vertical axis to ensure that the histogram covers a region of area one rather than the frequency of occurrences in each bin.