In this follow-up guest post on The Stubborn Mule, Mark Lauer takes a closer look at the relationship between national development and fertility rates.

STOP PRESS: Switzerland’s population would be decimated in just two generations if it weren’t for advances in their development.

At least, that’s what the modelling in a recent Nature paper projects.  The paper, widely reported in The New York Times, The Washington Post and The Economist, amongst others, was the subject of my recent Stubborn Mule guest post.  In that post, I shared an animated chart and some statistical arguments that cast doubt on the paper’s conclusion.  In this post, I’ll take a firmer stance: the conclusion is plain wrong.  But to understand why, we’ll have to delve a little deeper into their model.  Still, I’ll try to keep things as non-technical as possible.

First, let’s recap the evidence presented in the paper.  It comprised three parts: a snapshot chart (republished in most of the reportage), a trajectory chart, and the results of an econometric model.  As argued in my earlier post, the snapshot is misleading for several reasons, not least the distorted scales.  And the trajectory chart suffers from a serious statistical bias, also explained in my earlier post.  I’ll reproduce here my chart showing the same information without the bias.

That leaves the econometric model.  From reading the paper, where details of the model are sketchy, I had wrongly inferred that the model suffered the same statistical bias as the trajectory chart.  I have since looked at the supplementary information for the paper, and at the SAS code used to run the model.  From these, it is clear that a fixed HDI threshold of 0.86 is used to define when a country’s fertility should begin to increase.  So there’s no statistical bias.  However, I discovered far more serious problems.

Continue reading →

In this first ever guest post on The Stubborn Mule, Mark Lauer takes a careful look at the relationship between national development and fertility rates.

Recently The Economist and the Washington Post reported a research paper in Nature on the relationship between development and fertility across a large number of countries.  The main conclusion of the paper is that, once countries get beyond a certain level of development, their fertility rates cease to fall and begin to rise again dramatically.  In this post I’ll show an animated view of the data that casts serious doubt on this conclusion, and explain where I believe the researchers went wrong.

But first, let’s review the data.  The World Bank publishes the World Development Indicators Online, which includes time series by country of the Total Fertility Rate (TFR).  This statistic is an estimate of the number of children each woman would be expected to have if she bore them according to current national age-specific fertility rates throughout her lifetime.  In 2005, Australia’s TFR was 1.77, while Niger’s was 7.67 and the Ukraine’s only 1.2.

The Human Development Index (HDI) is defined by the UN as a measure of development, and combines life expectancy, literacy, school enrolments and GDP.  Using these statistics, again from the World Bank database, the paper’s authors construct annual time series of HDI by country from 1975 until 2005.  For example, in 2005, Australia’s HDI was 0.966, the highest amongst all 143 countries in the data set.  Ukraine’s HDI was 0.786, while poor old Niger’s was just 0.3.

A figure from the paper was reproduced by The Economist; it shows two snapshots of the relationship between HDI and TFR, one from 1975 and one from 2005.  Both show the well-known fact that as development increases, fertility generally falls.  However, the 2005 picture appears to show that countries with an HDI above a certain threshold become more fertile again as they develop further.  A fitted curve on the chart suggests that TFR rises from 1.5 to 2.0 as HDI goes from 0.92 to 0.98.

Of course, this is only a snapshot.  If there really is a consistent positive influence of advanced development on fertility, then we ought to see it in the trajectories through time for individual countries. So to explore this, I’ve used a Mathematica notebook to generate an animated bubble chart.  The full source code is on GitHub, including a PDF version for anyone without Mathematica but still curious.  After downloading the data directly from Nature’s website, the program plots one bubble per country, with area proportional to the country’s current population.

Unlike with the figure in The Economist, here it is difficult to see any turn upwards in fertility rates at high development levels.  In fact, the entire shape of the figure looks different.  This is because the figure in The Economist uses axes that over-emphasise changes in the lower right corner.  It uses a logarithmic scale for TFR and a reflected logarithmic scale for HDI (actually the negative of the logarithm of 1.0 minus the HDI).  These rather strange choices aren’t mentioned in the paper, so you’ll have to look closely at their tick labels to notice this.

To help focus on the critical region, I’ve also zoomed in on the bottom right hand corner in the following version of the bubble chart.

One interesting feature of these charts is that one large Asian country, namely Russia, and a collection of smaller European countries, dart leftwards during the period 1989 to 1997.  The smaller countries are all eastern European ones, like Romania, Bulgaria and the Ukraine (within Mathematica you can hover over the bubbles to find this out, and even pause, forward or rewind the animation).  In the former Soviet Union and its satellites, the transition from communism to capitalism brought a crushing combination of higher mortality and lower fertility.  In Russia, this continues today.  One side effect of this is to create a cluster of low fertility countries near the threshold HDI of 0.86 in the 2005 snapshot.  This enhances the impression in the snapshot that fertility switches direction beyond this development level.

But the paper’s conclusion isn’t just based on these snapshots.  The authors fit a sophisticated econometric model to the time series of all 37 countries that reached an HDI of 0.85, a model that is even supposed to account for time fixed-effects (changes in TFR due only to the passage of time).  They find that the threshold at which fertility reverses is 0.86, and that beyond this

an HDI increase of 0.05 results in an increase of the TFR by 0.204.

This means that countries which develop from an HDI of 0.92 to 0.98 should see an increase in TFR of about 0.25.  This is only about half as steep as the curve in their snapshot figure, but is still a significant rate of increase.

However, even this rate is rather surprising.  Amongst all 37 countries, only two exhibit such a steep rise in fertility relative to development between the year they first reach an HDI of 0.86 and 2005, and one of these only barely.  The latter country is the United States, which manages to raise TFR by 0.211 per 0.05 increase in HDI.  The other is the Czech Republic, which only reaches an HDI of 0.86 in 2001, and so only covers four years.  Here is a plot of the trajectories of all countries that reached an HDI of 0.86, beginning in the first year they did this.  Most of them actually show decreases in TFR.

So how do the authors of the paper manage a statistically significant result (at the 0.1% level) that is so widely different from the data?  The answer could well lie in their choice of the reference year, the year in which they consider each country to have passed the threshold.  Rather than using a fixed threshold as I’ve done above, they express TFR

relative to the lowest TFR that was observed while a country’s HDI was within the window of 0.85–0.9.  The reference year is the first year in which this lowest TFR is observed.

In other words, their definition of when a country reaches the threshold depends on its path of TFR values.  In particular, they choose the year when TFR is at its lowest.

Does this choice statistically bias the subsequent trajectories of TFR upwards?  I leave this question as a simple statistical exercise for the reader, but I will mention that the window of 0.85–0.9 is wider than it looks.  Amongst countries that reached an HDI of 0.9, the average time taken to pass through that window is almost 15 years, while the entire data set only covers 30 years.

Finally I’d like to thank Sean for offering this space for my meandering thoughts.  I hope you enjoy the charts.  And remember, don’t believe everything you see in The Economist.

UPDATE:

To show that the statistical bias identified above is substantial, I’ve programmed a quick simulation to measure it.  The simulation makes some assumptions about distributions, and estimates parameters from the original data.  As such it gives only a rough indication of the size of the bias – there are many alternative possibilities, which would lead to larger or smaller biases, especially within a more complex econometric estimation.

In the simulation, each of the advanced countries begins exactly where it was in the year that it first reached an HDI of 0.85.  Thereafter, a trajectory is randomly generated for each country, with zero mean for changes in fertility.  That is, in the simulation, fertility does not increase on average at all¹.  As in the paper, a threshold is found for each country based on the year with lowest TFR within the HDI window.  All shifts in TFR thereafter are used to measure the impact of HDI on TFR (which is actually non-existent).

Here is a sample of the trajectories so generated, along with the fitted response from the paper.

The resulting simulations find, on average, that a 0.06 increase in HDI leads to an increase of about 0.075 in TFR, despite that fact that there is no connection whatsoever.  The range of results is quite broad, with an increase of 0.12 in TFR also being a likely outcome.  This is half of the value found in the paper; in other words, simulations of a simplified case where HDI does not influence TFR at all, can easily generate half of the paper’s result.

Of course, if the result is not due to statistical bias, then the authors can easily prove this.  They need only rerun their analysis using a fixed HDI threshold, rather than one that depends on the path of TFR.  Until they do, their conclusion will remain dubious.

¹ For the technically minded, the HDI follows a random walk with drift and volatility matching those of advanced countries, and the TFR follows an uncorrelated random walk with volatility matching the advanced countries, but with zero drift.  The full source code and results have been uploaded to the Github repository.

FURTHER UPDATE:

More details can be found in the follow-up post to this one, Fertility Declines Don’t Reverse with Development.