There has been considerable attention on the paper released by Solomon Hsiang and Nitin Sekar on the National Bureau of Economic Research (NBER) website that claims that the sale of ivory to China in 2008 is the cause of the increase in illegal killing of elephants and demand for ivory.
I looked at this paper with interest because I, along with others, have thought hard about how or whether you can infer the cause of observed increases in demand for ivory. It is also an emotive and controversial topic and this means that it is important to get any analysis right. After serious consideration of this paper and discussion with Bob Burn (a co-author on papers [1,2] we have written that analyse the same data that they use here) we think that both their analyses and their logic for linking what they observe to the sale of ivory to China are flawed.
The authors sent me a copy of the paper when they released it on the NBER website and asked for comments (they note that it is also with a journal to be peer reviewed – to be clear I am not involved in this peer review – but given the renewed proposals for legal sales of ivory for the coming CITES CoP, they felt it important that their manuscript become public early enough to be part of the policy discussion on this issue). The details of my reply to the authors can be found here and I summarise some of my key points below.
Most of the analysis they present uses data from the Monitoring Illegal Killing of Elephants (MIKE) program. The data are the number of elephant carcasses found each year at a number of different MIKE sites across Africa and Asia and the proportion of these carcasses that were illegally killed – this proportion is called PIKE (Proportion of Illegally Killed Elephants). Because PIKE are proportions they are constrained to be between zero and one.
To illustrate their argument the authors plot a number of points showing average PIKE and a clear step change in the value of PIKE between 2007 and 2008. But the points in the graph are not raw data but model outputs. And the model they have used is wrong. For example, although PIKE is constrained to be between zero and one their model does not constrain these values to be between zero and one. They give many reasons for doing this including that to model the data correctly is complex, they wish to choose simplicity over complexity, and if they were to use more complex methods they would need to throw away 32.1% of the data.
Methods for analysing proportions, Generalised Linear Models (GLMs), are taught at undergraduate level on statistics courses. GLMs are actually quite intuitive, widely used and understood and not really all that complex. The authors have misunderstood the methods because you do not need to throw away 32.1% of the data – all of the data can be used. I explain what I think their misunderstanding is in my detailed response. Furthermore, it is not OK to use the simplest of methods if they are wrong and it is clearly preferable to use more complex methods if that is what is needed to correctly represent the data.
The consequences of not modelling the data correctly are that their results could be wrong and it is difficult to know how wrong it is.
Their argument is that in their modelling they tested whether there was evidence of a step change, or discontinuity, in the PIKE data in 2008. That is estimates of PIKE prior to the sale (up to 2007) are significantly lower than estimates of PIKE after the sale (from 2008 onwards) They say that their model shows that estimates of PIKE from 2003 to 2007 were significantly lower than estimates of PIKE from 2008 onwards.
The authors then look for a similar discontinuity in a number of variables they have selected to measure Chinese influence and presence in elephant range states. They consider these to be other potential drivers of the trade. They don’t find the same discontinuity in these variables between 2007 and 2008. Their conclusion is that if these drivers don’t show the step change then as everything else remained constant then the only explanation for the step change is the legal sale of ivory.
There are many things wrong with this, even if we were to ignore the fact that their models are not correct. In the paper they do not:
- provide an explanation as to their choice of potential drivers that they test
- discuss the global financial crisis of 2008. Could this also be a reason why the discontinuity is observed?
- talk about trends in the trade of other illegal wildlife products such as rhino horn and pangolin. These have also increased over the last few years and there have not been legal sales in these products.
- consider trade in other goods that might play a similar role to ivory within China. How has demand in these changed over the same time period? In which case how does this match with the demand for ivory?
- compared their models to a model which allows an increasing nonlinear trend in PIKE rather than a step change
The argument they use that a similar step change is not observed in their other potential drivers might work for a simple situation. But the illegal ivory trade is complex and dynamic with many different drivers operating on different spatial and temporal scales all along the trade chain. It is more likely that if the sale has had an effect it contributes to the increase in demand rather than being the sole reason for an increase in demand. Any analysis should therefore look at relative contribution of different drivers and how they describe changes in PIKE by modelling it in one comprehensive model.
Further criticisms of their approach and their modelling can be found here. To be clear, I am not commenting one way or the other about whether the sale of ivory is the, or one, reason for the illegal ivory trade. My concern is that this analysis and the conclusions it draws is flawed and should not be used to guide future policy on elephants.