Danger to Interview Teams versus Danger to Neighborhood Residents

In the Methods section of the The Lancet paper the researchers state: "The interview team were given the responsibility and authority to change to an alternate location if they perceived the level of insecurity or risk to be unacceptable." Some people have suggested that this discretion implies downward bias in the researchers' estimator. No information is presently available on how often and under what circumstances the field teams exercised their discretion to change sites. Nor is there information on how secondary sites were selected if and when a field team did switch sites. Thus, it is difficult to evaluate the possibility that danger avoidance might have caused downward bias. Here we simply point out that the opposite might be true; danger avoidance by field teams could cause upward bias in a household survey of conflict mortality.

The theory of political scientist Stathis Kalyvas, recently published in book form, is highly relevant to this discussion. The Kalyvas theory concerns the relationship between the control of territory and violence levels and types within the territory. The theory is complex and we cannot attempt a full treatment here but we do draw on some aspects relevant for assessing potential bias in The Lancet study.

Imagine a continuum of control levels ranging from 0 to 1 in which 0 indicates full control by an insurgent group, 1 indicates full control by the government and intermediate values indicate mixed control ranging from predominantly insurgent control near 0 to predominantly government control near 1. The Kalyvas theory predicts that violence will be lowest in areas of strong control by one side or the other, i.e., near 0 or 1. Areas strongly dominated by insurgents might have low violence levels simply because residents live in fear that if they do anything the insurgents do not like then they might be summarily executed or severely beaten. Mortality then might be low within insurgent-dominated areas. Yet an outsider entering such a zone might be extremely unwelcome and rightly perceive himself or herself to be in extreme danger.

The same could be true of government-dominated territory. Violence might be low since insurgents have little presence. Yet a field team entering to measure the number of conflict deaths might be perceived as trying to stir up trouble for the government and, therefore, be unwelcome.

Thus, danger to the survey team might be negatively correlated with danger to the local population. If field researchers use their discretion to avoid areas where they feel unwelcome they may inadvertently wind up in areas with higher mortality rates than the ones they were originally supposed to survey. There is no clear basis for The Lancet authors to assume that they would have underestimated casualties by avoiding surveying in high-conflict areas: The real effect could be any among those of neutral, underestmation bias or overestimation bias.