Stupid Time at the Lancet

In the current on-line Lancet (10/11/06), a team of Johns Hopkins authors headed by Prof. Gilbert Burnham present a calculation of Iraqi civilian casualties ten times greater than that of the Brooking Institute, and twenty times greater than that of the Pentagon.

“We estimate,” write the authors, “that as of July, 2006, there have been 654,965 excess Iraqi deaths as a consequence of the war.” 

This is not the first October surprise hatched by researchers from the Bloomberg School of Public Health, and published by the Lancet in the heat of an American election cycle.  A predecessor piece, rushed to press on Oct. 29, 2004, reached conclusions of similar magnitude.  Hopkins’ Les Roberts, the lead author in 2004, admitted that the timing was deliberate.  And the Lancet editors clairified their political intent.  They editorialized:

The invasion of Iraq, the displacement of a cruel dictator, and the attempt to impose a liberal democracy by force have, by themselves, been insufficient to bring peace and security to the civilian population. Democratic imperialism has led to more deaths not fewer.  This political and military failure continues to cause scores of casualties among non-combatants. It is a failure that deserves to be a serious subject for research. But this report is more than a piece of academic investigation.

Methodologically, the current Lancet authors surveyed 12,801 Iraqi residents in 1,849 household from May through early July, 2006.  They recorded respondents’ recollections of household deaths, supported (in most instances) by death certificates.  The researchers divided the respondents’ recollections into two periods: the 14 months preceding the invasion, and the 40 months after.  Comparing the death rates in the two time frames, they concluded that the death rate in Iraq has increased by a factor of 2.5.  Extrapolating the death rates in the pre- and post-invasion periods over the entire nation, they conclude that roughly 655,000 more Iraqis have died under the democracy than would have died under the Saddam-era baseline – excess deaths, as they say.

As in 2004, the Lancet study has garnered a great deal of press.  And as in 2004, the study’s conclusions have been widely dismissed by serious researchers.  The most obvious reason is that the Hopkins researchers don’t record 655,000 extra casualties – they extrapolate them.  The total deaths recorded in the household sample was 82 in the 14 pre-invasion months, and 547 in the 40 post-invasion months. 

Other organizations, many of them anti-war, maintain comprehensive databases that track civilian casualties in post-Ba’athist Iraq death-by-death.  The Brookings Institute’s Saban Center updates their database weekly, based on several sources:

  • Iraqi Body Count, whose meticulous tally is compiled from deaths reported in the Iraqi and international press;
  • Reports from the Iraqi Health Ministry, and the Baghdad Medico-Legal Institute; and
  • Numbers compiled the U.N. Assistance Mission for Iraq.

Using these sources, Brookings published a “low” estimate of 43,300 civilian casualties of the post-war, and a “high” estimate of 62,000, cumulative through summer, 2006.  The primary ambiguity is less “Who died?” than “How?”  When crime-induced deaths are subtracted from the total, the low end of the range results; and conversely.  

Brookings has recently concluded that it is no longer possible to separate crime-based violence from sectarian-based violence.  It therefore favors the higher estimate. But even this is less than one tenth of the  civilian fatalities calculated by the Lancet authors.

Anthony Cordesman, no friend to the Bush administration, maintains that the sectarian conflict in Iraq now constitutes civil war.  Cordesman, who holds the Arleigh Burke chair in strategy at the Center for International and Strategic Studies, termed the Lancet estimates “certainly way too high.” 

“This is not analysis,” he told the Associate Press.  “This is politics.”

How did the Lancet calculate a civilian death number that exceeded that of other war critics by an order of magnitude?

Prof. Burnham and colleagues defend their statistically extrapolated estimate of “excess death” against the purported imperfection of  observation-based methods.  “Our estimate of excess deaths is far higher,” they write in the Lancet, “than those reported in Iraq through passive surveillance measures.  This discrepancy is not unexpected.  Data from passive surveillance are rarely complete, even in stable circumstances, and are even less complete during conflict, when access is restricted, and fatal events could be intentionally hidden.”

But under what circumstances are “passive” methods unreliable?  Are we to distrust direct observation universally? 

In Saddam’s Iraq, there were no commercial TV stations, no commercial radio stations, and no independent print organs (newspapers, magazines).  Today, there are, respectively, 54, 114, and 268.  If one doubts the significance of these numbers, pick up an SRDS media guide for California – a state of comparable to Iraq in area and population.  The media choices of multi-cultural Angelos are less varied than those of Baghdadis. 

In contemporary Iraq, every party, every faction, every sect, has its own organ of communication, each eager to state its claims and air its grievances in the public square.  Atrocity stories – deaths, kidnappings, disappearances – are the daily stuff of screaming headlines and prime-time TV.  To accept the claims of the Hopkins researchers, one must first assume that the press of Iraq – free, diverse, and ubiquitous — misses nine out of ten killings on its own turf.  No other research group shares this assumption.

But if the violence in Iraq is well known, and fulsomely reported, a nagging question remains:  Why shouldn’t a careful extrapolation, a la Lancet, approximate the numbers compiled by passive (i.e. surveillance-based) means?

Baseline bungling
The conditions in which passive morbidity calculations should be rejected do not exist now. But they certainly existed under the Ba’ath.  Scholars of civilian mortality place the daily Saddam-era toll of regime-caused death between 75 and 125 citizens per day – roughly double-to-triple the average post-war mortality reported by the body-counters.  (The death toll of the past two months, unusually bloody in four of Iraq’s 14 provinces, has in fact matched this range.)

The variability in scholarly opinion on the Ba’ath-era death-rate has two roots.  First, when the Makhabaret was trundling dead Kurds and Shi’ites into group graves en masse, its was hard to maintain a precise registry.  Stalin, Saddam’s idol, had the same problem in the Ukraine.  

Second, scholars debate which corpses should count.  Just as students of post-war Iraq argue the status of street-crime victims, so the scholars of the Ba’ath quarrel over the half-million troops dead in Saddam’s wars.  These certainly were victims of polity:  The invasions of Iran and Kuwait were “wars of choice”.  But researchers generally exclude fallen Iraqi conscripts from Saddam’s body-count, leaving Ba’athist hands somewhat less bloody.  Veterans of the Iranian War may disagree.

The Hopkins researchers chose their “base-line” for pre-invasion Iraq carefully: January 1 2002 to March, 2003.  They chose to characterize Ba’athist violence by a period during which the Kurds were sheltered by a U.S.-imposed no-fly zone in the north.  They chose to calculate the “pre-war” death rate after the extermination of hundreds of thousands of Shi’a and Marsh Arabs in the South. 

Burnham and associates carved out a brief period of enforced peace within a 25-year regime singularly dedicated to war and internal slaughter.  They called it a “baseline.” And they compared that baseline against a period of war.

As a result, 300 of the 302 conflict-related deaths they report are “post-invasion.”

One wonders, though — Had they built their baseline in the Kurdish North to include the Anfal operation – a months-long assault that demolished thousands of villages and killed hundreds of thousands of peasants — whether Kurdish respondents might have recollected more than a death or two.  One wonders, had they extended their “pre-war” baseline to the end of Desert Storm, whether the families of Basra and Muthana mightn’t have recalled another loved one lost. 

A remarkably small number of additional pre-war deaths recollected by the 12,801 interviewees would have wiped out the “findings” of this study altogether. 

The Lancet authors consciously manipulated their pre-war baseline to advance their pre-selected post-war conclusion – that Saddam’s Iraq was less violent than Bush’s U.S. or Blair’s U.K.  All they have actually proved is that Excel reliably performs multiplication operations on data fed through it.

The pity is that this flawed John Hopkins study, widely ignored by war scholars, will be mindlessly cited by the anti-American minions of the mainstream press.  But that, after all, was its purpose.

This article is reprinted by permission of National Review Online


2 Responses to “Stupid Time at the Lancet”

  1. John Says:

    Mr. Nadler,

    You state that scholars of civilian mortality rates cite death rates of 75 to 125 per day caused by the regime in Saddam’s Iraq, yet fail to mention one, or their methodology. Saddam ruled for over 8,000 days so we are talking about over 800,000 dead civilians. Yet the Anfal campaign (conducted 15 years before the invasion of Iraq) accounts for no more than 182,000 dead, also the Shiite and Kurdish uprisings resulted in 60,000 dead (how you come to the conclusion that all Shiites and Kurds who took up arms to overthrow their governments are “civilians’ you will have to explain). So that is 250,000 dead Iraqis from the two major instances of mass casualties in Iraq. This number has yet to be confirmed in the excavation of mass graves in Iraq as something on the order of less than 50,000 bodies have been uncovered. So I assume that you include in the total the number of Iraqis killed in the Iran/Iraq War and the Gulf War (regardless of who killed them and why), but Iraq cannot be held responsible for these deaths as Iran is as much to blame for the Iran/Iraq was as Saddam was, so it is patently false to say that all these deaths were regime sanctioned.

    Secondly, Saddam’s Iraq post-1991 was clearly less violent than the current incarnation, in the run-up to the war. The study is looking at how many excessive deaths were caused by the war that would not have otherwise occurred, so it is pretty pointless to go back 15 years ago, when the level of violence was different. Also the inclusion of the media in Iraq does not meant that they will give an accurate accounting of the war dead. Reporters do not track down every single death, and in fact the Iraqi Health Ministry reports three times as many deaths from violence as the media does. Perhaps the Lancet report is a flawed piece of hackery that doesn’t deserve the light of day, but your shoddy scholarship refuting it is hardly any better.