Most of my half-dozen readers will by now have heard about the research conducted by Les Roberts, Riyadh Lafta, Richard Garfield, Jamal Khudhairi and Gilbert Burnham, published in the current issue of The Lancet, estimating the number of civilians killed during the war against Iraq (so far). If you have not already done so, please read the paper; it is interesting, and will tell you much more about it than any number of `about the research' pieces in the newspapers or (god forbid) on people's web logs.
In this case, the lack of precision does not hinder the clear identication of the major public-health problem in Iraq-- violence.
Very briefly, the way the study was conducted was to conduct surveys of clusters of households randomly assigned to bits of Iraq weighted by population. Each survey asked householders to record deaths of members of the household in a period prior to the invasion of Iraq and in a similar period after the invasion. From these data they estimated the death rate before and after the invasion, and from this formed an estimate of the total number of excess deaths from the invasion.
During September, 2004, many roads were not under the control of the Government of Iraq or coalition forces. Local police checkpoints were perceived by team members as target identification screens for rebel groups. To lessen risks to investigators,we sought to minimise travel distances and the number of Governorates to visit,while still sampling from all regions of the country.
-- and the survey certainly isn't as accurate as the ideal which could be expected in a country at peace. The headline figure -- of a conservative estimate of 98,000 deaths due to the war -- is the center of a 95% confidence interval which stretches from 8,000 to 194,000 deaths. (Note that this figure does not include information from Fallujah, an area which has suffered more violence than most and in which it was very difficult to do the survey. Including the figured from Fallujah increases the estimate by about 200,000 but broadens the confidence interval further.)
Specifically, under the assumptions of the survey, the authors estimate that there is a 95% chance that the two limiting values they obtain enclose the true number of deaths. Loosely, you can turn this around and regard this as a 95% chance that the true value lies in the interval (and hence a 5% chance that the true number lies outside that interval). Further (if we assume a symmetric distribution, which is plausible), there is a 50% chance that the total number of deaths exceeded 98,000, and a 2.5% chance that it was less than 8,000.
To put this into English, it means `we're not sure exactly how many, but a hell of a lot of people almost certainly died'. Make up your own mind, but in my view the low estimates of the number of dead in Iraq should now be regarded with deep suspicion.
There have been several criticisms of this study, most of them rather silly. Tim Lambert and Daniel Davies on Crooked Timber discuss and rebut a variety of complaints. A few more criticisms should, perhaps, be added:
``100,000 is a lot more than the number of casualties estimated by the Iraq Body Count. Therefore it must be wrong.''
Actually this discrepancy doesn't tell us anything about the relative accuracy of the two studies. The Iraq Body Count is counting civilian deaths reported by the news media, while the Lancet study is estimating the total number of excess deaths caused by the war. This is also the answer to a criticism made by a commenter on Crooked Timber, who wrote,
One criticism that you dont take on here is the observation that 100,000 people in a year and a half is 183 people a day. This would imply that there'd be some days when thousands of extra people died, which would surely have shown up in the news somewhere.
Well, no. That's not how newspapers work. Journalists report events -- they don't go out into the streets to count every dead body, especially when many of those deaths are the indirect consequence of violence. And in the places where violence is going on, there probably aren't a lot of journalists on the streets anyway.
This one's much more interesting. The Lancet study reported an infant mortality rate of 29 per 1,000 live births (95% confidence interval 0 -- 64), as compared to a UNICEF estimate of 107 per 1,000 live births. (For comparison, the value for the UK in 2003 is 6 per 1,000 births.)
Two immediate comments: firstly, the UNICEF figure has no error estimate, and so is difficult to compare to the new estimate; secondly, the old Iraqi régime may have inflated infant mortality figures to support the argument that sanctions were responsible for a large rise in infant mortality. (I can't justify the latter argument -- it's just a hypothesis; in any case, the Coalition Provisional Authority quoted a similar figure, 108 deaths / 1,000 births, in June this year.)
One other possibility: the researchers counted the numbers of births and deaths among the interviewed samples over a fixed interval. This will only give an accurate estimate of the infant mortality rate if the birth rate doesn't vary substantially (a child who dies during the survey period may have been born before it). However, to explain the discrepancy, a large increase in the birth rate in the immediate pre-war period would be required, and there's no reason to suppose that had happened.
This issue, unlike most of the others, is a bit troubling. But the bulk of the deaths estimated by this study occurred among adults, not children, so -- unless this is evidence for a wider problem -- this doesn't affect the basic conclusion; in any case, the survey compared the households' pre- and post-war circumstances using the same methodology, and it is the ratio of pre- and post-war mortality rates from which the estimate for the total numbers of deaths was derived.
Another source of criticism of this research has been 10 Downing Street. Last Friday, the Official Spokesman complained that,
Firstly, the survey appeared to be based on an extrapolation technique rather than a detailed body count. Our worries centred on the fact that the technique in question appeared to treat Iraq as if every area was one and the same. In terms of the level of conflict, that was definitely not the case. Secondly, the survey appeared to assume that bombing had taken place throughout Iraq. Again, that was not true. It had been focussed primarily on areas such as Fallujah. Consequently, we did not believe that extrapolation was an appropriate technique to use.
Almost all national-level statistics are based on extrapolation (or, more accurately, sampling). Are we supposed to believe that (say) the Census or (for another example) Government research showing public support for ID cards are worthless because they did not count each individual living person in the country?
The implication that no casualty figures could be accurate unless they are derived from a `detailed body count' is also absurd, especially given that Coalition forces have refused to conduct any such research; in any case, a `body count' would severely underestimate the total number killed -- partly because many bodies will not be recovered (for instance, those killed when bombed buildings collapse), and partly because it's now impossible accurately to count the bodies of those who have already died and been buried.
Further, the survey did not treat `Iraq as if every area was one and the same', as even a cursory inspection of the paper will tell the reader. Similarly, the survey did not `assume that bombing had taken place throughout Iraq'; instead, samples were taken at numerous locations in order to account for the geographical distribution of damage (many of the sampled areas were unbombed, as you would expect). Specifically, as I have remarked, the headline number excludes Fallujah, because of the high concentration of bombing and difficulties of conducting the survey there.
And yesterday, the Official Spokesman was at it again, repeating the same false statements and adding some new stuff to confuse the lobby journalists:
Asked to explain further the Government's previous concerns and doubts about the methodology applied in the ``Lancet'' article about the number of Iraqi deaths, the PMOS replied because it relied on the extrapolation technique assumed, that Iraq was uniform in terms of intensity of conflict. It wasn't. The article also assumed that bombing was general throughout Iraq, which was not the case. The Iraqi Department of Health had issued figures that showed over a 6 month period there were about 3,000 deaths, which was a long way short of the figures quoted in the ``Lancet''. The Iraqi DOH measured those figures by the number of people who came into hospitals throughout Iraq, and it was very difficult to rely on any such figures quoted in the ``Lancet'' with any certainty.
So: the Iraqi Department of Health counted the number of people who came into hospitals and then died. And they got a different figure from a survey which attempted to estimate the total number of people who died, whether in or out of hospital. Quelle fucking surprise. It's -- at the risk of pointing out the obvious -- a war zone! People who get hit by a LASER-guided bomb don't have time to go to hospital before they die! Of course the fucking numbers differ.
What's going on here, then? I am not sure, and indeed am having trouble untangling my cynicism about others' competence from my cynicism about others' honesty. But my best guess: part of this is misdirection -- the idea, I think, being to try to convince journalists who may be too lazy to check the story properly that It's Not Quite As Bad As They Think; as for the rest, well, the charitable explanation is that the Downing Street press office doesn't know any statistics and doesn't know anyone who does. The more cynical explanation is that this is simple and shameful dishonesty, a feeble attempt to discredit a study which suggests that the war against Iraq has killed the very people it was supposed to save by the tens of thousands.
Moving on, many supporters of the war have been embarrassed by this study, apparently because they do not understand what a war is. For the avoidance of doubt, the fact that one hundred thousand people may have been killed as a result of the late unpleasantness does not necessarily mean that the war was a mistake or `wrong', any more than the fact that a third to one half that number were killed in a single day in Dresden in February 1945 made the war against the Nazis a mistake or `wrong'. (Of course, the civilians killed in Dresden were killed deliberately, whereas -- charitably -- most of those killed in Iraq have been killed recklessly.)
It may tell us more about the conduct of the war (is it really a good idea to supress an insurgency by dropping five hundred pound bombs on urban areas, even if they are LASER-guided?); but in my opinion it is more telling that many apologists for the present war -- usually the first people to compare Saddam to Hitler and Iraq in 2003-4 to Germany in 1939-45 -- have confused these two issues. In wars, people die, often by mistake and often in large numbers. The case the war apologists have now to make is not about the exact numbers of deaths -- later, we will better know how many died, but for the moment 98,000 is the best estimate available -- but whether the deaths were necessary.
Lastly, as a brief comment on the conduct of the war, I will draw your attention to this film taken from an F16 flying over Fallujah in April. (Don't pay too much attention to the commentary on that page, but the video is linked from there; sadly it's in the proprietary -- and rubbish -- Microsoft `Media Player' format, but `xine' will play it.)
Channel 4 News covered this footage, which shows a pilot guiding a bomb towards a building, presumably under instructions from a forward air controller on the ground. While the bomb is falling, a crowd of people appears in the street adjacent to the building and the pilot asks whether he should direct the bomb onto them. The person on the ground tells him to, and he does.
There is some controversy about whether the people bombed -- from a rough count there are thirty to forty people on the video -- were combatants or not. You can read about that in the sites I've linked to. That's not what I want to draw your attention to, though. Here are some frame grabs from the video (lousy quality, sorry). First, a general view from before the crowd emerges, to give a sense of scale:
Note cross-hair, showing the original target of the bomb, designated by a LASER shining from the 'plane onto the ground. The pointing of the LASER can be changed as the bomb is falling, allowing the target of the bomb to be altered. The street in the center of the picture is quite wide -- about 40 meters, judging from the size of the people in the second picture, and it is surrounded by two- and three-storey buildings.
(As an aside, remember that the controller on the ground may have a much better view of proceedings than the pilot -- these people may have been running towards his position shooting at him, for instance, and we would not be able to tell that from the video.)
Bearing in mind the width of the street and the nature of the surrounding area, consider this:
Safe distances for unprotected troops are approximately 1,000 meters for 2000-pound bombs and 500 meters for 500-pound ones. Even protected troops are not entirely safe within 240 meters of a 2,000-pound bomb or 220 meters of a 500-pound bomb.
During the war proper -- before the `Mission Accomplished' stunt -- the Americans dropped about 18,000 guided bombs. (This ignores a further 10,000 or so `dumb' bombs.) As a brief estimation exercise, if all of those were dropped in urban areas with a population density of 5,000/km² (probably an underestimate), then for a uniform distribution of targets we'd expect around 100,000 casualties from bombs with a 20-meter lethal radius. That's certainly an overestimate (even the US Air Force no longer bombs at random) but the order of magnitude is... suggestive.