[ Home page | Web log ]
Francis Irving, Martin Keegan, various others and myself have been working on Downing Street Says, a site which scrapes the transcripts of briefings given by the Prime Minister's Official Spokesman, and the Prime Minister's occasional press conferences, and sticks them into a web log which breaks up the text by topic (where possible) and allows readers to comment on what the government is saying, just like lobby journalists can. (Credit where credit is due: Francis wrote the screen scraper for briefings; I wrote the one for the much-more-occasional press conferences, and various other supporting scripts; Martin helped me set up the execrable Movable Type web logging software; and the various others assisted with domain names, helpful suggestions, etc.)
I should say that I am not a crazed techno-idealist and I do not believe that web logging and web loggers are going to change the world. I don't believe that simply sticking government documents into a web log will make any difference to the governance of the country. My hope, however, is that people will find the thing sufficiently interesting that they will write useful comments on it; some readers, with any luck, will be better able to do in-depth research of the issues behind the briefings than lobby journalists -- who cannot be experts on everything, work to tight deadlines and, who knows, might in a few cases even be a little lazy -- and if members of the lobby use the site too, then it may contribute to more searching and informative questioning at future briefings. I hope I'm proved right.
Since I've now done my idealism for the week (that was the previous paragraph, in case you didn't spot it) I will now return you to your regularly-scheduled cynicism. Last Monday I went to a thing called Con Con UK, a recap of various talks given at EtCon, the O'Reilly `Emerging Technologies' Conference a little while ago. Frankly there was very little to say about this, and what there is to say isn't very complimentary. A few things stick in my memory:
The most interesting ConCon talk was given by Tom Steinberg on lessons from the Howard Dean fiasco in the United States.
In my opinion, the real lesson of the Howard Dean campaign is nothing really to do with technology or the internet. Rather, it's an analogy with the 1983 Labour manifesto, `the longest suicide note in history'. Like the Labour party of 1983, the political internet is a machine for taking the most extreme views of cranks and nutters and distilling them into an unelectable political platform.
(I didn't say this on Monday, but it's relevant.) A while ago, Anthony Wells wrote about the idea of a `New Political Movement' growing up in the UK, using the internet to build support, as Howard Dean did. He identified three groups of people whose views aren't represented by the main parties -- far-left `anti-globalisation' protesters; younger, socially liberal people; and far-right racist fucknuts who read the Daily Mail -- and who could therefore benefit from such a movement, and concluded,
If new political movements do spring up on the internet, it is my fear that they will be those of the extreme right, and extreme left.
Exactly this effect seems on the cards for the completely bonkers Your Party, an internet-based political party which intends to determine its policies through opinion polling. James Graham points out (you may have to search for `Your Party, Your Fudge' to get to the right article) that Your Party is already in tremendous trouble, because all the people they've asked about their European policy either (a) hate the EU, or (b) think it's a jolly splendid thing. As a result, they've been forced to select for their platform a completely anodyne statement about the EU, but of course that just means that the answers to the follow-up questions have drifted towards the extremes.
The internet, as a political tool, is a kind of LASER for idiocy: idiocy bounces back and forth between a bunch of web logs, reinforcing itself into a coherent beam of insanity, which can then be focussed on a political problem... on which it has no effect at all.
For the obligatory Silly Graph expected by my readers, here's the histogram of results from Am I Sig Or Not?, the silly web poll where you get to decide which of my email signature quotes you like. (You can see the best and worst quotes, according to the Internet's unwashed masses.)
I was on holiday last week, hence the lack of posts. Like many other web-loggers, I'm going to post a few holiday photos, but by using them to point out an interesting meteorological phenomenon, I'll pretend that this post actually has content:
These were taken at an altitude of about 2,000 meters above sea level, near Alpbach in Austria. The brown colour in the clouds is caused by Saharan dust, blown north across the Mediterranean. The visual effect was really striking, especially since we had, at the time, no idea what caused it. It's a cliché to call such things `apocalyptic', but that's really how it looked. One person (not me, before you ask) half-jokingly suggested that a nuclear war had destroyed everything outside the valley we were in, and that (a) what we were seeing was the smoke; and (b) we should enjoy ourselves while we still could.
I've white-balanced the pictures above to make the snow look white, which is more-or-less how it seemed to the eye; the pictures come out of the camera looking even more freaky:
The clouds are also mentioned in this Reuters piece, which points out their effects on glacial melting: the dust is dark, and when it settles on the glaciers, it decreases their albedo and makes them melt faster in the summer. As one of my companions said, now is the time to go skiing in the Alps -- while you still can....
A long time ago, Winston Churchill is said to have said that `the best argument against democracy is a five-minute conversation with the average voter'. Similarly, one of the best arguments against a free press may be to read the work of Melanie Phillips, who lately dressed up an attack upon playwright Sir David Hare with some statistics which she supposes discredit his recent play about railway privatisation, The Permanent Way.
The Times reports that,
Adrian Lyons, director general of the Railway Forum, an industry lobby group, wrote to Sir David before opening night asking him to correct the plays central message that privatisation had made the railways dangerous. Mr Lyons pointed out that the rate of train crashes had halved since BR was broken up and sold off in 1994--96. While the privatised industry had suffered a series of high-profile crashes at Southall, Ladbroke Grove, Hatfield and Potters Bar, none had resulted in as many casualties as the disaster at Clapham in 1988, in which 35 people had died.
and Phillips, displaying the same willingness to bend the truth that had me chortling all the way through her recent piece of twaddle about global warming, claimed that this showed that,
Since the driving point of the play appears to be that privatisation was responsible for rail accidents and deaths, this would appear to be a pretty damning blow to Hare's credibility.
Well, maybe, maybe not. Like me, Melanie Phillips hasn't seen the play, but let's assume that her precis is accurate. What about those rail accidents then?
Mr. Lyons makes two claims: that the rate of train crashes has halved since privatisation; and that no crash as bad as the Clapham one has happened since privatisation either. Both of these statements are very carefully chosen. Each, as it stands, is true. The number of train crashes (or, as the HSE describes them, `significant train incidents') has been falling since privatisation (as can been seen from the HSE's annual report on railway safety; the relevant plot is on page 30), from 0.47 per million train miles to 0.20 per million train miles last year. And there has, happily, been no accident as ghastly as the 1988 Clapham accident since then.
But this information isn't really relevant. I -- and, I suspect, the rest of the travelling public -- probably don't care whether they are involved in a `significant train incident' unless they are killed or seriously injured; and they probably don't much care how many others are killed or injured when they are involved in a crash. A more sensible approach is to look at the total number of deaths in rail accidents.
As ever, it is hard to draw any very useful conclusions simply by looking at the plots. Testing the hypothesis that either deaths per year or deaths per passenger kilometer have been falling yields the following:
|Variable||Mean pre-1993||Mean post-1993||t-statistic||p-value|
|deaths per year||14.3||10.3||0.941||0.821|
|deaths per billion passenger km||0.386||0.240||1.435||0.917|
The problem with the latter -- beyond the rather weak significance of the result -- is that there doesn't seem to be a very strong correlation between deaths and usage anyway, so analysing death rates on this basis is bogus: (R-squared is about 0.1.)
Now, as with road accidents, we suffer from the fact that rail accidents are generated by a very random process (it doesn't seem to be Poisson, as it happens) and so a naive analysis like this probably won't get us that much further.
But I would be very cautious to report that we yet know anything about the long-term aggregate effects of privatisation upon railway safety. Anecdotal and expert evidence about shoddy maintenance should give us more pause than these statistics, or those facts which Adrian Lyons so carefully chose to make his case.
At this point I should say something about the notion that `facts' about railway safety -- even if they were true -- can be used to `discredit' a work of drama about railway privatisation. (I'm obviously handicapped here by not having seen it yet and therefore can speak only very generally.)
I'm very unhappy with this idea. A play is, after all, a work of fiction (even if based in accounts of real events). Should the play be pulled from the stage if it is somehow discovered that privatisation has made the railways safer? Of course not, any more than an absence of evidence for effective witchcraft in eleventh-century Scotland should stop productions of Macbeth.
The playwright's job, in the end, is to entertain the public; the journalist's job is also to entertain, but starting from the facts. I have no evidence that David Hare has failed in this duty -- whereas Melanie Phillips certainly has.
Some weeks ago in the pub, I was introduced to some anarchist friends-of-a-friend, who were hell-bent on the overthrow of global capitalism. I thought this was sad, as it seems to me that global capitalism is a jolly good thing. In any case it's never nice to trample on the ambitions of others and so we had an interesting discussion about quite how global capitalism might be overthrown. I would say `watch this space', but as I say it's not me who's interested in overthrowing capitalism, so this would be quite the wrong `space' to watch; and anyway I think capitalism's got a good few years left in it yet.
Another, not entirely unrelated, topic which came up was the question of electoral systems. Tom Steinberg also dropped a similar question into an email a little whole ago, so I thought I'd recycle my thoughts from the pub. (Thus turning the purpose of this web log -- usually characterised as `giving my friends early warning of what I'm likely to bore them with in the pub' -- on its head. Whatever.) Arguably most of the following is obvious, but that's never stopped me before. (At this point I'm also going to say the words `Arrow's theorem', in order to stop anybody from sending me email complaining that I'd left them out. I don't discuss Arrow below, though.)
Presently MPs are elected by geographical regions. This has advantages (each constituent has an identifiable MP, who lives locally and can easily be contacted by the local citizenry; this was more important before the railway, telephone and so forth were invented), and disadvantages (chiefly, that MPs are likely to have parochial interests; this leads to `pork-barrel' politics, especially, for some reason, in the United States). Another problem with geographical constituencies is that they are vulnerable to gerrymandering, especially when the relevant boundary-setting authority is under political influence (as in parts of the USA -- see Economist articles passim).
As a thought-experiment, consider an alternative way to partition the country into constituencies. Instead of dividing it up geographically, we will divide the population into equally-sized groups, at random. We allow the members of each group to communicate with one another via a mailing list or interactive TV or whatever, and come the general election they vote and return a single MP. (The point here is that lots of people don't know their neighbours any more, so why should they be expected to vote with them?)
It turns out that this is a staggeringly bad electoral system, because each constituency is statistically identical. Just as an opinion poll sample of 1,000 people almost always yields more Labour than Conservative voters (at the moment), these random constituencies would turn overwhelmingly to whichever party had the greater support. Considering constituencies of 50,000 voters and a two-party election, the results will look like this:
|support for party in population||fraction of seats won by party|
-- not much good, unless you're one of the 51.0% of people, and even then you might balk at an outcome which would have shamed Saddam Hussein. (For the interested reader, the reason for this is that the probability that more than a given number of the voters will choose one party in any given random sample is given by a Poisson distribution. With a large number, N, of voters each with probability p of voting for party A, this looks like a normal distribution with standard deviation sqrt(pN) -- this is a very narrowly-peaked distribution. In this case it means that the electoral system collapses the whole dynamic range of its output onto the middle ~0.6% of its input. Since I'm rapidly gaining a reputation for filling my articles with supporting pictures, here's one to pad things out:
-- not promising.)
So random constituencies do not make for a representative electoral system. What if we allow people to choose their constituency ahead of time?
(The question which inspired this was, `why do we have MPs who represent geographical regions, rather than issues?' The answer to that is historical, rather than theoretical, but this thought experiment was supposed to be some kind of answer to how you might go about doing that.)
Let's maintain the condition that constituencies are equally sized and fixed in number. We need a fair algorithm for filling the constituencies with voters. So, we propose a random first-come, first-served procedure, in which each member of the electorate is asked in sequence which constituency they would like to join (by number perhaps). If that constituency is full, they are asked to choose again, until they have selected a constituency.
How does this behave?
It turns out that this isn't any good either. Suppose that you are a supporter of a minority party. Your aim is to win as many seats in Parliament as possible, which means putting 25,001 of your fellows in as many constituencies as possible. You agree to fill constituency no. 1 first, then 2, and so forth.
Your opponents (again imagine a two-party election) would like to minimise the number of seats you win. To do this, all they need do is put their voters into the same constituencies you've chosen; since there are more of them than there are of you, they can systematically outvote you in every constituency you contest, leaving them unchallenged in every other constituency. (This assumes that communication between a party and its members is public; this seems likely in practice. If it isn't, then this system degrades to the equally bad random-constituencies one.)
Right. Enough of this digression into bonkers electoral systems. What (if anything) have we learned?
The thought-experiments lead (circuitously) to the following observation: our current electoral system only works at all because political opinion is geographically clustered. If you remove the geographical element, you'll also have use something that's not first-past-the-post, unless you're prepared to put up with incredibly extreme election results.
Why do geographical constituencies work, then? My guess is that it's to do with the localisation of industry (in the broad sense). An urban constituency which contains a car plant will vote for protectionism; an urban constituency which contains a port will vote for free trade. A rural constituency consisting of sheep farms will vote for subsidies; one that consists of market gardens will vote against. Aldershot will vote for war, and Islington for peace (well, prior to New Labour, anyway). And so forth.
As industries becomes less geographically localised -- probably inevitable with the decline of manufacturing -- our geographical system will work less and less well. We can see this already in rural areas which are turning Labour as a result of migration from the cities. Farmers traditionally vote Tory because they were `in favour of the things everybody else is against': fox hunting, agricultural subsidies, conservative social policy, and so forth. Now, farming is dying and rural yuppies take their urban political views with them, and that increasingly means votes for Labour.
Going out on a limb, does this mean we should expect to see Conservative support for proportional representation some time this century?
This is a peculiar story. It turns out that the Ministry of Defence has more Swingfire missiles than it knows what to do with, and is disposing of them. (I think they're obsolete, or something.) Now, like those batteries which say on the side that you must dispose of them carefully and not just leave them in the wheelie bin for the local council to collect, anti-tank missiles can't just be chucked in the landfill. (Unlike batteries, Swingfire missiles contain a 7kg warhead and so the instruction `do not dispose of in fire' is to be taken seriously.)
Apparently what the Ministry of Defence (or, rather, their contractor QinetiQ, who used to be DERA) actually do is to take their spare missiles down to the beach, wait for the tide to come in, and then blow them up. Really.
The last time they tried this, they didn't tie the missiles down properly, and twenty of them supposedly floated off down the Bristol Channel. (The Swingfire missile is about a meter long and 17cm in diameter. It weighs 27kg, and so on the face of it the missile should be slightly too dense to float. It's possible that some heavy component is removed from the things before they're disposed of.) The contractors did, helpfully, inform the Ministry of Defence that the missiles had been lost, but only 14 hours after they drifted off (presumably during that time the QinetiQ people were trying to think of a plausible `dog-ate-my-homework' style excuse to explain what they'd done).
The local MP, Ian Liddell-Grainger, is apparently (and quite reasonably) upset that 140kg of high explosives is bobbing about off the coast of his constituency, ready to wash ashore and blow some unlucky voter into bite-sized chunks, should they be strolling along the foreshore at the wrong moment. (The Ministry of Defence assures us that the missiles aren't armed and can't go off. And as we know, all electronic equipment carries on working exactly as designed after being dunked in salt water for several months.)
Mr. Liddell-Grainger (who is a Major in the Territorial Army) decided to take matters into his own hands, and wandered down to Kilve, five miles west of Hinkley Point nuclear power station, where he found some fragments of missile. Donning his flak jacket, he then returned them to the Ministry of Defence, earning something of a rocket for `endangering the public' by doing so. (Though, as we have learned, the missiles were completely safe. Hmm....)
Stranger and stranger, the BBC now tells us -- based on information from a chap called Jim O'Halloran from Janes Defence -- that these missiles aren't the missing Swingfires, on the basis that they're too small and the fins are the wrong shape. Which seems plausible: the Swingfire has a diameter of 17cm, and unless Mr. Liddell-Grainger has very large hands, it's not likely that he could hold one like this: (picture adapted from this story at the BBC)
-- he seems to have got all 17cm of the missile between his thumb and forefinger, which is a bit implausible. (17cm is about ¾ of the short side of a piece of A4 paper. So if you crease a piece of A4 down the middle, then fold a quarter of it over, you can see what size the missile is.)
Further, the missile he's holding has fins whose length is about 35% the diameter of the missile; the Swingfire's fins are about 85% of the diameter of the missile:
(The 85% is obtained by multiplying the 60% shown above by sqrt(2) to account for the oblique aspect of the fin shown.)
So Mr. Liddell-Grainger has, apparently, found a completely different kind of missile washed up on the beach. The Bristol Channel must be swimming with the damned things. Forget North Korea, now is the time to disarm the Irish Sea!
(I have to say that, when as a child I went on holiday to the seaside, I didn't find any anti-tank munitions lying in the surf. I feel rather cheated. They would have brightened up many a dull seaside day.)
More seriously, I'd love to know how QinetiQ came up with its scheme for disposing of obsolete weapons. Wouldn't it be safer just to flog them to some third-world dictator? Of course, dumping obsolete weapons at sea is something of a tradition here, with old phosphorus incendiary bombs and other such junk regularly washing ashore on the Isle of Man and in Cumbria and Scotland, occasionally causing injuries. The Government don't even know how much stuff was dumped in Beaufort's Dyke (the deepest part of the Irish Sea), though there's at least 14,500 tonnes of phosgene shells down there.
(Actually, disposing of chemical weapons at sea probably was the sensible thing to do, though it was done in a fairly haphazard manner and in bad weather the ships carrying the stuff often tossed it over the side just out of harbour, which does not inspire confidence. No doubt disposing of the munitions in the deep ocean -- rather than coastal waters -- would have been more expensive, but with hindsight the additional cost sounds reasonable.)
But in the case of conventional weapons, surely they could just have buried them and then set them off, which sounds like it has less potential for cock-ups. Or dismantled them. Anything other than leaving them on the beach and having them wash away....
As a piece of incestuous web-logging commentary, I'd like to thank whichever of my half-dozen readers nominated me for the Guardian's political web log awards. However, the major effect of discovering this has been to make me feel guilty that I haven't written anything for the past few days (and that the last time that I did I made a glaring error). Oh dear. So I'll just pad this out by publishing something Tom asked about, relating to my piece on parliamentary coordinates.
Tom wanted to know which MPs have moved the greatest distance between the 1997 and 2001 Parliaments. This isn't a completely meaningful question, because the axes for the two Parliaments are quite different (they're defined by distinct sets of divisions). However, because the two axes seem to maintain their rough meaning between the two Parliaments, it's still possible to go through this exercise, though I'm not sure how useful it is.
Firstly, as mentioned before, the parties move a bit in total. Here's the mean change in position for the three major parties from 1997 to 2001:
To pick out `interesting' MPs from this bunch, I've looked for the MPs who moved the greatest distance relative to their party. (This is completely ad-hoc, by the way, and I'm not going to attempt to justify it.) Anyway, here are the results:
I've picked out the few most mobile MPs of each party. Some observations: (these are fairly tenuous; you may be able to draw your own, more interesting, interpretations; feel free to drop me a line)
Paul Marsden defected from Labour to the Liberal Democrats. George Galloway was kicked out of the Labour Party because of his stance on Iraq. Kate Hoey is pro-hunting, and that turns out to be very important on the x-axis in these plots. (Anne Widdecombe, who is anti-hunting, comes out left of the Tory pack for the same reason, though that's not shown in the above plot.)
Phil Willis and Mark Oaten both acquired front-bench jobs in the 2001 Parliament. Mike Hancock joned the defence Select Committee. I suspect that's responsible for their larger-than-average changes in position (but I haven't checked). (By the way, the Guardian's Aristotle tool is good for checking this sort of information; from Google, you can usually just type the MP's name and the word `Aristotle', then hit `I'm Feeling Lucky'.)
Michael Portillo left the front bench in 2001, yet moved in roughly the same direction as Stephen O'Brien, who became a whip in 2001, then Shadow Secretary of State for Industry in 2003. This is slightly surprising, and I don't know why it happened. Gillian Shephard (who didn't move very far, to be honest) returned to the back benches in 1999, which may explain the change.
Incidentally, in the above plots, I've used the same dodgy scaling I described in the previous discussion. Skipping this yields a plot which is dominated by MPs whose attendance record has changes:
-- so I'm content that the scaling, though dodgy, is useful.
Peter Hain, commenting on the current travails of the dear leader and his party, has apparently slipped out the proposal to elect members of a new House of Lords on the basis of party list elections run in parallel with general elections. (This is more-or-less the least representative and legitimate of all electoral systems, but that's by the by.) According to that Observer piece,
Yesterday, Hain outlined a compromise on Lords reform whereby peers would be directly elected according to the share of the vote gained by each party in Commons elections.
No party would be likely to gain a majority -- even the 2001 landslide gave Labour only 40 per cent of the vote.
Though The Scotsman differs, saying that the composition of the Lords will reflect that of the Commons, this making the upper house pointless:
Now part of the Upper Chamber could be appointed according to the relative strength of parties in the Commons in a compromise suggested by two Cabinet ministers.
Interviewed by the Times last week, Hain seemed, happily, to confirm the first interpretation of this:
Mr Hain favours all peers being directly elected under a system of proportional representation at the same time as MPs, even though he concedes that this would probably ensure that no party had a majority in the House of Lords and bring an end to the tradition of crossbenchers.
Though note also,
``What I favour is the secondary mandate. The same votes cast in general elections would go into a regional pool, each providing maybe 50 peers to a chamber of 600.''
-- meaning that this will be a pretty pissant election. The other 550 peers will be appointed by the existing independent appointments commission, which selects non-party-political peers. It's not clear whether Hain's proposal is that the only party political members of the Lords will be those elected on the party list, which, although a lousy system, probably wouldn't be too bad); or whether they will join a bunch of crony peers appointed in the traditional way, which would be ghastly.
(Update: no, I'd misread that, as Etienne Pollard kindly pointed out. According to the Times article, all the peers will be elected. It's the article in the Scotsman which suggests that the elected peers will serve alongside those from the independent appointments commission. So the previous paragraph is basically bollocks. Sorry.)
My prediction? It'll be a mess (but I'm not sure exactly what kind of mess).
As an aside, I remember a long time ago thinking that election to the House of Lords should be by lot, as with jury service, rather than by political appointment or direct election. After the last Lords reform cock-up, Charlie Stross described how such a thing could work. Worth a read, and a place in the annals of `fantasy electoral systems', anyway.
I'm not sure why I bothered to do this, but maybe it will amuse others of a certain disposition. Jacquelyn Arnold linked to this set of poll results in which respondents were asked to rate `swear words' on a scale from `very severe' through `fairly severe' and `quite mild' to `not swearing'. (I'd have thought a better design would be to ask people to order `swear words' by severity, or perhaps compare them pairwise, but maybe that doesn't fit NOP's methodology.)
Let it never be said that I could let a rather silly set of data go by without turning it into an equally silly graph. Here I've amalgamated the middle two categories into a single `sort-of offensive' one to give a chart with two degrees of freedom:
(As an aside, one correspondent pondered whether he was more worried that I'd plotted the above diagram, or that the data had been collected in the first place....)
I've marked in red the words which fall into the FCC's list of seven dirty words; I don't think these translate very well to the British context, as you can see from the fact that they're scattered all over the plot. You can also get the chart as a PDF file if for some reason you want to print it out and hang it on your wall or something.
One thing troubles me about this survey. Do you suppose that (for instance) the 24% of people who apparently think that the term `Paki' is `not swearing' don't think that it's offensive at all? Or are they just using a narrow definition of `swear word'?
Some time ago I suggested to Francis Irving that he apply principal components analysis (as used in my Political Survey and elsewhere) to parliamentary voting records. Francis very kindly sent me an enormous file full of data so that I could do all the work myself.
Some background is available at the Public Whip FAQ page; basically, from time to time Parliament divides (votes), and MPs who are present may vote `aye' (yes) or `noe' on the motion. There is no usual way for an MP to register an abstention (occasionally one may vote `aye' and `noe' in a single division, but this is rare and abstention and absence can't usually be distinguished). According to The Public Whip's statistics, the mean attendance in divisions is about 62%.
The simplest representation of these data is to consider each MP's voting record as a vector of all the divisions in which they voted. Each element of the vector is +1 if the MP voted `aye' in the corresponding division, -1 for `noe', and 0 for `not present', which could be the result of laziness, abstention, or the MP having better things to do with their time. (Tony Blair, for instance, rarely votes; Gerry Adams never does.) Note that the assignment of -1/0/+1 is arbitrary. Because many divisions are rather sparsely attended, I've restricted my analysis to divisions in which 500 or more (out of 659) MPs vote; this is arbitrary too, but has the effect of restricting the analysis to the to 15-25% of divisions with the largest attendance. These are likely to be the most controversial, and therefore interesting, divisions, since otherwise all those MPs wouldn't have bothered to turn up for them.
Anyway, given this set of vectors we can find the covariance matrix for the divisions, which tells us how the votes in one division correspond to votes in others. If every MP voted the same way in two different divisions, the covariance between those divisions would be large and positive; if every MP who voted `aye' in the first division voted `noe' in the second, the covariance would be large and negative; and if the votes in the two divisions were completely unrelated, the covariance would be about zero.
Principal components analysis picks out the combinations of the divisions which (in some sense) best explain the variations in the data. I wrote some notes about the procedure for the Political Survey; the procedure is the same here, except that instead of survey questions we have divisions. The idea is that we'll extract from the data political axes which describe how MPs behave.
The Public Whip has data from the 1997 and 2001 Parliaments. I've looked at both. The results are not earth-shattering, but they are somewhat interesting....
First, how many significant axes are there? I can't be bothered to do this properly (by sampling from marginal distributions), so instead here's what is apparently called a scree plot, a plot of the first few eigenvalues:
-- from this we conclude (by handwaving, basically) that the first two eigenvectors are significant. The first eigenvector is much more important than the second. We shall see why in a moment.
Plotting the data for the two parliaments yields the following. First, 1997:
Observe that the MPs cluster by party affiliation. This is promising, and suggests that the whole thing may not be a total waste of time. (I've picked the signs of the x and y coordinates to put Labour on the left and the Conservatives on the right; obviously this is an arbitrary distinction.) Some comments:
The fact that MPs don't attend all votes is a pain, because it means we can't tell whether an MP lies outside the cluster of their party because they're a rebel, or because they're lazy. Two ways to fix this suggest themselves:
Both of these are ad hoc. The first one sounds more elegant, but in fact leaves the first eigenvector unchanged with the second eigenvector representing the voting fraction plus some noise. This produces a rather less interesting plot that looks like this: (these are data from the 1997 Parliament; the scaling is slightly different, but that makes no difference to the plot itself)
The second idea is easy to explain and hard to justify. We take each MP's position (x, y), and plot them instead at (x/f, y/f), where f is the fraction of divisions in which they voted. This should move them out towards the edge of the plot, to join their political fellows. Implicitly, this makes the assumption that MPs are failing to attend votes, rather than choosing to abstain: the scaling is moving them to the position that they would be in had they voted in all divisions in the manner suggested by their votes in the divisions they did attend. This is obviously not right, but it's not so catastrophically wrong as to be useless, either. Here are two views of the difference this makes for the 1997 Parliament data:
Applying this scaling, we get for the 1997 Parliament:
and for 2001:
Before making any more comments, we should look at what the two eigenvectors mean. The two are quite different. Looking at the absolute values of the components -- that is, how strongly the votes in the various divisions contribute to an MP's horizontal position on the graph, we get this:
-- that is, the top 90% of divisions contribute about equally to the axis. Basically these are the divisions in which MPs from the Labour and Conservative parties voted on opposing sides; it is a `government/anti-government' axis. The majority of divisions are like this; typical examples include this division in which Conservative and Liberal Democrat members called for a Civil Service Bill, and Labour members opposed them; or this division on an amendment to the Criminal Justice Bill which was defeated by Labour.
The second one is rather different:
It is dominated by a few divisions in which the Liberal Democrats opposed both the Labour and Conservative parties. Typical examples include this division on the case for an invasion of Iraq, or this on the withdrawal of state support from the families of asylum seekers. Essentially this is an `axis of Lib-Demmery'.
The actual list of divisions and component strengths which make up the two axes are here:
-- there are links from those pages to the pages for each division at The Public Whip.
Although the divisions voted on in the two Parliaments are obviously different, the axes retain their meanings between the two. So we can compare the positions of the parties between the two Parliaments. Basically we find,
These two points are consistent with Liberal Democrat strategy to move right better to compete with the Conservatives for votes in Conservative/Liberal Democrat seats. (Remember the `decapitation' strategy?) The change in Tory position is, I think, too small to say much about, but it's suggestive of another chapter in the endless Tory search for `clear blue water', except that here we're talking about separation from the Liberal Democrats, rather than those post-Thatcherites on the Labour benches. (And note that behaviour in Parliament is not interchangeable with policy as advertised to the public. The subjects of most divisions are chosen by the government, not the Conservative or Liberal Democrat parties; Opposition parties can't control the agenda in Parliament in the way they can in their manifestos and campaigns. So the waffle above should be taken heavily salted.)
In 1997, the Conservative Party contained two quite distinct clumps; the rightmost of them contained 32 MPs, but it's not clear what they had in common. That group included among others Michael Heseltine, who famously `bought his own furniture'; Iain Duncan Smith, who later became leader; ex-minister Brian Mawhinney; alleged vulcan John Redwood; Bill Cash, the veteran Euro-sceptic; Edward Heath, the embittered leader from the 1970s; and Michael Colvin, who died in 2000. Their attendance at divisions varies from 54% in the case of Michael Colvin (who died in 2000) to almost 86% in the case of Duncan Smith, so the cluster doesn't seem to be an artefact of the scaling procedure. If somebody more familiar with the Tory party would like to tell me what these people have in common, I'd be much obliged. (The whole list is: Paul Beresford, William Cash, Michael Colvin, Patrick Cormack, Stephen Dorrell, Iain Duncan Smith, Peter Emery, David Faber, Christopher Gill, Teresa Gorman, Edward Heath, Michael Heseltine, Peter Lloyd, Brian Mawhinney, Patrick Nicholls, Owen Paterson, John Redwood, Andrew Rowe, Richard Shepherd, Caroline Spelman, Michael Spicer, Anthony Steen, Peter Tapsell, Charles Wardle, Bowen Wells, Shaun Woodward, Tim Yeo.)
In the 2001 Parliament, there was no such second clump.
It remains to say that there's a fair amount of handwaving in the above.
Another interesting exercise is to see where notable political personalities fall on the plot. Here are some examples, for 1997:
And for 2001:
If your browser supports Java, you can use Julian Todd's viewing applet to explore the data interactively, rather than just trusting me to pick MP names at random. Follow links for the 1997 Parliament; or the 2001 Parliament. (Disclaimer: I don't really do Java. So you're rather on your own with these. I don't have a browser which does Java, but the `appletviewer' tool from the Java development kit seems to do the right thing.)
I'm not sure how useful any of this is as a way to look at MPs' voting records, but it's vaguely fun and seems to be novel. Drop me a line if you can think of any interesting ways to use the data. Hopefully Francis and Julian will implement this for their MP map....
Continuing on the spam theme, I just received junk mail from a company called `UKsoftwarehouse' who were stupid enough to put a UK mailing address in their email. So, I thought, time to call upon the splendid and worthwhile Communications Privacy Directive, which makes spam illegal within the European Union.
Foolishly I'd assumed that it would be a simple matter to complain -- I could forward them the spam, with my contact details, and they could visit their boundless wrath upon the spamming wankers at UKsoftwarehouse, extracting a juicy cash settlement to compensate me for receiving the sodding message in the first place (and to pay for tea and biscuits over at the Information Commissioner's place).
What is the number assigned to the line you normally use to access this email account?
-- very helpful to those of us living in the post-MODEM age; and,
I have clearly indicated any information which I do NOT wish to be passed onto the caller/sender of the messages in question.
Huh? Pass on information to the sender of the spam? Do I look like I want them to send me more junk? But the best question of the lot is,
4. Further Action
Has the receipt of these messages had any practical impact on you? (e.g. prevented urgent message from being received, costs incurred)
To which I answered,
Yes. 15 minutes to find, print and fill out this form: £20. Plus 26p for the stamp.
Somehow I don't think they'll be sympathetic to my claim. Oh well.
According to the Information Commissioner's website, I should expect a response
within 35 working days
-- truly this is business at the speed of thought....
So, then, that Hutton Report, eh? (Yes, I know I should have written this last week, but I didn't have time. And these thoughts are slightly unstructured. I'm sure you can all deal with that.)
Lord Hutton, given the rather loose remit
urgently to conduct an investigation into the circumstances surrounding the death of Dr. Kelly
had a free choice of terms of reference. By selecting particular terms of reference, he could have distributed blame among the Government and the BBC in more or less any way he chose. It is interesting that he selected terms which yielded the result they did. But even within his chosen terms of reference, I still think his conclusion about Gilligan's report was wrong.
Let's start with semantics. Chapter 6 paragraph 220 of the Hutton Report states: (emphasis mine)
The term ``sexed-up'' is a slang expression, the meaning of which lacks clarity in the context of a discussion of the dossier. It is capable of two different meanings. It could mean that the dossier was embellished with items of intelligence known or believed to be false or unreliable to make the case against Saddam Hussein stronger, or it could mean that whilst the intelligence contained in the dossier was believed to be reliable, the dossier was drafted in such a way as to make the case against Saddam Hussein as strong as the intelligence contained in it permitted. If the term is used in this latter sense then, because of the drafting suggestions made by 10 Downing Street for the purpose of making a strong case against Saddam Hussein, it could be said that the Government ``sexed-up'' the dossier. However, having regard to the other allegations contained in Mr Gilligan's broadcasts of 29 May I consider that those who heard the broadcasts would have understood the allegation of ``sexing-up'' to be used in the first sense which I have described, namely that the Government ordered that the dossier be embellished with false or unreliable items of intelligence.
(It's worth pointing out that Hutton here means `intelligence which was known at the time to be unreliable or false', since it's clear that much of the intelligence was in fact unreliable and false.)
I cannot remember how I interpreted the term `sexed up' at the time of Gilligan's reports, but a brief search of the newspaper archives suggests that it was not, in fact, generally interpreted as Hutton suggests. For instance, The Mirror seems to have been the first paper to publish an editorial on the subject, on 30th May 2003, the day after Andrew Gilligan's broadcasts on the Today programme. It read, (emphasis mine)
Voice Of The Daily Mirror: Spinner's pitch.
THERE is a terrible ring of truth about the allegation that 10 Downing Street ordered a dossier on Iraq to be ``sexed up''.
Even though the claim by an intelligence officer has been denied, it sounds like the sort of thing No 10 would say.
Note the subtlety. They aren't asking for lies. Just for the dossier to be more headline-grabbing.
And grab the headlines it did, with the stark charge that Saddam could mobilise weapons of mass destruction in 45 minutes.
This is not compatible with Hutton's claim that, at the time it was broadcast, the term `sexed up' was understood to mean `embellished with known incorrect information'. Indeed, a week later, on June 6th, the same paper reported,
Voice Of The Daily Mirror: Spelling it out.
Now we learn that Downing Street sent the report on weapons of mass destruction back to intelligence chiefs six to eight times.
No 10 says it didn't order the dossier to be sexed up.
What was wrong with it then? The spelling?
And we know that the report was `sexed up' in the sense of being exaggerated, selectively edited, and re-worded. For instance, the 16th September 2002 version of the dossier contained the claim that, (emphasis mine)
The Iraqi military may be able to deploy chemical or biological weapons within forty five minutes of an order to do so;
three days later, on the 19th, this had changed to,
Iraq's military forces are able to use chemical and biological weapons, with command, control and logistical arrangements in place. The Iraqi military are able to deploy these weapons within forty five minutes of a decision to do so.
There is no evidence that additional intelligence appeared which justified the strengthening of that claim. Similarly, an early draft of the report contained the statement that,
Saddam Hussein is willing to use chemical and biological weapons if he believes his regime is under threat;
later, the qualification `if he believes his regime is under threat' was removed. Christopher Hitchens argues unpersuasively that this doesn't matter; but it is clearly a change of substance, just like strengthening the `45 minutes' claim. In any case, the semantics of the term `sexed up' seem to me slightly irrelevant.
Yet Hutton claims that Andrew Gilligan's report that the dossier was `sexed up' was `unfounded'.
It is hard to defend Gilligan's reporting. He failed to take proper notes or record his conversations with Dr. David Kelly, and therefore left himself open to the charges put to him by Hutton. But the facts remain that,
We do not know exactly what Dr. David Kelly told Gilligan. But what he reported was compatible with what we know Dr. David Kelly told the Newsnight reporter Susan Watts. Particularly important here is the transcript of a conversation between Watts and Kelly on 30th May 2003. In particular, Kelly appears to have made the same allegations about the insertion of the `45 minutes' claim having been at the behest of Alastair Campbell, despite a general feeling that it was unreliable.
Unfortunately, Watts and Kelly in their conversation only allude to Kelly's previous statements, and Kelly back-pedals on some of his statements, for instance in, (page 5 of the transcript)
Watts: OK just back momentarily on the 45 minute issue. I'm feeling like I ought to just explore that a little bit more with you. The um... err.... So would it be accurate then, as you did in that earlier conversation, to say that it was Alastair Campbell himself who...?
Kelly: No I can't. All I can say is the Number Ten press office. I've never met Alastair Campbell, so I can't... But... I think Alastair Campbell is synonymous with that press office because he's responsible for it.
What Kelly told Watts is certainly consistent with the claims that Gilligan made: that instructions were received that the dossier be `sexed up'; that these instructions came from Alastair Campbell (though perhaps this was Kelly's own embellishment, using Campbell's name to anthropomorphise the Downing Street press office); and that the `45 minutes' claim was inserted or emphasised on these instructions.
None of this means that the particular claim that Alastair Campbell caused the `45 minutes' claim to be inserted into the dossier was true. But it does mean that it's plausible that Kelly made the statement to Gilligan. Kelly appeared to be -- was -- a credible source. His back-pedalling in conversation with Watts suggests that he may have told Watts -- and probably Gilligan -- more than he could be certain of. But Gilligan did not know this. If Gilligan's report repeated statements that Kelly had made -- something we can't prove, because of Gilligan's incompetence as a reporter -- then his report, far from being `unfounded', was perfectly proper. The question of embellishment of the dossier was certainly one of public interest, and Gilligan's report of Kelly's statements brought it to the attention of the public. (None of this detracts from Gilligan's failure to keep proper records.)
(As an aside, Kelly's conversation with Watts also addresses another interesting question about the `45 minutes' claim. The dossier repeated the claim that weapons could be deployed -- i.e., used -- or made ready for use within 45 minutes of an order to do so. It seems that Kelly did not believe that this was true. He felt that an incompetent interrogator had picked `45 minutes' out from the testimony of an informant, without understanding to what it applied. Kelly told Watts that,
I could give other explanations [of the claim], which I've indicated to you: that it was the time to erect something like a Scud missile or it was the time to fill a 40-barrel multi-barrel rocket launcher.... I mean I have no idea who debriefed [the source for the `45 minute' claim]. Quite often it's someone who has no idea of the topic and the information comes through and people then use it as they see fit.
We don't know whether Kelly meant filling a multi-barrel rocket launcher with rockets, perhaps with chemical or biological warheads; or whether he meant filling the munitions with the chemical or biological agent, which might well take 45 minutes, but is very unlikely to be done under battlefield conditions immediately before the weapon was used. Erecting a Scud missile might take 45 minutes, but then nobody was claiming that Iraq had any remaining Scud missiles and there's no evidence that they'd ever solved the very difficult problem of how to use a Scud to effectively deploy chemical or biological agents.)
I gather that we are now to have an inquiry into the quality of intelligence about `weapons of mass destruction'. (I remind readers that anyone who uses the term `weapons of mass destruction' as if it actually refers to a meaningful and distinct category of munitions is either ignorant or seeking to mislead, as I've said before.) I'm going to go out on a limb here and predict that such an inquiry will not conclude that the our intelligence services are incompetent and not worth the money they cost; but rather that honest mistakes were made as a result of underfunding, and the solution is to spend more on them. I refer those interested to Phillip Knightley's splendid -- but hardly faultless -- book, The Second Oldest Profession, in which he argues that as a result of historical accident and failure of oversight, the spy services of modern states have evolved into expensive, unimpeachable, self-justifying, and fundamentally useless organs. It would be foolish to pre-judge the results of the putative new inquiry, but certainly the evidence so far supports my guess....
This is all done with wwwitter.
Copyright (c) Chris Lightfoot; available under a Creative Commons License. Comments, if any, copyright (c) contributors and available under the same license.
Hosted and supported by