Date - 22 June 2004
Some time ago I wrote about the speed camera partnerships being rubbish at statistics and effectively using regression of the mean to inflate their effectiveness. [link] If you've forgotten what regression to the mean is, a brief explanation is at the bottom of this page.
At the time I suggested that either they were being dishonest and using bad statistics to further their cause, or they were simply stupid.
Never attribute to malice what can be explained by stupidity. - M.N. Plano
Now they have a big list of sites where cameras were not effective - 743 of them to be exact. [link]. If they remove these cameras - as popular wisdom would suggest you should - and then measure the changes you run into another set of statistics where regression to the mean will have a big effect.
As demonstrated previously, if you divide a population into two brackets, high and low, we expect that afterwards both will regress towards the true mean, so the average of the high numbers will drop, and the average of the low numbers will increase. In this case we expect that at 'unsuccessful' cameras [high accidents] the number of accidents will fall, and that at 'successful' cameras [low accidents] the number of accidents will increase.
Oh Dear
I predict, that purely by the appalling use of statistics practiced by the Safety Camera Partnerships, that they've shot themselves in the foot and that, the statistics for the next few years will show that accidents decrease when cameras are removed and will increase slightly when cameras are left. This does not mean that the cameras are causing accidents, but, If you believe that the current decrease in accident rate is purely due to speed cameras, you must also believe that the subsequent increase I predict will be due to speed cameras.
The Camera Parnerships are screwed, in order to use their simplistic measures 'look at the statistics at the camera sites', you have to accept the conclusions 'remove the cameras that don't work' which bring on the result 'cameras cause accidents'. They're only way to avoid the 'cameras cause accidents' charge is to use proper statistics in the first place, but they've spent far to long ignoring real statistics because the numbers didn't look as good.
Start removing cameras that have an excellent safety record, that will bias the regression to the mean in the other direction.
Hope that cameras really have a massive effect on the accident rate such that more than half of the claimed reduction was real, rather than statistical. If that's the case the statistics will still work out in your favour but not that well.
Do arbtrary major works on the road so you can remove the cameras from the statistics - obviously this will be hard to pull off if you've put a camera on every road in your county.
Arrange a major accident for camera sites just after the cameras have been removed. So, for example, if a camera site had a poor safety record, encourage small children to play chicken with cars at the site. The publicity value of a bunch of squashed six year olds at a site that used to have a camera to your organisation is priceless. You could easily make up an 'economists can prove this is cost effective' argument too - compare the effectiveness of the publicity of a pair of squashed six year olds against £10,000,000 of advertising. This is of course complete rubbish - but probably better than the typical Safety Camera Partnership statistics.
Stop arguing about regression to the mean, they're not listening, the public doesn't understand or care. Simply push them to remove the cameras that don't work and keep the ones that do and wait for the statistics to fall in your favour. You should also push for the installation of cameras at places with no accidents, e.g. outside schools, hospitals etc. since the accident rate can only go up.
Be prepared, the statistics are going to show that cameras cause accidents, make sure all your publicity material is ready.
Do your statistics right, otherwise your flawed methodology might bite you in the arse.
It occurs when a random variable X is used as the basis to select subjects to measure a change in X. A simple example is the number of times a dice rolls a six. If you roll ten dice six times each, some will not roll a six, some will roll one, some will roll two, but on average it will be one six per dice. If you then take only the dice that got two or more sixes, on average for those dice that's two sixes per dice. If you redo the test with just those dice the new average will again be one six and it appears that you've reduced the number of sixes per dice - you haven't it's just that your initial selection had an artificially high mean and it's now regressed to the true mean on the second test.
I've been told this article is offensive. It is. It is not offensive because it uses the word 'fucked' in the URL. It is offensive because it seriously presents the option of murdering small children as a cheap form of advertising.
"The party has pledged to review all cameras in England and Wales and scrap those which have not reduced accidents if they come to power."
A review of speed cameras has been called for in Blackpool after a report revealed the number of accidents has increased in 11 out of 50 sites.
ROAD smashes doubled on a mile-long stretch of motorway AFTER a speed camera went up. And now traffic chiefs have sparked fury by installing two MORE of the hated yellow traps in the same place.
(There are no where near enough statistics here to conclude a change in accident rate. However, becasue the partnerhips persist in using a single year of figures as 'proof', they have no counter argument when the random fluctuations don't go their way.)
Compare and contrast, the executive summary :
Both casualties and deaths were down - after allowing for the long-term trend, but without allowing for selection effects (such as regression-to-mean) there was a 22% reduction in personal injury collisions (PICs) at sites after cameras were introducted. Overall 42% fewer people were killed or seriously injured.
emphasis mine. They've attemped to quantify the regression to the mean though in appendix H :
P143 : For this part of the analysis data were required concerning the numbers of PICs and FSCs (for which predictive models are available) rather than KSIs (for which they are not).
In English - it's not possible to correct the published KSI data.Using the FSC data as the closest match to the published KSI data we discover :
The overall average observed reduction in FSCs is 55%. After allowing for trend and RTM effects, the overall average reduction in FSCs attributable to these cameras is 10% of those observed in the baseline period. RTM effects account for a fall of 35% with trend accounting for a further fall of 9%. Thus RTM accounts for about three fifths of the observed reduction in FSCs with the effects of the cameras and trend each accounting for a fifth.
emphasis mine. But it gets worse, once you account for the errors in the measurement, for the fixed cameras the scheme effect is between +8.7% and -39.5% (table H7). This means the DTI can't currently disproved the statement 'Speed cameras cause accidents.'.