Any #stats or #math nerds out there want to offer me an opinion? One of my security programmes that I run will be tracking time-to-decision (typically measured in calendar days). We do between 50 and 125 decisions in a year, so there are only 5-10 data points in a typical month. As you can imagine, with any sort of human approval process, there will be outliers where things will go very quickly ("no way in hell") and some that will go very slowly.
I want to report on time-to-decision and I want to blunt the impact of outliers on our statistics. If there's one decision that takes 6 months and the others take a couple weeks, I don't want the one outlier making us look bad. The math question:
I was gonna use a trimmed mean, but reading about Winsorised means is also interesting. I was assuming I'd use a trimmed mean excluding the bottom 5% and top 5% and then report an average of the remaining 90%.
Anybody have better ideas? Anybody with opinions on trimmed v winsorised means?
@paco that’s actually a really interesting question. If you have 15 data points though, excluding the top and bottom 5% is only going to be one value at each end.
There’s something to be said for just visualising the data and inspecting, or for plotting the variance on subsets of the data - when you add an outlier in the date set, the total variance should go up a lot.
If you plot a years worth, does it look sort of like a normal distribution?
@blacktraffic Thanks. It’s no so much for graphing. I’m just looking at raw numbers to try to understand health of the mechanism. Are we getting faster? Slower? Like given this month’s number compared to last month and year to date, are we good/bad/indifferent? I should have been clearer. I don’t want to plot a year at a go, but rather measure month by month and make value judgements.
@paco btw, I would never just take any kind of mean of data without expressing the variance too, so I would plot points with error bars. If the error bars are stupidly large, remove the outliest outlier and repeat.
Add comment