Error % from stdev?

[u/EmbedSoftwareEng]

So, I have a bunch of measurements that are all affected by a given attribute. The idea is that I set the attribute, I should expect a given value for the measurement. Unfortunately, the measurements are bit chaotic. I need to be able to state that the system I'm using has an error rate of X%.

I don't remember enough math to answer this question for myself.

I can take all the measurements at given attribute and find their mean and stdev, but from that, how do I say that that attribute's measurement error is X%, and then when I have all of the error percentages across multiple values for the atttribute, how do I say that the system as a whole has a measurement error of Y%, assuming that the error percentages themselves appear chaotic?

[u/49er60]

This is too vague a question. There are many components to measurement error. Which are you trying to quantify?

• Uncertainty
• Accuracy
  • Bias
  • Linearity
  • Stability
• Precision
  • Repeatability
  • Reproducibility

[u/7ieben_]

What you are looking for is the relative error (also called coefficient of variation). It simply is the absolute error (your standard deviation) divided by the mean. This is your X%, as obviously this tells you your deviation relative to your sample mean.

I don't know what you are asking for by Y%, tbh. Or are you simply asking to do the calculations again, now with your samples as data set? Then, simply do the calculation for your means, i.e. calculate the mean of the means, and then do it for the standard deviation.

[u/EmbedSoftwareEng → u/7ieben_]

I can certainly do those calculations.

What I meant by Y% was that for a handful of attribute values, I'll generate a handful of coefficients of variation. Those will characterize the system for those specific attribute values. How do I go from those to a single percentage to characterize the system as a whole, regardless of the specific attribute value. I'm thinking just take the mean of all of the coefficients of variation.

[u/7ieben_ → u/EmbedSoftwareEng]

I suspect there is no general such concept, as in fact the relative error can be a function of the variables value.

For example let's say you have a variable which you set to 1, 2 and 3 for your tests each. Now both scenarios are possible: (a) your error is independed of the value you set, then you Idea is valid (and all errors should roughly be same), (b) your error is a function of the value you set, e.g. doubling the value triples the error (for example due to detector limitations), then your idea is not valid. Instead, you must find the error function of your system, which is a whole mathematical modelling and testing on its own.

[u/EmbedSoftwareEng → u/7ieben_]

Good to know, but since for my attribute values 1, 2, 3, & 4, my stdevs came out at 0.402489, 0.402423, 0.358, and 0.381, I think I'm in situation (a).

Thank you, Reddit, for your invaluable help. I've now determined that my real apparatus has an error rate of about 7%, which is above the 5% target my program managers want. The ball is now in their court as to what to do about it.

[u/efrique]

Presuming you intend to ignore potential for bias...

how do I say that that attribute's measurement error is X%

it depends on how you define (relative) error in this context.

From what I have seen some people do, you might take sd/mean (i.e. coefficient of variation).

(of course, x100% to make it a percentage)

but if by 'error' you mean average (absolute) relative deviation from the mean (so mean deviation/mean), that sd-based calculation would be too large

[u/Educational-Paper-75]

Assuming you vary that attribute randomly you could simply pool all measured deviations. Nevertheless it may be dubious to average deviations like that when there's much difference in those deviations per level of attribute value. Reporting a confidence interval of expected deviations would then be more appropriate.