Calibration and Measurement Uncertainty: Controlling Instruments

What calibration and measurement uncertainty are

Calibration compares an instrument against a traceable standard under defined conditions and reports the indication error. Measurement uncertainty is the interval around a result in which the true value is expected to lie with a stated probability (usually about 95%, with coverage factor k = 2).

Calibrating is not adjusting, and it does not guarantee the instrument is good. The certificate shows error and uncertainty; your company decides whether the instrument fits the job by comparing that result with the product tolerance.

Visual map: traceability chain and calibration interval

The map shows the traceability chain up to the international standard, the uncertainty budget of the example and the calibration-interval cycle.

Traceability and the concepts people mix up

Metrological traceability is an unbroken chain of calibrations, each with its own uncertainty: the shop-floor gauge is calibrated against a working standard, that against a reference standard in an accredited laboratory, and that against national standards linked to the International System of Units (SI). Accredited laboratories work to ISO/IEC 17025, and accreditation bodies that sign the ILAC arrangement recognize each other.

• Calibration: determines error and uncertainty. It does not change the instrument.
• Adjustment: an intervention to reduce error. It requires a new calibration afterward.
• Verification: an internal check, against a standard, that the error is still within the limit. Between calibrations it is the cheapest control.
• MSA / Gauge R&R: evaluates the whole measurement system (operator, method, part), not just the instrument. See MSA and Gauge R&R.

ISO 9001 (clause 7.1.5) requires monitoring and measuring resources to be suitable and maintained and, where traceability is a requirement, calibrated or verified at specified intervals against traceable standards, with documented information retained. See ISO 9001; in automotive, IATF 16949 adds requirements on laboratory scope.

How uncertainty is calculated

The GUM method combines the sources of uncertainty, each converted to a standard deviation (standard uncertainty u):

• Type A: comes from repeated data. u_A = s ÷ √n.
• Type B: comes from other information, such as the standard's certificate, resolution and temperature. For a rectangular distribution, u = a ÷ √3, where a is the half-width. For a certificate with U and k, u = U ÷ k.

u_c = √(u₁² + u₂² + u₃² + …)  |  U = k × u_c (k = 2 for about 95%)

Worked example: 0–25 mm micrometer

Fictional example. A micrometer with 0.001 mm graduation measures a 10.000 mm gauge block (certificate: U = 0.12 µm, k = 2). Five readings: 10.002, 10.001, 10.002, 10.003, 10.002 mm. Mean = 10.0020 mm, so the indication error = +2.0 µm. Sample standard deviation s = 0.707 µm.

Result: error of +2.0 µm with U = 0.9 µm (k = 2). Note: a full budget would also include temperature and measuring-force deformation; the example leaves them out for simplicity.

Decision: does the instrument pass?

Assume the company sets a maximum permissible error of ±4 µm for this micrometer (an internal criterion derived from the part tolerance). A simple, conservative rule that puts uncertainty in the calculation:

Accept if |error| + U ≤ maximum permissible error  →  2.0 + 0.9 = 2.9 µm ≤ 4 µm

The instrument conforms. The ratio of permissible error to uncertainty is 4 ÷ 0.865 ≈ 4.6, above the 4:1 rule of thumb widely used in industry. If |error| + U exceeded 4 µm, the right move is to adjust, derate, restrict its use or withdraw it, and assess earlier measurements as a nonconformity.

Calibration interval and instrument control

No standard sets one fixed interval: you define it from use, environment, stability and criticality (see ILAC-G24 / OIML D10). The best data is history: the drift between calibrations. In the example, assume errors of +1.2, +1.6 and +2.0 µm over the last three years: drift of 0.4 µm/year. The usable limit is 4 − 0.9 = 3.1 µm; 1.1 µm remain, about 2.7 years. Even so, the 12-month interval can stay until more data builds up.

1. Register each instrument with a unique ID, range, resolution and owner.
2. Set the acceptance criterion from the product tolerance, not from what the manufacturer states.
3. Choose a laboratory whose accreditation scope matches the quantity and range.
4. Label with the due date and protect against unauthorized adjustment.
5. Run intermediate checks with a bench standard.
6. Review the history and the interval; handle every failure with corrective action.
7. Include the topic in the internal audit and the control plan.

Common mistakes and limitations

• Confusing error with uncertainty: error has a sign and can be corrected; uncertainty is the doubt that remains.
• Accepting a certificate unread: check calibrated points, range, uncertainty and the acceptance criterion.
• Calibrating at a single point: error varies across the range. Calibrate where the instrument is used.
• Ignoring the environment: temperature, vibration and dirt change the measurement.
• One fixed interval for everything: without history, the interval is a guess.
• Thinking calibration replaces MSA: a calibrated gauge can sit in a system with high operator or method variation.

Frequently asked questions

What is the difference between calibration and verification?

Calibration determines error and uncertainty against a traceable standard, usually by a laboratory. Verification checks that the error is within the limit, often in-house.

What is measurement uncertainty?

It is the doubt left in a result. It is expressed as U, for example ±0.9 µm with k = 2, which covers about 95% of reasonable values.

What is metrological traceability?

An unbroken chain of calibrations linking your instrument to national standards and the SI, each link with its own uncertainty.

How long should the calibration interval be?

No standard sets a fixed period. Define it from use, stability and criticality, using drift history; 12 months is a common starting point.

Is a calibrated instrument an approved instrument?

No. Calibration only reports error and uncertainty. Approval comes from comparing it with the maximum permissible error defined for the use.

How do you calculate expanded uncertainty?

Combine the standard uncertainties as a root sum of squares and multiply by k. In the example: 2 × 0.432 = 0.865 µm.

About the author

Vagner Soares

Lean Manufacturing & Behavioral Management Specialist

Over 20 years in the automotive and metalworking industries (GM and Dana), Lean Manufacturing practitioner since 2006. SENAI instructor and mentor in Brazil’s Brasil Mais Produtivo program, delivering consulting, training and audits for 50+ companies, combining quality, productivity and people development.

LinkedInInstagramFull profile →