// GLOSSARY · METHODOLOGY
The statistical measure of agreement between independent evaluators scoring the same performance against the same rubric.
Inter-Rater Reliability — The statistical measure of agreement between independent evaluators scoring the same performance against the same rubric.
Inter-rater reliability (IRR) is the formal measure of whether a scoring system is consistent across people. High IRR means the rubric and the calibration practice are producing the same score for the same performance, regardless of who is observing.
OCTAAR treats IRR as an operational metric, not a research artifact. It is monitored across the cycle, surfaced as a leading indicator, and used to trigger evaluator recalibration before drift becomes a finding.
// ALSO KNOWN AS
IRR, Inter-observer reliability, Inter-rater agreement
// DOMAIN
Cross-domain
// RELATED TERMS
Evaluator Calibration
The continuous practice of measuring and correcting variance between evaluators so a given score means the same thing across observers, units, and cycles.
Calibrated Rubric
A scoring rubric whose scale itself has been calibrated — not just the wording — so each score maps to a defined, defensible performance state.
// LAST UPDATED
· OCTAAR Methodology Team
// SEE IT IN OPERATION