How to calculate IOA (interobserver agreement), with worked examples
Interobserver agreement (IOA) is the degree to which two observers who measure the same behavior independently record the same values. You calculate it to check that your data are believable before you make decisions from them. Each formula below gives a percentage; the method you choose depends on how the behavior was measured.
Quick reference
| Data type | IOA method | Formula |
|---|---|---|
| Event / count | Total count | smaller count ÷ larger count × 100 |
| Count in intervals | Mean count-per-interval | average of (smaller ÷ larger) for each interval × 100 |
| Count in intervals | Exact count-per-interval | intervals with identical counts ÷ total intervals × 100 |
| Trials (DTT, probes) | Trial-by-trial | trials agreed ÷ total trials × 100 |
| Interval recording | Interval-by-interval | agreements ÷ (agreements + disagreements) × 100 |
| Low-rate behavior (intervals) | Scored-interval | agreements on occurrence ÷ intervals where either observer scored it × 100 |
| High-rate behavior (intervals) | Unscored-interval | agreements on non-occurrence ÷ intervals where either observer left it unscored × 100 |
| Duration | Total duration | shorter duration ÷ longer duration × 100 |
| Duration per episode | Mean duration-per-occurrence | average of (shorter ÷ longer) for each occurrence × 100 |
Worked examples
Total count IOA
Observer A counts 18 hand-raises; Observer B counts 20.
18 ÷ 20 × 100 = 90%
Total count IOA is quick but loose: both observers could reach similar totals while scoring different instances.
Mean and exact count-per-interval IOA
A session is split into four intervals.
| Interval | Observer A | Observer B | Smaller ÷ larger |
|---|---|---|---|
| 1 | 3 | 3 | 1.00 |
| 2 | 0 | 1 | 0.00 |
| 3 | 2 | 2 | 1.00 |
| 4 | 5 | 4 | 0.80 |
Mean count-per-interval = (1.00 + 0.00 + 1.00 + 0.80) ÷ 4 × 100 = 70%
Exact count-per-interval = 2 identical intervals ÷ 4 × 100 = 50%
If both observers record 0 in an interval, count that interval as full agreement (1.00).
Trial-by-trial IOA
Two observers score 10 discrete trials and agree on 9 of them.
9 ÷ 10 × 100 = 90%
Interval-by-interval, scored-interval and unscored-interval IOA
Ten 10-second partial intervals. Observer A scored the behavior in intervals 2, 5 and 7. Observer B scored it in intervals 2, 5 and 8.
Interval-by-interval: 8 agreements ÷ (8 + 2) × 100 = 80%
Scored-interval: either observer scored intervals 2, 5, 7, 8. Both scored 2 and 5. 2 ÷ 4 × 100 = 50%
Unscored-interval: at least one observer left intervals 1, 3, 4, 6, 7, 8, 9, 10 unscored. Both left 1, 3, 4, 6, 9, 10 unscored. 6 ÷ 8 × 100 = 75%
Same data, three different answers. With a low-rate behavior, interval-by-interval IOA looks high just because both observers agree on many empty intervals; scored-interval IOA is the stricter check. With a high-rate behavior, use unscored-interval IOA for the same reason.
Total duration and mean duration-per-occurrence IOA
Observer A records three tantrums of 10, 30 and 45 seconds (85 s total). Observer B records 12, 30 and 50 seconds (92 s total).
Total duration: 85 ÷ 92 × 100 = 92.4%
Mean duration-per-occurrence: (10/12 + 30/30 + 45/50) ÷ 3 × 100 = (0.833 + 1.000 + 0.900) ÷ 3 × 100 = 91.1%
How much IOA is enough?
A commonly cited guideline is that IOA should be at least 80%, with 90% or higher preferred, and that IOA should be collected in at least 20% of sessions, ideally 25–33%, spread across conditions (Cooper, Heron & Heward, 2020). Check your organization's or funder's requirements, which may be stricter.
Common mistakes
- Observers comparing notes during the session (IOA must be independent).
- Using total count IOA when the data were collected in intervals.
- Reporting interval-by-interval IOA for a behavior that occurs in only a few intervals.
- Collecting IOA only in baseline, or only on good days.
- Vague operational definitions. Low IOA is often a definition problem, not an observer problem.
Sources
- Cooper, J. O., Heron, T. E., & Heward, W. L. (2020). Applied Behavior Analysis (3rd ed.). Pearson.
This guide is general professional information. It does not replace the judgment of a qualified behavior analyst or other licensed professional, and it is not medical or legal advice. NeuroPath is not affiliated with or endorsed by the BACB®.