NeuroPath Toolkit

Toolkit Guides

How to calculate IOA (interobserver agreement), with worked examples

Interobserver agreement (IOA) is the degree to which two observers who measure the same behavior independently record the same values. You calculate it to check that your data are believable before you make decisions from them. Each formula below gives a percentage; the method you choose depends on how the behavior was measured.

Quick reference

Data typeIOA methodFormula
Event / countTotal countsmaller count ÷ larger count × 100
Count in intervalsMean count-per-intervalaverage of (smaller ÷ larger) for each interval × 100
Count in intervalsExact count-per-intervalintervals with identical counts ÷ total intervals × 100
Trials (DTT, probes)Trial-by-trialtrials agreed ÷ total trials × 100
Interval recordingInterval-by-intervalagreements ÷ (agreements + disagreements) × 100
Low-rate behavior (intervals)Scored-intervalagreements on occurrence ÷ intervals where either observer scored it × 100
High-rate behavior (intervals)Unscored-intervalagreements on non-occurrence ÷ intervals where either observer left it unscored × 100
DurationTotal durationshorter duration ÷ longer duration × 100
Duration per episodeMean duration-per-occurrenceaverage of (shorter ÷ longer) for each occurrence × 100

Worked examples

Total count IOA

Observer A counts 18 hand-raises; Observer B counts 20.

18 ÷ 20 × 100 = 90%

Total count IOA is quick but loose: both observers could reach similar totals while scoring different instances.

Mean and exact count-per-interval IOA

A session is split into four intervals.

IntervalObserver AObserver BSmaller ÷ larger
1331.00
2010.00
3221.00
4540.80

Mean count-per-interval = (1.00 + 0.00 + 1.00 + 0.80) ÷ 4 × 100 = 70%

Exact count-per-interval = 2 identical intervals ÷ 4 × 100 = 50%

If both observers record 0 in an interval, count that interval as full agreement (1.00).

Trial-by-trial IOA

Two observers score 10 discrete trials and agree on 9 of them.

9 ÷ 10 × 100 = 90%

Interval-by-interval, scored-interval and unscored-interval IOA

Ten 10-second partial intervals. Observer A scored the behavior in intervals 2, 5 and 7. Observer B scored it in intervals 2, 5 and 8.

Interval-by-interval: 8 agreements ÷ (8 + 2) × 100 = 80%

Scored-interval: either observer scored intervals 2, 5, 7, 8. Both scored 2 and 5. 2 ÷ 4 × 100 = 50%

Unscored-interval: at least one observer left intervals 1, 3, 4, 6, 7, 8, 9, 10 unscored. Both left 1, 3, 4, 6, 9, 10 unscored. 6 ÷ 8 × 100 = 75%

Same data, three different answers. With a low-rate behavior, interval-by-interval IOA looks high just because both observers agree on many empty intervals; scored-interval IOA is the stricter check. With a high-rate behavior, use unscored-interval IOA for the same reason.

Total duration and mean duration-per-occurrence IOA

Observer A records three tantrums of 10, 30 and 45 seconds (85 s total). Observer B records 12, 30 and 50 seconds (92 s total).

Total duration: 85 ÷ 92 × 100 = 92.4%

Mean duration-per-occurrence: (10/12 + 30/30 + 45/50) ÷ 3 × 100 = (0.833 + 1.000 + 0.900) ÷ 3 × 100 = 91.1%

How much IOA is enough?

A commonly cited guideline is that IOA should be at least 80%, with 90% or higher preferred, and that IOA should be collected in at least 20% of sessions, ideally 25–33%, spread across conditions (Cooper, Heron & Heward, 2020). Check your organization's or funder's requirements, which may be stricter.

Common mistakes

Sources

This guide is general professional information. It does not replace the judgment of a qualified behavior analyst or other licensed professional, and it is not medical or legal advice. NeuroPath is not affiliated with or endorsed by the BACB®.

More free guides