Interobserver reliability of physical signs elicited and diagnoses made on middle ear examination were studied. Effect on reliability of variation in observer knowledge, patient age and instrument were defined. Two board-certified, color-seeing pediatric specialists who worked in the same clinic examined consecutively 350 ears with the same otoscope for each case. Over a period of 4 mo. 4 different otoscopes were used to examine at least 50 ears with each. The following groups of findings were graded independently by the pediatrician according to pre-set criteria: color, landmarks and light reflex. Diagnosis was also recorded. At no time did the observers discuss their findings or diagnoses. Degree of interobserver agreement was expressed by the reliability coefficient (values given below) and the K coefficient [the portion of overall agreement not based on chance]. Overall agreement on findings ranged from 0.50 for landmarks to 0.85 for air-fluid level. Overall agreement for diagnoses was 0.72. Specific agreement on physical findings ranged from 0.0 for presence of air-fluid level to 0.85 for absence of air-fluid level. Specific agreement on diagnoses ranged from 0.08 on serous otitis to 0.56 on otitis media and 0.61 on normal. After the age of 1 yr, agreement did not increase with age. Agreement did not change significantly with different instruments or as the study progressed. Equivalence of observer knowledge strongly increases agreement, while patient age less than 1 yr decreases agreement, and type of instrument does not affect agreement. Clinicans should be trained to recognize middle ear signs and diagnoses according to specified criteria to maximize interobserver agreement.