PURPOSE : This study examined the concurrent validity of the Objective Structured Assessment of Technical Skill ( OSATS ), a new test of technical skill for general surgery residents . METHOD : Twelve residents ( six in their senior , or fifth , year and six in their junior , or third , year ) at the University of Toronto in 1994-95 were ranked within level of training according to their OSATS marks and by surgical faculty . Correspondence between OSATS and faculty rankings was assessed using Spearman rank-order correlation coefficients . RESULTS : The correlations between test scores and faculty rankings were generally high for the senior residents but low for the junior residents . CONCLUSION : Scores on the OSATS accurately reflect the independent opinions of faculty regarding the technical skills of senior residents , suggesting that it is a valid measure of technical skill for these individuals . The scores did not , however , reproduce faculty rankings of the junior residents . Whether this was a failing of the OSATS or the faculty rankings requires further study .