User perceptions of multi-source feedback tools for junior doctors
- 1 February 2010
- journal article
- Published by Wiley in Medical Education
- Vol. 44 (2), 165-176
- https://doi.org/10.1111/j.1365-2923.2009.03565.x
Abstract
The effectiveness of multi-source feedback (MSF) tools, which are increasingly important in medical careers, will be influenced by their users' attitudes. This study compared perceptions of two tools for giving MSF to UK junior doctors, of which one provides mainly textual feedback and one provides mainly numerical feedback. We then compared the perceptions of three groups, including: trainees; raters giving feedback, and supervisors delivering feedback. Postal questionnaires about the usability, usefulness and validity of a feedback system were distributed to trainees, raters and supervisors across the north of England. Questionnaire responses were analysed to compare opinions of the two tools and among the different user groups. Overall there were few differences. Attitudes towards MSF in principle were positive and the tools were felt to be usable, but there was little agreement that they could effectively identify doctors in difficulty or provide developmental feedback. The text-oriented tool was rated as more useful for giving feedback on communication and attitude, and as more useful for identifying a doctor in difficulty. Raters were more positive than other users about the usefulness of numerical feedback, but, overall, text was felt to be more useful. Some trainees expressed concern that feedback was based on insufficient knowledge of their work. This was not supported by raters' responses, although many did use indirect information. Trainees selected raters mainly for the perceived value of their feedback, but also based on personal relationships and the simple pragmatics of getting a tool completed. Despite positive attitudes to MSF, the perceived effectiveness of the tools was low. There are small but significant preferences for textual feedback, although raters may prefer numerical scales. Concerns about validity imply that greater awareness of contextual and psychological influences on feedback generation is necessary to allow the formative benefits of MSF to be optimised and to negate the risk of misuse in high-stakes contexts.Keywords
This publication has 32 references indexed in Scilit:
- How ratings vary by staff group in multi-source feedback assessment of junior doctorsMedical Education, 2009
- Specialty-specific multi-source feedback: assuring validity, informing trainingMedical Education, 2008
- 360 degree assessment (multisource feedback) of UK trainee doctors: Field testing of team assessment of behaviours (TAB)Medical Teacher, 2007
- mini-PAT (Peer Assessment Tool): A Valid Component of a National Assessment Programme in the UK?Advances in Health Sciences Education, 2006
- The Foundation Programme assessment tools: An opportunity to enhance feedback to trainees?Postgraduate Medical Journal, 2006
- A literature review of multi-source feedback systems within and without health services, leading to 10 tips for their successful designMedical Teacher, 2006
- ‘Team observation’: a six-year study of the development and use of multi-source feedback (360-degree assessment) in obstetrics and gynaecology training in the UKMedical Teacher, 2006
- Use of SPRAT for peer review of paediatricians in trainingBMJ, 2005
- Development and reliability testing of TAB a form for 360° assessment of Senior House Officers’ professional behaviour, as specified by the General Medical CouncilMedical Teacher, 2005
- Heuristics and BiasesPublished by Cambridge University Press (CUP) ,2002