Innovation report · Academic Medicine · 2021
Doing More With Written Feedback: Improving Learner Satisfaction and Reflection With the LEAF (Learner-Engaged Analysis of Feedback) Method
Saddawi‑Konefka D, Sullivan A, Beltran C, Baker K. doi:10.1097/ACM.0000000000004046
The question
Written feedback for doctors in training tends to be nonspecific, overly positive, and aimed at personal traits more than behavior. Many people who write it avoid naming areas for improvement, a reluctance known as the “mum effect.” Organized efforts to improve what faculty write have produced only modest effects. Saddawi‑Konefka and colleagues set out to help learners get more out of the written feedback they already receive.
The LEAF (Learner-Engaged Analysis of Feedback) method pairs a learner with a coach. Before any sessions, the written comments for a whole group of learners are sorted once into themes (such as clinical judgment or technical skill) to build a profile of what a typical learner receives. In each session, the coach and learner sort the learner’s own comments into the same themes and compare the counts. Because criticism often goes unwritten, a theme that appears much less often than usual may point to an area for improvement, and one that appears more often may point to a strength. The learner then reflects and sets goals. The method was introduced in 2017 for anesthesiology residents at Massachusetts General Hospital, during their twice-yearly meetings with the program director, who served as coach for every session.
What it found
In the program’s anonymized 2018 survey, 65 of 74 residents responded (88%). Of these, 54 had had a LEAF session and 11 had not. Satisfaction was rated from 1 (“not at all satisfied”) to 5 (“extremely satisfied”).
- Residents who had a LEAF session rated their satisfaction with written faculty feedback higher (mean 3.1 versus 2.5).
- They also rated feedback from program director meetings higher (mean 3.8 versus 2.8).
- Satisfaction with two kinds of feedback unrelated to LEAF, verbal feedback from faculty and feedback from faculty mentors, did not differ significantly between the groups (mean 3.1 versus 2.7, and 3.4 versus 3.4). The authors included these as a check against residents simply rating everything higher.
- Forty-one residents wrote comments about the method. The positive comments described getting a “big picture” of their themes (20), setting goals (8), uncovering blind spots (4), and interpreting comments (3). The negative or neutral comments questioned the validity of the process (7) or noted that it took substantial time (4).
Why it matters
LEAF works with the written feedback a program already collects and does not require faculty to change what they write. After the coding index is developed, the authors report that the ongoing cost at their institution is approximately 1 hour per learner per year for LEAF sessions. They describe it as a possible supplement to other efforts to improve feedback. More about LEAF is at leafmethod.org.
This was one program’s experience, without random assignment, and it measured satisfaction rather than changes in behavior.
How to cite it
Saddawi-Konefka D, Sullivan A, Beltran C, Baker K. Doing more with written feedback: improving learner satisfaction and reflection with the LEAF (Learner-Engaged Analysis of Feedback) method. Acad Med. 2021;96(10):1425-1430. doi:10.1097/ACM.0000000000004046
More on the publications page. The complete record is on PubMed and ORCID.