2022•Surgery in Practice and ScienceOpen access

The Lake Wobegon effect is real: All general surgery residents appear to be better than average

Thomas Szabo Yamashita, Vicky J.-H. Yeh, Sarah B. Lund, Mariela Rivera, Travis J. McKenzie, John M. Stulak, Stephanie F. Heller, James W. Jakub

Open full text 1 citations

Abstract

Resident evaluations remain controversial. Metric systems are employed to objectify evaluations. Our institution adopted the question "Relative to all other trainees that I have supervised over my career, I place this individual's performance in the following percentile" with the goal of objectifying the evaluations. We report the distribution and the correlation of this value to other evaluation parameters. Design: Retrospective single-institution review of general surgery resident evaluations (2008-2020). Demographic characteristics, level of training and internal evaluation variables were collected. Percentile distribution and association between subjective and objective measures were analyzed. Results: 3117 evaluations (116 residents) were reviewed. 96% had the percentile question answered. Faculty ranked 68% of residents at or above 80th percentile. Faculty subjective evaluation on residents' overall performance and technical skills correlate with resident objective assessment scores in PGY-1 and -2 but not in PGY-3 to PGY-5. Conclusion: Rank failed to produce a bell-shaped curve, remaining right-skewed; indicating grade inflation. The lack of association between variables indicates that both assessments remain useful for residency programs. Work is needed to benchmark evaluations, ensuring proper feedback about performance.

Open-access reader

About this research paper

What this paper is about

Resident evaluations remain controversial. Metric systems are employed to objectify evaluations. Our institution adopted the question "Relative to all other trainees that I have supervised over my career, I place this individual's performance in the following percentile" with the goal of objectifying the evaluations. We report the distribution and the correlation of this value to other evaluation parameters. Design: Retrospective single-institution review of general surgery resident evaluations (2008-2020). Demographic characteristics, level of training and internal evaluation variables were collected. Percentile distribution and association between subjective and objective measures were analyzed. Results: 3117 evaluations (116 residents) were reviewed. 96% had the percentile question answered. Faculty ranked 68% of residents at or above 80th percentile. Faculty subjective evaluation on residents' overall performance and technical skills correlate with resident objective assessment scores in PGY-1 and -2 but not in PGY-3 to PGY-5. Conclusion: Rank failed to produce a bell-shaped curve, remaining right-skewed; indicating grade inflation. The lack of association between variables indicates that both assessments remain useful for residency programs. Work is needed to benchmark evaluations, ensuring proper feedback about performance.

Why it matters

OpenAlex reports 1 citations for this work. Citation counts describe recorded attention and do not establish research quality.

Key contribution

A contribution statement is not available in the OpenAlex record.

Method / approach

Method details are not available in the OpenAlex metadata.

Main findings

Findings are not separately available in the OpenAlex metadata.

Limitations

Limitations are not available in the OpenAlex metadata.

Applications

Application details are not available in the OpenAlex metadata.

Available abstract

Resident evaluations remain controversial. Metric systems are employed to objectify evaluations. Our institution adopted the question "Relative to all other trainees that I have supervised over my career, I place this individual's performance in the following percentile" with the goal of objectifying the evaluations. We report the distribution and the correlation of this value to other evaluation parameters. Design: Retrospective single-institution review of general surgery resident evaluations (2008-2020). Demographic characteristics, level of training and internal evaluation variables were collected. Percentile distribution and association between subjective and objective measures were analyzed. Results: 3117 evaluations (116 residents) were reviewed. 96% had the percentile question answered. Faculty ranked 68% of residents at or above 80th percentile. Faculty subjective evaluation on residents' overall performance and technical skills correlate with resident objective assessment scores in PGY-1 and -2 but not in PGY-3 to PGY-5. Conclusion: Rank failed to produce a bell-shaped curve, remaining right-skewed; indicating grade inflation. The lack of association between variables indicates that both assessments remain useful for residency programs. Work is needed to benchmark evaluations, ensuring proper feedback about performance.

Key concepts: Medicine, Environmental science, Psychology

Related papers

Back to paper searchBrowse research topicsOriginal source
The Lake Wobegon effect is real: All general surgery residents appear to be better than average — Research Paper | ScholarLens