Friday 3 June 2011

Student evaluations

Like many universities, Lancaster has systematic student evaluation: at the end of a module, our students are invited to fill in an online form mixing Likert-type questions (1 = strongly disagree to 5 = strongly agree) with free comment boxes. Any good teacher wants to know how well her teaching is going, but here are three reasons to doubt that forms like this are going to tell her.

1. Irrelevant correlations: many studies suggest that evaluation scores don’t reliably track teaching quality. For example, high scores are correlated with teachers who: (a) match gender stereotypes in dress, mien, and social behaviour [1]; (b) mark generously, or more generously than students expect [2]; and (c) present in an enthusiastic style, regardless of content [3].

2. Anonymity breeds contempt: as anyone who’s read youtube comments knows, anonymity encourages some people to speak expressively and without normal filters: to say things they’d never say to your face, haven’t really thought about, and couldn’t defend. There’s good reason to make anonymous channels of communication available to our students, but not owning their words in evaluations may not promote honesty or usefulness, let alone civility [4, 5].

3. Competent judges? Evaluations may tell us whether our students like us and our teaching or not. But if we want to know whether we’re good teachers of our subjects, why think our students are competent to judge? They typically have no teaching experience. They don’t know what they don’t know, or what they need to know, or how to gain that knowledge. The best learning is often unsettling, inconclusive, and not what we expected. Perhaps someone in the middle of that difficult process of development isn’t in the best position to judge how well it’s going.

None of these are reasons to stop listening and responding to our students. But they are reasons to wonder whether the evaluation tools we use actually tell us what we want to know.

References:

  1. Kierstead, D., D’Agostino, P., & Dill, H., ‘Sex Role Stereotyping of College Professors: Bias in Students’ Ratings of Instructors’, Journal of Educational Psychology 80(1988): 342-4.
  2. Greenwald, A. & Gillmore, G., ‘Grading Leniency is a Removable Contaminant of Student Ratings’, American Psychologist 11(1997): 1209-17.
  3. Naftulin, D., Ware, J., & Donnelly, F., ‘The Doctor Fox Lecture: A Paradigm of Educational Seduction’, Journal of Medical Education 48(1973): 630-5.
  4. Penny Arcade
  5. xkcd

Further discussion:

(This post is a version of a piece I wrote for Lancaster University's independent email newsletter Subtext).

No comments:

Post a Comment