Advice on annual evaluations
It’s spring, and in most USF colleges, faculty are either working on or have just finished their reports of activities in 2007, and both peer committees and chairs are starting or anticipating the review process. Some reminders of provisions in the Collective Bargaining Agreement (PDF) and some common-sense advice:
- Student surveys are mediocre indicators of teaching quality. The eight-item survey used at USF and across the State University System was not written by USF faculty, nor was the decision to adopt them made with any meaningful input from USF faculty. As far as I am aware, there is no published evidence about the coherence or meaning of the items, and I wince every time I hear someone refer to the survey results by the term “evaluation.” Evaluation is a process involving thoughtful judgment, not the statistical summary of eight poorly-written Likert scale items. It is probably best to see the survey scores as a rough indication of satisfaction, not a substitute for the thoughtful judgment of peers or chairs. The relevant language from section 10.4(A) of the Collective Bargaining Agreement reads: The teaching evaluation must take into account any relevant materials submitted by the employee, including the results of peer evaluations of teaching, and may not be based solely on student evaluations [sic] when this additional information has been made available to the evaluator [emphasis added].
- Distance-learning courses inherently have problems related to student surveys. I have heard of at least two problems tied to the surveys of online courses. One is the low response rate of students in online courses when there is no opportunity to have a group as a captive audience and hand out forms. The second is a problem of security: Because Oasis does not consistently remove students who have dropped courses from Blackboard or other sites with secure-access links, students who have dropped a course will occasionally still have access to the student survey when it is placed online for distance-learning courses. As a consequence, especially for tenure-track faculty whose assignments are primarily in online courses, be very cautious in interpreting summary statistics for student surveys in online courses.
- Evidence of scholarship can be varied. Some departments or schools are “mixed marriages” of scholars from different disciplines. In some cases, people who work primarily on articles are down the hall from those whose recognized work are conference proceedings or where projects are primarily books. In other cases, some faculty engage in monographic research while others are engaged in creative scholarship. In all such interdisciplinary departments, responsibilities are reciprocal: In summarizing their own work, faculty are responsible for explaining the standards of their fields to colleagues and for putting the work of one year in the broader context of a scholarship agenda.* (In general,we should make sure that we have included all materials in our evaluation packets that we want the evaluators to look at, and the material should be accurate.) In return, peer committees and chairs are responsible to go to some effort to understand the standards of another colleague’s field. The relevant section of 10.4(B) reads: Evidence of research and other creative activity shall include, but not be limited to, published books; articles and papers in professional journals; musical compositions, paintings, sculpture; workings of performing art; papers presented at meetings of professional societies; and research and creative activity that has not yet resulted in publication, display, or performance.
- Evaluations are judgments, not calculations. There is no guarantee in the Collective Bargaining Agreement that a peer review committee or a chair will arrive at a result that the person being evaluated thinks should happen. The first and third points above focus on parts of the Collective Bargaining Agreement that guarantee input from the faculty member or professional employee being evaluated. But the judgment of peer committees and chairs/supervisors is independent, as long as it is fair and consistent with the Collective Bargaining Agreement.
- Procedures are in the hands of department/unit faculty. These procedures will usually dictate how people are selected for peer review committees and sometimes specify substantive expectations within a department/unit. CBA section 10.3(A)(2) puts the procedures for annual evaluations in the hands of a majority vote of (in-unit**) employees in each department or similar unit. If you came to USF recently, you probably did not vote for the procedures used, but your more senior colleagues and other former members of the department/unit did. Those procedures must be available in your department/unit offices, and they should have been given to you when you first came to USF.
- Be aware of and proactive about the human dimensions of receiving criticism. Ever hear the horror stories of layoffs announced by e-mail or people who broke up relationships via text messaging? I know of one faculty member in Florida who had a similar experience, many years ago when the faculty member was an assistant professor: the department chair sent an e-mail to the faculty, saying that the final annual evaluations were in everyone’s boxes, and everyone had to sign them indicating receipt by the end of the week. To the assistant professor’s horror, the evaluations had factual inaccuracies, gave advice on research that was inconsistent with the faculty member’s disciplinary expectations, and somehow calculated the average of all averages on the 8-item survey for each class down to the thousandths of a point… including for a course with 8 students. All of this was conveyed in the cold style of a written evaluation that appeared in the mailbox. Similar problems appeared in the evaluations of other junior faculty for the department. The assistant professors in question knew about a provision of the statewide contract that still exists in the USF Collective Bargaining Agreement: faculty and professional employees can ask for a discussion with the person who wrote the evaluation before it is finalized and placed in the person’s evaluation file (10.3(A)(1)). The evaluations were eventually rewritten, but not before considerable anguish by junior faculty and embarrassment to the chair. The lesson for chairs: if you have serious feedback in an annual evaluation, figure out how to convey it in person, because your department colleague has the right to that discussion anyway, and it is in both your colleague’s and your interest for you to be proactive. (For chairs and everyone else who finds themselves having difficult conversations, I recommend Kerry Patterson et al.’s Crucial Confrontations, published in 2005.)
- Avoid hearsay evidence. This should be common sense, but I have occasionally heard of chairs or peer committees in a few Florida universities who have relied on hearsay and other fragmentary evidence for serious negative judgments in an evaluation. The Collective Bargaining Agreement gives broad discretion in evaluation to the consideration of evidence from faculty and professional employees and also from others with a close professional relationship tied to someone’s assignment (section 10.2). However, there are other provisions of the Collective Bargaining Agreement that limit anonymous material in the evaluation file (11.5) or forbid the maintenance of material in an evaluation file when that material contains demonstrably false statements (11.7), and the careless use of hearsay evidence might be considered defamatory by a faculty member. Relying on hearsay undermines the credibility of an evaluation with the person being evaluated, especially if it appears that a peer committee or chair spent more time considering the hearsay evidence than the material that the faculty member submitted. Here is one test for the credibility of an evaluation: Does the evaluation demonstrate that a reasonable amount of time was spent reading the materials that the faculty member submitted, or could the person being evaluated reasonably conclude that no one read what he or she may have spent several days writing?
* – Tenure-track faculty can learn how to put their work into a broader context in the occasional members-only tenure workshop. The next workshops are March 3, March 4, and March 5.
** – The bargaining unit for UFF-USF includes ranked faculty outside the College of Medicine, chairs in the colleges of Education and Arts & Sciences, and several categories of professional employees. Yes, in-unit professional employees on grant-funded positions have the same evaluation rights as ranked faculty. E-mail me if you would like an example of an evaluation procedure that addresses the differences in needs between ranked faculty (especially tenure-track faculty) and professional employees.
