How Professors Get the Grade

Getting Familiar With Faculty Evaluations

default article imageStudents always know when it’s about to happen. With less than two weeks left of classes, the professor wraps up one class a few minutes early, reaching for a manila envelope on their desk. A brief speech about something called the SIRs is brought to your attention as a questionnaire is distributed to you and your peers. The professor quietly leaves, giving you a few minutes to look it over and an­swer the questions laid out before you.

For once, you are the grader scoring your professor. Think hard about the last 16 weeks, because the “grade” you give your instructor and course overall turns out to be more significant than you thought.

According to Provost Thomas Pear­son, there was no formal student eval­uation system when he came to then- Monmouth College in 1978. However, there were ways students could leave feedback, such as a book of comments that he believes was placed in the Reg­istrar’s Office. “Students could come by and see what other students thought about faculty, courses, things like that. But it was very random, rather infor­mal and the comments to the best of my knowledge were not used in any formal way by faculty,” Pearson says.

Pearson says it wasn’t until the mid- 1980’s that a formal student evalua­tion system was introduced as part of negotiation for the Faculty Asso­ciation. The University saw this as important to evaluate faculty in order to make decisions in terms of tenure, candidacy, continuance and promo­tion. “Having a more formalized pro­cess was felt by the administration to be at the advantage of everybody.”

The evaluation system’s instru­ment agreed to use were known as Educational Testing Service’s (ETS) Student Instructional Reports, or as most call them, SIRs. It is a nation­ally normed instrument; Pearson has heard that using one “consistently over time provides the best results in terms of evaluating faculty perfor­mance and curricular development.”

The questionnaire handed to stu­dents is designed by ETS, so the Uni­versity has no control on what content or questions appear on it. The Provost’s Office explained that the University has a contract with ETS; the annual costs for the SIRs are $9,500 for pro­cessing and $4,500 for materials.

Pearson says the reports are in­tended to serve two purposes. “One, to serve the faculty in terms of giving them the sense of student feedback on the course on the various aspects from the organization and planning, communication, the interaction be­tween faculty and students, assign­ments, course outcomes, workloads and overall performance. It’s meant to be helpful to faculty in order to review it in order to think of ways to improve their instruction.” The second purpose it serves, Pearson continues, directly feeds into the evaluation processes for continuance, tenure and promotion.

Susan O’Keefe, Associate Vice President for Academic Adminis­tration, explains the administration process of SIRs. “We know which faculty are expected to complete the SIRs in a given year. We notify them, and they are asked to select which of their classes they will have the SIR administered in,” she explains, add­ing that one restriction is a class must have at least five or six students in it.

Pearson adds that tenured profes­sors distribute SIRs to half of their classes every other year, as required under the faculty contract. For the fac­ulty who are on probationary status, if they teach eight courses a year, they have to distribute SIRs to six of them.

Materials are ordered and received in the Provost’s Office, and when the faculty notifies them which classes they would like to be evaluated in, the materials are then distributed to the dean’s offices in each of the schools, and then made into packets for each faculty member who is participating. Instructions are included for each pro­fessor, such as picking a student assis­tant to return the materials to the Reg­istrar’s Office, leaving the room when the SIRS are being filled out, etc.

Once they are returned to the Regis­trar’s Office, they are repackaged and mailed to ETS where they are analyzed and then sent back to the University and distributed for the faculty to review.

One of the reasons to stay with the SIR and not use a different evaluation system is because it’s a nationally stan­dardized instrument, Pearson explains. “It really helps us not only see how the faculty member does over a period of time in various courses, but it’s indexed against the national average for faculty teaching those courses. So, we actually have yardsticks we can use and conse­quently we can make a determination if the faculty is consistently in the bot­tom 10 percent nationally…that’s not a good place to be as opposed to be more towards the center or above the center.”

As for the faculty who dislike the SIR, Pearson says, “They don’t see it as especially helpful in terms of develop­ing their teaching. They see the catego­ries as rather fixed, unless the faculty have written questions that students fill out- supplemental questions of the form. Sometimes they feel the instru­ment is not especially valuable. I think others of us have the perspective that it is valuable and that one needs really to know how to interpret the results.”

Senior Ajda Dotday, a senior fi­nance major, likes the idea of faculty providing supplemental questions. “If they handed out more question­naires that asked what students would change about the class, then profes­sors could understand what students are looking for,” Dotday says.

When asking faculty to comment on the SIRs, some chose not to comment.

Harvey Allen, assistant professor of education, feels his students understand the importance of the SIR. “Since I teach only graduate students who are training to be principals or supervisors, I think they take the evaluation pro­cess very seriously. Perhaps putting the whole process online might be a better way to collect the data,” Allen says.

Once the SIRs get returned, faculty members have the option of getting them interpreted from the Center for Excellence in Teaching and Learn­ing (CETL). “The director of that center is particularly good in terms of working with faculty who request to sit down and have their SIRs ex­plained. In addition, each department chair is supposed to meet with their faculty after the reports have come in and talk about how the faculty mem­ber has done and how improvements might be made,” Pearson says.

In the last 10 years, the SIR was transformed to the SIR II which pro­vided more information. However, Pearson says these reports are not the only methods the University uses to evaluate faculty. Besides SIRs, there are observations that faculty do of their colleagues and that deans do of faculty; when faculty come up for tenure, department chairs phone students to get their opinion on the faculty member and there are also alumni surveys of faculty.

Several students complain about filling out the reports each semes­ter; many of them feel they do not see changes made after academi­cally rating their professors. How­ever, Pearson says, “I think that is a misperception because they are heavily used in terms of evaluat­ing teaching. Because of the nature of the information and the fact that they’re collected over a period of time, they’re often given consider­able weight among the various fac­tors in determining teaching. There are faculty who have said that SIRs appear to be the only instrument that are used for evaluating teach­ing, which is not true. But I would say just because of the nature that they tend to be pretty significant weight, and so I think faculty would say they hope that students fill them out conscientiously, and try to dis­sociate their performance in the course with their evaluation of the instructor and the course.”

George Mena, a senior psychol­ogy major, believes the SIR is an appropriate method. “I think a teacher is more than just a person who is hired to provide a lecture, and there is a lot of planning that should be put into course meetings. SIRs give us, the students, a chance to criticize the extent to which the professor does this,” Mena says.

There have been some rare cases, Pearson explains, where the packets have not been delivered or certain information has not come back from ETS. “But when faculty apply for continuance and so forth, they write self-evaluation statements where they are asked to really re­flect on the results of the SIRs and if there are missing SIRs to explain what happened in that semester, and if they did not perform particu­larly well they have the opportunity to explain what they think are the factors,” Pearson says.

If a faculty member is repeatedly receiving low scores on the reports, Pearson says the University will look into the situation. “If it’s a pat­tern and a significant problem, even with a tenured faculty, that would be the basis for bringing together a com­mittee to review the faculty’s perfor­mance in terms of continuation.”

Greg Cenicola, a junior criminal justice major, believes that the evalua­tions are good, but biased. “Most kids don’t care and don’t take them seri­ously,” he says, “so kids just fill them out with one number; or you get the kid who’s not doing good in class because they never show up or do anything and give the teacher ‘1's.’ So I think as an idea they’re awesome, but how practi­cal are they is the question.”

Other students have heard that if they fill out an entire SIR with all one number, (‘5’s’ being the highest score, ‘1’s’ being the lowest) that their scores are automatically not counted. Pearson says he does not know how ETS scores each report, but both he and O’Keefe do not believe that is the case. “The worst thing that students can do in the evaluation process is not take them seriously or put things down that are not correct,” he says.

Pearson talks of a common misper­ception of faculty and even of students-that only faculty who are popular, easy and give high grades get good SIRS. “I’ve been looking at them for over 20 years as Provost and, in fact, that’s not true. It may happen in some cases, but in many cases I found that faculty who are really good, who are demanding but are also accessible and supportive of students do very well on SIRs.”

Something the University is consid­ering is adding an evaluation instru­ment for online classes. “That issue is being negotiated now. That’s a de­ficiency we need to fix because more and more faculty are teaching online. We’ll definitely want to use a national online instrument, what that will be re­mains unseen,” Pearson said.

Other ideas include evaluations for adjunct professors, as well as summer evaluations. Pearson says in the past, evaluating adjuncts with SIRs have historically been a cost issue, but with more adjuncts teaching courses at the University, “Having the SIRs used by all of the elements of the faculty would give us a better read on performance, as opposed to a department using a separate instrument for adjuncts, and something else for online, as so forth, so we’ll definitely be looking into that.”