Overview: Paper Evaluation Process
The procedures departments follow in administering questionnaires, conducting interviews, or facilitating focus groups can affect the amount and quality of the data collected. This document focuses on suggestions for administering written evaluations, whether on paper or electronically. Departments are encouraged to consult with E&A if they choose to conduct interviews or focus groups in place of written evaluations.
The purpose of these suggestions is to ensure that students have the information they need to provide informed evaluations of the candidate’s teaching, and to maximize the number of valid questionnaires that are completed and returned.
Cover letters. Whether the evaluation data are being collected on paper or electronically, it is essential that the evaluation forms be accompanied by a cover letter (paper evaluations), or preceded by an email (Form Creator questionnaires), from the department chair. Sample cover letters for the initial distribution of both paper and electronic questionnaires, as well as reminder letters, are available on the E&A website.
Paper evaluation forms: Initial and follow-up mailings
- Distribute questionnaires at least 6 weeks prior to the due date for the chair’s recommendation, if the questionnaire is being distributed only to current students. Distribute them at least 8 weeks prior to that time if the sample of respondents includes alumni. This will permit at least two follow-up requests and increase the likelihood of an adequate return rate. In most comprehensive reappointment reviews, student evaluation questionnaires are distributed early in the spring semester.
- Make sure the form includes information about when and where to return the evaluation. Although return envelopes should be included with every mailing (see below), sometimes respondents misplace return envelopes. Including return information on the form itself increases the likelihood that it will be completed and returned. If the form is two-sided, be sure it says “Continue on reverse,” or language to that effect, at the bottom of the first page.
- Put tracking numbers on each form before mailing. If E&A provides the sample to the department, this information will be stored in a MS Excel file where students have already been given tracking numbers. E&A recommends that each student's tracking number be written in ink on one corner of the questionnaire mailed to that student. This will allow the department to avoid sending follow-up requests to students who have already returned their questionnaires. E&A further recommends that tracking numbers be detached from the returned surveys (but retained separately for purposes of cross-checking), to ensure that respondents' identities cannot be linked to any responses.
- Include a pre-addressed (and, if necessary, stamped or postage-paid) return envelope. All mailings, whether initial or follow-up, should include return envelopes (size 9, so they fit into regular-sized 10 outer envelopes) addressed to the department chair to increase the likelihood of participation. Return envelopes for alumni, if included in the sample, should include postage as well.
- Aim for a return rate of at least 60%. A response rate of 60% or better minimizes the likelihood of significant differences between students who choose to participate and students who do not. This increases the representativeness of the resulting data. It may take three or more mailings to approach the 60% benchmark, especially if the sample includes alumni.
- Track responses to target follow-up mailings. Use the tracking numbers to limit second and third requests to respondents who have not returned their questionnaires.
- Allow about one week between on-campus mailings and about ten days between off-campus mailings. Generally, students who complete the questionnaire will do so in the first few days after receiving the request, so waiting longer than 7-10 days will not yield more returns.
- Prepare follow-up requests the same way initial requests are prepared. Every follow-up request for completion of the questionnaire should include another cover letter, another copy of the questionnaire with the student's tracking number on it, and a return envelope addressed to the department chair.
