Institutional Research & Planning 2003-04 Annual Report

Susan Canon, Director
May 2004

The Office of Institutional Research and Planning (IRP) was established in 1998 to provide relevant and accurate information about St. Olaf College in order to monitor strategic characteristics of the College, to provide public web-based information about the College, and to aid senior administrators and program managers in assessing progress and making decisions to meet institutional goals. Beginning in June 2003 the IRP office has a staff of 2.17 FTE - a full time director, a full time research associate, and a consulting statistician from the Mathematics Department. IRP reports to the Assistant Provost.
The work of the office is focused on five major kinds of activities:

  1. Overseeing a system of regular surveys of students, faculty, staff, and alumni.
  2. Preparing and validating college data for submission to external agencies.
  3. Providing a variety of reports and advice for internal use by committees and managers.
  4. Helping guide the college data procedures to ensure consistency and utility.
  5. Maintaining an informational website for communicating with various constituencies.

This annual report highlights IRP activities in each of the five major areas, followed (in section F) by a summary of internal activities required to maintain the office itself. A final section (G) outlines potential new projects.

A. Surveys

A chief activity of IRP is to conduct surveys - of students, faculty, staff, parents, and alumni - and to communicate insights from these surveys to various college constituencies. These surveys are conducted on a regular basis – annually, biennially or less frequently as appropriate. Most surveys (e.g., the HERI (Higher Education Research Institute) College Student Survey and the HEDS (Higher Education Data Sharing consortium) Senior Survey) are externally prepared and compiled, while a few (e.g., Administration/Staff Survey) are locally developed. Externally prepared surveys, while generic in character, offer the advantage of comparative benchmarks that, when properly used, can be of help in improving the College's educational program.

Most surveys stretch over two years - one for administration, the next for interpretation. Thus this section includes both surveys conducted in 2002-03 whose analysis and interpretation took place in 2003-04 as well as surveys conducted during 2003-04. Results from some of these surveys will not be available until next year.

  • Administration/Staff Survey. This survey was designed by former IRP Director Jim Tallon and first administered in Spring 2001. The survey was conducted in Fall 2003 with very minor revisions. The overall response rate was 44%. An IRP Note summarizing the results was distributed in February and the detailed frequencies compared to the 2001 results were posted on the IRP website. In April the IRP staff gave a Noon-time Seminar presentation examining the results by various sub-groups (gender, division of employment etc.) and briefly summarizing the written comments. This presentation was also posted on the IRP website. The full report on comments was sent to Cabinet in April.
  • HERI College Student Survey. St. Olaf administered this national survey to sophomores and juniors this past fall. Although the response rate was fairly low (35%) a summary report and complete data tables have been posted to the IRP website. An IRP Note highlighting some results was sent to the community at the end of May.
    · National Survey of Student Engagement (NSSE). St. Olaf participated in the Spring 2003 NSSE and received the results of that survey in November 2003. The IRP Director worked closely with the Director of ARP to prepare a summary of that data, which was sent to the community in November. That summary is posted on the IRP website. The IRP staff has also completed a summary of the pilot questions regarding technology use and distributed it to appropriate faculty and staff.
  • Recent Graduates. The IRP office conducts an annual survey of the previous year's graduates for the Minnesota Private College Council (MPCC) and the St. Olaf Center for Experiential Learning (CEL). This survey collects information about the primary activity of our graduates (graduate school/employment/volunteer work). The IRP staff prepared final reports on the Class of 2002 for MPC and CEL including a final report on CEL usage from the class of 2002 (this portion of the survey is done biennially). We also added the results from the class of 2002 to the web summary of the primary activities for the classes of 1997 to 2001. The staff is currently compiling and preparing the data on the class of 2003, which had a response rate of 58%.
  • HEDS Senior Survey. St. Olaf participates in this national survey every other spring semester. This year the IRP Director worked closely with IIT staff to prepare the survey in web format. We also participated in the newly organized GradFest event. The combination of these two factors gave us our best response rate ever – over 71%. The staff is in the process of preparing the data to send to HEDS.
  • HEDS Alumni Survey. This spring IRP administered a “ten-years out” follow-up alumni survey to the class of 1993. We had a 36% response rate. The five-years out survey had been done in 1998 but due to scheduling conflicts we were unable to conduct the survey in 2003.

In addition to our regularly scheduled surveys the IRP staff assisted other survey efforts on campus, both with survey design/implementation assistance as well as data summaries, as follows:

  • Community Time Survey of faculty, staff and students
  • Oberlin Group Survey of Librarian Faculty Status
  • Library Coffee House Feasibility Survey of students
  • IIT Faculty Technology Use Survey

B. External Reports

Most of the reports listed below are completed every year.

  • College Guides. Completed surveys for U.S. News and World Report, Peterson's, College Board, Princeton Review, and many other college guide surveys.
  • Common Data Set. Completed the 2003-04 Common Data Set (CDS) with appendices, posted them on the web, and updated the CDS web summaries. We also submitted our CDS electronically to the newly organized Common Data Set Exchange (CDSX) for CDS data sharing among colleges and universities.
  • ELCA. Completed and submitted reports to the ELCA on admissions, financial aid, enrollment and retention.
  • HEDS. Completed various data reports for the Higher Education Data Sharing (HEDS) collaborative: Admissions, EADA Athletics (NCAA), Endowment (NACUBO), Graduation & Retention, Strategic Indicators, Faculty Compensation, Faculty Demographics, Voluntary Support of Education (CAE), FTE & Student Faculty Ratios, Applications, Deposits, Tuition & Fees, Finance.
  • IPEDS. Submitted surveys on Institutional Characteristics and Completions for the federal government's Integrated Postsecondary Education Data System (IPEDS) in Fall 2003. Winter data collection included Employees by Assigned Positions (EAP), Fall Staff, Faculty Salaries and Fall Enrollment. Spring 2004 surveys completed were Student Financial Aid, Finance, and Graduation Rate data.
  • Salary Reports. Compiled and submitted the AAUP, IPEDS, CUPA and ACM reports on faculty salary and benefits.
  • Staff & Employee Reports. Prepared data, in conjunction with the HR staff, for the IPEDS Fall Staff and EAP reports and the ELCA Faculty, Staff & Administration report. Also supplied HEDS with supplemental demographic information on the AAUP faculty salary report. Summarized three years of EAP reports for the IRP webstie.

Various other reports are submitted to ACM, ELCA, department accreditation agencies, auditors and other organizations that are either reviewed and supplemented by the IRP office and then sent, or are simply copied to the IRP office.

In addition to the above, St. Olaf College participated this year in the National Study of Faculty Students (NSoFaS:04) conducted by the US Department of Education's National Center for Education Statistics (NCES). This study comprised of two parts - the National Postsecondary Student Aid Study (NPSAS:04) and the National Study of Postsecondary Faculty (NSOPF:04). While the surveys themselves were conducted by RTI International on behalf of NCES, the St. Olaf IRP Office provided extensive data on our students and faculty for the background analyses as well as contact information for RTI.

C. Internal Report

  • Peer Group Analysis and Recommendation. Upon request of President Thomforde and the new St. Olaf Strategic Plan, the IRP Director and Consulting Statistician to IRP worked extensively in the summer and early fall to prepare and analyze data from other liberal-arts institutions in order to review the list of St. Olaf peer institutions. A recommended list of 20 institutions was prepared and presented to Dean’s Council, Cabinet and the Board of Regents.
  • Draft Strategic or Key Indicators. In conjunction with the above project, the IRP Director demonstrated the use of specific key indicators to use as benchmarks with the proposed peer institutions as a means to assess progress, as requested by the Strategic Plan.
  • Financial/Faculty Salary Reporting. Worked with Special Assistant to the Provost and former IRP Director Lynn Steen to detail faculty compensation by rank and employment status (FT/PT) and summarize in relation to the overall expenses of the college. Data from FY99-FY03 were posted on the IRP website and Steen wrote an IRP Note in June 2003. Data for FY04 is nearly completed to update the website. Prepared annual faculty salary summaries for Assistant Provost and RPC.
  • Academic Program Costs. The IRP Director worked with the Assistant Provost and Special Assistant to the Provost to prepare detailed cost data on all academic programs for the Dean’s Council.
  • Advising on Data. The IRP staff frequently works with other campus offices (e.g., Admissions, Communications, Community Life & Diversity, Grant & Foundation Relations) on collecting and verifying data.

D. Data Coordination

A chief responsibility of IRP is to help establish and maintain the integrity and consistency of data that is stored, transformed, and reported from several different computer systems. Progress has been made during the past year, but most of this activity is on-going:

  • Policy and Procedures. The IRP Director continues her oversight of policy and procedures to ensure data integrity and appropriate relationships between the faculty database in the Provost's Office and the Lawson financial system in the Human Resources Office. This year included assisting with the training of the new Information Management Coordinator in the Provost’s Office.
  • Institutional Review Board. The IRP Director has worked extensively with the IRB this year, not only reviewing applications for human subjects research, but working to finalize policy documents, application forms, and a user-friendly website.
  • Student Records. The IRP Director continues to advise on the redesign of the student record system with a goal of making consistent and transparent the relation between student enrollment records and faculty instructional records.
  • Alumni Tracking. The IRP Director prepared a time and cost estimate for Cabinet to set up and maintain a comprehensive and on-going collection of data to include professional accomplishments of our alumni together with their admissions/academic characteristics. With Cabinet’s approval, she directed an initial organizational meeting and prepared an outline of the database and its data elements. However, this project has languished and should be revived in the coming year.

E. Public Communication

A chief function of the IRP office is to respond to questions about data from faculty, staff, and students. Requests are quite varied; this year they included requests from The Manitou Messenger staff regarding campus safety, faculty tenure, and career fluctuations in female graduates, from the Nursing program review team regarding Lutheran college nursing programs and detailed data on St. Olaf Nursing graduates, providing data and direction to at least four students for senior or class projects, and preparing summary data and detailed student records on graduation, retention, GPA and other characteristics of eligible students in support of the SSS grant application. IRP also reviews and updates data for use in various campus and admissions publications.

Another type of information sharing takes place with queries and studies initiated by members of HEDS--the Higher Education Data Sharing collaborative. Joy Johnson responds regularly to various e-mail queries from HEDS institutions. We receive summary reports from every query that we answer, and each summary is sent to the relevant office or department. This year St. Olaf responded to approximately 40 requests, covering topics such as policies for recognizing Wicca organizations, first payment policies for new faculty, staffing and funding levels in health services in colleges outside metropolitan areas, honorary degrees and honorariums for commencement speakers, orientation and graduation fees. Also this year we conducted a HEDS survey of our own regarding self-scheduled exams and honor codes. Data was collected and summarized for the Registrar and the Curriculum and Educational Policies Committee (CEPC).

On campus, most broad communication from IRP takes place in the form of e-mail "IRP Notes" that are sent to all faculty, staff and student leaders. Topics of the 2003-04 notes are:
06/25/03 Faculty Compensation Details
08/26/03 External Reports Inventory
02/13/04 Administrative and Staff Survey
04/23/04 Admin/Staff Survey-Seminar Presentation
05/25/04 College Student Survey 2003 (CSS)

Finally, the IRP Office responds to various ad hoc requests from off-campus. This year these included:

  • Worked with a representative of the Class of 1974 to find archived data on an extensive longitudinal study done by the Office of Educational Research from 1970-1974 and provided recent comparative data from the CDS and CIRP surveys. Data will be presented at their 30th reunion.
  • Proofed and corrected data on print materials from Neuger Communications Group for Admissions brochures.
  • Summary of data on science majors for a reporter to the St. Olaf Alumni Magazine.

F. IRP Office Activities

Internal activities of the IRP Office for 2003-04 include:

  • Job Descriptions/Performance Appraisals. Revised the job descriptions for both the Director and Research Associate positions and completed both performance appraisals as part of the campus-wide effort to standardize this process.
  • Computer/Software Upgrades. Susan Canon worked with the Provost Office staff over the summer in conjunction with the upgrade to Microsoft Office 2000, especially as it affected the various faculty databases. This spring the Lawson upgrade also entailed some work for the faculty databases and revisions of data entry procedures. The IRP Office also upgraded to SPSS 12.0. Both Susan Canon and Joy Johnson received new computers this year as the old computers were not functioning well with the upgraded software.
  • Electronic Data Files Inventory. Joy Johnson continues to work with student workers to update and revise the electronic inventory of all IRP data and reports, both electronic and paper.
  • Professional Development (SC). Susan Canon attended the HEDS summer conference in Portland, ME as well as the January conference in Charleston, SC. Susan serves on the HEDS Technology Committee and was just nominated for the Board of Directors (elections will be held in June 2004). She also attended the AIRUM fall conference in Bloomington. On campus, Susan participated in several information sessions for supervisors.
  • Professional Development (JJ). Joy participated in the Excel and beginning Access classes offered by IIT.


G. Looking Ahead

The one staffing change in the IRP office is that Consulting Statistician Paul Roback will no longer have a course release to assist the office, due to receipt of an NSF grant for student statistical research.

In addition to continuing all regular work of the office as outlined above, several projects of significant size and potential benefit to the College remain high on our agenda.

  • Web Site. This summer the IRP student worker will work together with the St. Olaf webmaster and the IRP Director to upgrade the IRP website to the “new look” of the overall St. Olaf website. We will also use this opportunity to revise some of the structure of the website.
  • Survey Schedule. In consultation with the ARP Director and Assistant Provost, the IRP Director has begun a revision of the current IRP survey schedule. There are several factors to consider, including revisions in the HEDS schedule of offering surveys and potential continued use of the pre-NSSE instrument (see below).
  • Alumni Outcomes Tracking Database. Although some preliminary work was done on this in the current year, it should become a priority to create an alumni database that systematically collects information on alumni achievements after graduation (graduate degrees, employment, etc.) and connects that information to St. Olaf data about admissions, demographics, majors, grades, etc. This database could be used as the source of many different investigations, such as (a) an often-requested study of the relation between entering students' major interests, their graduation majors, and their initial post-baccalaureate pursuits and (b) a longitudinal retention study to determine how different factors contribute to students finishing or leaving St. Olaf and, for those who transfer or drop out, whether they complete their studies at other institutions.
  • Strategic or Key Indicators. Finalize and implement the use of a quarterly or annual report tracking key strategic indicators about the college including data from admissions, financial aid, advancement, enrollments, graduation, and faculty and administrative staffing. Benchmarking data could be provided on many indicators as a systematic way to monitor the health of the college in various ways, to provide information for planning purposes, and to keep college constituencies well informed.
  • Pre-National Survey of Student Engagement (NSSE). In conjunction with the ARP Office, IRP will participate in this pilot study of incoming students, to be followed up with administration of NSSE 2005 in the spring. If NSSE continues with this project we will then determine if we want to continue to participate in it, and on what cycle.
  • Cross-Survey Study. There are many items that are similar across existing surveys of different populations (incoming students, current students, faculty, parents, alumni) and it would be valuable to conduct a cross-survey study of those.
  • External Review. It has been seven years since the college last conducted an external review of its institutional research activities. Now that the IRP Director has had a year in her new position, it would be an opportune time to carry out a follow-up review to help establish priorities for services the office could or should be providing.