Institutional Research & Planning 2004-05 Annual Report

Susan Canon, Director
May 2005

The Office of Institutional Research and Planning (IRP) was established in 1998 to provide relevant and accurate information about St. Olaf College. The office monitors strategic characteristics of the College, provides public information about the College, and aids senior administrators and program managers in assessing progress and making decisions to meet institutional goals. Although the 2003-04 academic year staff included a 0.17 FTE release time for a consulting statistician from the Mathematics Department, this has been discontinued due to his commitment to an NSF grant for student statistical research. The IRP office has two full-time staff members: a Director and a Research Associate. IRP reports to the Assistant Provost.


The work of the office is focused on five major kinds of activities:

  1. Overseeing a system of regular surveys of students, faculty, staff, and alumni.
  2. Preparing and validating college data for submission to external agencies.
  3. Providing a variety of reports and advice for internal use by committees and managers.
  4. Helping guide the college data procedures to ensure consistency and utility.
  5. Maintaining an informational website as well as providing data as requested from various internal and external constituencies.

This annual report highlights IRP activities in each of the five major areas, followed (in section F) by a summary of internal activities required to maintain the office itself. A final section (G) outlines potential new projects and looks at priorities, goals, challenges, and opportunities for the future.

A. Surveys

A chief activity of IRP is to conduct surveys - of students, faculty, staff, parents, and alumni - and to communicate insights from these surveys to various college constituencies. Most surveys are conducted on a regular basis – annually, biennially or less frequently as appropriate. Most surveys (e.g., the HERI (Higher Education Research Institute) Faculty Survey and the NSSE (National Survey of Student Engagement and Beginning College Student Survey) are externally prepared and compiled, while a few (e.g., Recent Graduates Survey) are locally developed. Externally prepared surveys, while generic in character, offer the advantage of comparative benchmarks that, when properly used, can be of help in improving the College's educational program.

Most surveys stretch over two years - one for administration, the next for interpretation. Thus this section includes both surveys conducted in 2003-04 whose analysis and interpretation took place in 2004-05 as well as surveys conducted during 2004-05. Results from some of these surveys will not be available until next year.

  • Beginning College Student Survey (BCSS). During Week One last fall, St. Olaf College participated in a pilot study from the National Survey of Student Engagement (NSSE). This study provided baseline data for all incoming freshmen, who were later administered the NSSE survey (see below). Although IRP has received the results, more detailed analysis will be done when the NSSE results are in.
  • Higher Education Research Institute (HERI) Faculty Survey. St. Olaf participates in this triennial survey of faculty and will receive the results later in the summer. Response rate was 45%
  • National Survey of Student Engagement (NSSE). St. Olaf has participated in this national survey in odd-numbered years since Spring 2001. This year IRP surveyed all first year students, as a follow-up to the BCSS, and almost all seniors, as we participated in the on-line version for the first time. We had a record response rate over 75% and expect to receive our data next fall. We also participated in a follow-up phone survey of non-respondents.
  • Recent Graduates. The IRP office conducts an annual survey of the previous year's graduates for the Minnesota Private College Council (MPCC) and the St. Olaf Center for Experiential Learning (CEL). This survey collects information about the primary activity of our graduates (graduate school/employment/volunteer work). The IRP staff finalized reports on the Class of 2003 for MPC and CEL last summer. The MPCC report on the class of 2004, which had a response rate of 48%, has also been completed.
  • CEL Program Review. In conjunction with the Class of 2004 Recent Graduates survey, the IRP office distributed a program review survey for the Center for Experiential Learning. Those results were compiled and reported to CEL in May – response rate on that survey was 44%.
  • HEDS Senior Survey. St. Olaf participated in this national survey last spring for the class of 2004. The data was posted to the IRP website last fall and an IRP Note reporting on the results was sent to the community in January 2005.
  • HEDS Alumni Survey. In spring 2004 IRP administered a “ten-years out” follow-up alumni survey to the class of 1993 (one year late due to a revised administration schedule at HEDS). The data was posted to the IRP website in January and an IRP Note on the results was sent to the community in April. A summary was mailed to the class of 1993 in May.
  • ACHA-National College Health Assessment (NCHA). This survey was administered in spring 2004 by the Wellness Center. The staff coordinator position was eliminated so IRP obtained the data and prepared a data summary for the Student Life division.
  • ACM Engaging Today's Students With the Liberal Arts Survey. Participated in this survey of graduating seniors (Class of 2005) to examine what students think about how important certain items are to a “liberal arts education” and how important these items are to the students themselves in their education. Administered the survey at GradFest and sent the data to an ACM contractor for analysis.
  • Administration/Staff Survey. Although this survey was conducted in Fall 2003 and final reports were made in Spring 2004, the IRP Director gave two more presentations of the data in September 2004 to College Council and to the Advancement Division.

In addition to its own survey schedule, the IRP Director assisted with survey design/implementation for other survey efforts on campus, as follows:

  • Customer Service/Satisfaction surveys from President’s Office
  • Athletic Committee Survey of faculty
  • Family Studies program review survey of alumni
  • Faculty Development Survey for Center for Innovation in the Liberal Arts’ Bush Foundation grant final report and renewal proposal
  • Environmental Values & Literacy surveys for incoming first-year students
  • Intellectual Diversity survey of students

B. External Reports

Most of the reports listed below are completed every year.

  • College Guides. Completed surveys for U.S. News and World Report, Peterson's, College Board, Princeton Review, and many other college guide surveys.
  • Common Data Set. Completed the 2004-05 Common Data Set (CDS) with appendices, posted them on the web, and updated the CDS web summaries. We also submitted our CDS electronically to the Common Data Set Exchange (CDSX) for CDS data sharing among colleges and universities.
  • ELCA. Completed and submitted reports to the ELCA on admissions, financial aid, enrollment and retention.
  • HEDS. Completed various data reports for the Higher Education Data Sharing (HEDS) collaborative: Admissions, EADA Athletics (NCAA), Endowment (NACUBO), Graduation & Retention, Strategic Indicators, Faculty Compensation, Faculty Demographics, Voluntary Support of Education (CAE), FTE & Student Faculty Ratios, Applications, Deposits, Tuition & Fees, Finance.
  • IPEDS. Submitted surveys on Institutional Characteristics and Completions for the federal government's Integrated Postsecondary Education Data System (IPEDS) in Fall 2004. Winter data collection included Employees by Assigned Positions (EAP), Faculty Salaries and Fall Enrollment. Spring 2005 surveys completed were Student Financial Aid, Finance, and Graduation Rate data.
  • Faculty Salary Reports. Compiled and submitted the AAUP, IPEDS, CUPA and ACM reports on faculty salary and benefits.
  • Staff & Employee Reports. Prepared data, in conjunction with the HR staff, for the IPEDS Fall Staff and EAP reports and the ELCA Faculty, Staff & Administration report. Also supplied HEDS with supplemental demographic information on the AAUP faculty salary report.
  • Financial Aid. Assisted Financial Aid Director in compiling unit record data for two major financial aid studies: Minnesota Private Colleges Research Foundation (MPCRF) study of all students enrolled in 2003-04 as well as a pilot study for the National Collegiate Athletic Association (NCAA).



Various other reports are submitted to ACM, ELCA, department accreditation agencies, grant agencies, auditors and other organizations that are either reviewed and supplemented by the IRP office and then sent, or are simply copied to the IRP office.

In addition to the above data submissions, the IRP Director has been involved in various national discussions regarding data collections. She has met with ACM staff and other IRP Directors in the ACM regarding potential changes to the ACM data collection schedules and content. She has closely followed developments at the National Center for Educational Statistics (NCES) regarding changes to existing data reporting as well as the proposed national unit-record data collection. She has submitted comments to the NCES in both of these areas. Both the Director and the Research Associate participate in ongoing electronic discussions regarding the Common Data Set and various data issues with HEDS members.

C. Internal Reports

The IRP Director has become involved in many more campus committees this year, to provide access to existing data and information, to provide an institution-wide perspective on various issues, and to offer relevant services provided by the IRP office.

Committe Membership:

  • Inquiry in Support of Student Learning (ISSL).
    • GE Outcomes. Served on work-groups to draft learning outcomes for FOL and ORC, then worked with the rest of the committee to prepare outcomes on remaining GE requirements for review by CEPC and eventual faculty approval.
    • Survey Item Analysis. Collaborated with Academic Research & Planning (ARP) to classify survey items related to teaching and learning into a searchable grid for use in various assessment efforts.
    • First-year information literacy assessment pilot project. Provided feedback for a survey instrument that assesses student experiences, behaviors, knowledge, critical capacities, and dispositions with respect to information literacy.
    • Teagle & Lumina Foundations grant applications. Worked with Director of ARP to submit two successful grant applications to collaborate with Carleton, Macalester and Grinnell to undertake pilot assessment of effective writing, critical thinking, analytical reasoning, and global outlook.
  • Institutional Review Board. The IRP Director has continued her work with the IRB this year, primarily reviewing applications for human subjects research, but also tweaking changes to policy documents, application forms, and the IRB website.
  • Staff Development Task Force. IRP Director served on this committee to help organize, advertise and host campus-wide staff development workshops, seminars and presentations.
  • Cabinet Inclusivity Committee. IRP Director served on this committee to provide information and input for the update to the Diversity Design Team progress report. Also began work with the Dean of Community Life & Diversity to design and implement a Campus Diversity Assessment survey in 2005-06.
  • Registrar’s Academic Advisory Group. Joined this group in the spring semester, which is primarily a resource-sharing, learning group to understand more about other offices in the institution that work with the Registrar’s office.

Data analysis and reports:

  • GE data for Class of 2004. Worked with Registrar’s Office staff to compile course enrollment data for Class of 2004 as part of GE Task Force study. Wrote summary report.
  • Grades Study. Prepared grades earned and GPA data for Senior Faculty Advisor Lynn Steen to do an analysis of GPAs by instructor, department, and Faculty.
  • Peer Group Revision. Upon request of President Thomforde, the IRP Director reviewed the composition of the St. Olaf peer group that was established in 2003-04. A recommended list of 24 institutions was prepared and presented to College Council and the Board of Regents.
  • Staff & Administrative FTE. Compiled data to provide information about staff and administrative positions at the College. Combining data from a report two years ago, posted a distribution of staff and administrative FTE and headcounts by division, program, and employment status from Fall 2002 and Fall 2004 on the IRP website and wrote an IRP Note summarizing the information.
  • Committee on the First Year Experience. Prepared information regarding first year students, including CIRP survey results.
  • Advising on Data. The IRP staff frequently works with other campus offices (e.g., Admissions, Communications, Community Life & Diversity, Government & Foundation Relations) on collecting and verifying data.

D. Data Coordination

A chief responsibility of IRP is to help establish and maintain the integrity and consistency of data that is stored, transformed, and reported from several different computer systems. Progress has been made during the past year, but most of this activity is on-going:

  • Policy and Procedures. The IRP Director continues her oversight of policy and procedures to ensure data integrity and appropriate relationships between the faculty database in the Provost's Office and the Lawson financial system in the Human Resources Office. This year included assisting with the training of another new Information Management Coordinator in the Provost’s Office.
  • Student Records. The IRP Director continues to advise on the redesign of the student record system with a goal of making consistent and transparent the relation between student enrollment records and faculty instructional records.
  • Data Privacy/Confidentiality. Joined preliminary discussions among IIT, Business Office, and Human Resources staff regarding institution-wide policy statements on data privacy, data storage, data sharing and confidentiality.

E. Public Communication

A chief function of the IRP office is to respond to questions about data from faculty, staff, and students. Requests are quite varied; this year they included requests from a student doing a report for the Board about St. Olaf’s rankings in various publications, additional data for the SSS grant application, and numerous requests from Community Life & Diversity and Government & Foundation Relations, including information for the Kresge Foundation science initiative. IRP also reviews and updates data for use in many campus and admissions publications and websites.

Another type of information sharing takes place with queries and studies initiated by members of HEDS--the Higher Education Data Sharing collaborative. Joy Johnson responds regularly to various e-mail queries from HEDS institutions. We receive summary reports from every query that we answer, and each summary is sent to the relevant office or department. This year St. Olaf responded to approximately 35 requests, covering topics such as student residence requirements, pre- and post-tenure sabbatical policies, health center staffing and whether or not institutions hold classes on Labor Day.

On campus, most broad communication from IRP takes place in the form of e-mail "IRP Notes" that are sent to all faculty, staff and student leaders. Topics of the 2004-05 Notes are:
09/28/04 Retention and Graduation Rates
12/07/04 AAUP Faculty Compensation Report
01/07/05 Senior Survey, Class of 2004
03/30/05 Staff & Administrative FTE, Fall 2004
04/25/05 Class of 1993 Alumni Survey, Spring 2004

These IRP Notes as well as all other IRP reports and data summaries are posted on the IRP website. This past summer the IRP student worker worked with the St. Olaf webmaster and the IRP Director to upgrade the IRP website to the “new look” of the overall St. Olaf website. Some structure of the website was also revised and updates to webpages is ongoing.

Finally, the IRP Office responds to various ad hoc requests from off-campus. This year these included:

  • Information for CosmoGIRL!'s Best College Guide
  • Updates for the Colleges that Change Lives (CTCL) website
  • Data for www.privatecolleges.com college guide

F. IRP Office Activities

Internal activities of the IRP Office for 2004-05 include:

  • Professional Development (SC). Susan Canon attended the HEDS January conference in Santa Fe, NM. Susan serves on the HEDS Technology Committee and was just nominated for the Board of Directors (elections will be held in June 2005). She also attended the AIRUM fall conference in Bloomington. On campus, Susan participated in several information sessions for supervisors. Susan has applied for the Graduate Certificate in Institutional Research program at Penn State’s World Campus and will begin taking courses online this summer.
  • Professional Development (JJ). Joy Johnson participated in the DreamWeaver and advanced Access classes offered by IIT. She will be attending the national AIR Forum in San Diego this month.


G. Looking Ahead

In addition to continuing all regular work of the office as outlined above, several projects are just getting underway for the coming year:

  • Move IRP office. IRP has been requested to move into the Administration Building 100 suite to accommodate a closer proximity to Admissions for the Financial Aid Office. We hope to complete this move in the summer 2005.
  • Cohort retention analysis. Work with a student from the Center for Integrative Research has began for an analysis of student characteristics as they relate to retention and graduation. This work will be completed during the summer 2005, although depending on the results, there may be some follow-up work by the IRP staff.
  • Academic Program Costs. Picking up an on-going project from previous years, the IRP Director will work with a new design of summary of expenses broken out by academic program for Dean’s Council and eventual wider distribution.
  • Biology/Chemistry initiative. The IRP Director will work with a grant-funded faculty task force exploring an integrated science approach to teaching introductory biology and chemistry. She will prepare data on current enrollment patterns, grades earned, and high-school preparation in these areas.
  • Campus Climate Assessment. At the request of President Thomforde, the IRP Director is working with a task force led by Dean Eida Berrio to design and implement a Campus Climate Assessment survey to be administered to all faculty, staff and students during the 2005-06 academic year.
  • Cross-Survey Study. In continuation of the survey item analysis project with ARP, the IRP staff will begin compiling the data from existing surveys to accommodate the cross-survey item analysis format.
  • Teagle and Lumina Foundation Grants. With these successful grant applications, IRP will be working closely with ARP to undertake pilot assessment of effective writing, critical thinking, analytical reasoning, and global outlook. This will include the implementation of the Collegiate Learning Assessment (CLA) over the next four years, as well as collaboration with Carleton, Macalester and Grinnell on other assessment and faculty development programs.

There are several projects of significant size and potential benefit to the College that remain high on our agenda. We plan to initiate or revitalize these projects in the coming year; some projects may take more than a year to complete or will be ongoing.

  • External Review. Given the close interaction between ARP and IRP and overlap of mission in many areas, we will be coordinating an external review of both offices together in the coming year. This will provide us with the opportunity to examine the distribution of work between the two offices, the efficiencies that may be obtained by working more closely together, and the impetus to establish priorities for both offices.
  • Alumni Outcomes Tracking Database. Although some preliminary discussions have taken place, no work has been done to create an alumni database that systematically collects information on alumni achievements after graduation (graduate degrees, employment, etc.) and connects that information to St. Olaf data about admissions, demographics, majors, grades, etc. This could likely become a priority project for the next few years. The database could be used as the source of many different investigations, such as (a) an often-requested study of the relation between entering students' major interests, their graduation majors, and their initial post-baccalaureate pursuits and (b) a longitudinal retention study to determine how different factors contribute to students finishing or leaving St. Olaf and, for those who transfer or drop out, whether they complete their studies at other institutions.
  • Strategic or Key Indicators. In the 2004-09 Strategic Plan the IRP Director was charged with establishing a peer group of colleges in order to “make regular and consistent use of the peer group to evaluate the work of the college.” Although the peer group has now been established, an ongoing challenge for IRP is to encourage the use of a quarterly or annual report tracking key strategic indicators about the college. This report could include data from admissions, financial aid, advancement, enrollments, graduation, and faculty and administrative staffing. Benchmarking data could be provided on many indicators as a systematic way to monitor the health of the college in various ways, to provide information for planning purposes, and to keep college constituencies well informed.
  • Study Abroad/Financial Aid analysis. As part of the college’s ongoing commitment to incorporating a global perspective, IRP will work with Financial Aid to examine the financial profile of study abroad participants, to help determine if the cost of such programs is prohibitive – or perceived to be prohibitive – to lower-income students.
  • Need-sensitive admissions analysis. With the completion of at least two admissions cycles using need-sensitive admissions, IRP will need to work with the Treasurer and Financial Aid Office to determine the impact of this change for the institution, both financially and on the profile of the student body.

Looking at the longer term of the next 3-5 years, IRP will face some significant challenges. The staff must maintain the ongoing workload of reporting required data and information consistently and accurately to external and internal constituencies. This work is usually set by external deadlines or internal publication schedules. At the same time the staff is expected to provide analysis of survey data, work on specific requested projects, and contribute valuable exploration of institutional and comparative data. IRP is uniquely positioned in the institution to coordinate information-sharing and work on research that crosses all divisions of the college. However, the staff is limited in its ability to initiate many new projects by its time commitment to existing data collections and reporting. Coordinating efficiencies with the ARP staff, better utilizing student workers and working with student researchers through the Center for Interdisciplinary Research will enable the IRP Office to accomplish more of its goals.