Institutional Research and Planning (IRP) 2006-2007 Annual Report

Susan Canon, Director
May 2007


The Office of Institutional Research and Planning (IRP) was established in 1998 to provide relevant and accurate information about St. Olaf College. The office monitors strategic characteristics of the College, provides public information about the College, and aids senior administrators and program managers in assessing progress and making decisions to meet institutional goals. The IRP office has two full-time staff members: a Director and a Research Analyst. IRP reports to the Assistant Provost.

The work of the office is focused on five major kinds of activities:

  1. Overseeing a system of regular surveys of students, faculty, staff, and alumni.
  2. Preparing and validating college data for submission to external agencies.
  3. Providing a variety of reports and data summaries for internal use by committees and managers.
  4. Helping guide the college data procedures to ensure consistency and utility.
  5. Maintaining an informational website as well as providing data as requested from various internal and external constituencies.

This annual report highlights IRP activities in each of the five major areas, followed (in section F) by a summary of internal activities required to maintain the office itself. A final section (G) outlines potential new projects and looks at priorities, goals, challenges, and opportunities for the future.

A. Surveys

A chief activity of IRP is to conduct surveys - of students, faculty, staff, parents, and alumni - and to communicate insights from these surveys to various college constituencies. Most surveys are conducted on a regular basis – annually, triennially or less frequently as appropriate. Most surveys (e.g., the HEDS [Higher Education Data Sharing Consortium] Senior Survey) are externally prepared and compiled, while a few (e.g., Recent Graduates Survey) are locally developed. Externally prepared surveys, while generic in character, offer the advantage of comparative benchmarks that, when properly used, can be of help in improving the College's educational program.

Most surveys stretch over two years - one for administration, the next for interpretation. Thus this section includes both surveys conducted in 2005-06 whose analysis and interpretation took place in 2006-07 as well as surveys conducted during 2006-07. Results from some of these surveys will not be available until next year.

  • Parent Survey. In Spring 2006, the IRP office conducted an internally designed survey of parents of students in the classes of 2005-2009. The IRP Director sent an IRP Note describing parents’ satisfaction levels with student services and information resources to the community in June, and followed with an IRP Note regarding parents’ satisfaction with their students’ academic experiences and development in September. In October the IRP Director presented the data for an Informed Workplace session. In addition, all frequency data and selected graphs comparing the 2006 and 2002 responses were posted to the IRP website.
  • HEDS Alumni Survey. Also in Spring 2006, IRP administered a “ten-years out” follow-up alumni survey for the Class of 1996 as well as a “five-years out” survey for the Class of 2000. Detailed data results, as well as comparisons to previous class surveys, have been posted to the IRP website.
  • Campus Climate Survey. This survey, initiated by the Dean of Community Life and Diversity (CLD), was administered to the entire St. Olaf community in early Spring 2006. After helping prepare some preliminary reports in May 2006, the Director of IRP continued to work together with the Dean of CLD and members of the President’s Advisory Committee on CLD Concerns to present this data to the community throughout the fall semester.
  • Recent Graduates. The IRP office conducts an annual survey of the previous year's graduates for the Minnesota Private College Council (MPCC) and the St. Olaf Center for Experiential Learning (CEL). This survey collects information about the primary activity of our graduates (graduate school/employment/volunteer work). The IRP staff finalized reports on the Class of 2005 for MPCC and CEL last summer and posted summaries to the IRP website. The MPCC report on the class of 2006, which had a response rate of 51%, has already been completed and summary results have been posted to the IRP website. More detailed reports will be finalized for the CEL during the summer.
  • HEDS Senior Survey. In Spring 2007 IRP participated in the first online version of the HEDS Senior Survey to be hosted by HEDS on getsurveyed.com. Tables were set up at GradFest to encourage participation, as well as an e-mail invitation and reminder from the Dean of Students. Final response rate was 66%; raw data is currently being summarized by the IRP Research Analyst and will be available to the community by the end of the summer.
  • Staff Survey. This internally-designed survey was administered for the third time in May 2007, with a final response rate of 52%. IRP staff will analyze the data over the summer to present to the community by fall semester.
  • American College Health Association’s (ACHA) National College Health Assessment (NCHA). IRP assumed the administration of this survey from the Dean of Students’ Office, with the intention of working it into the regular triennial survey cycle. The survey was administered online to all current students beginning the week after spring break and received a total response rate of 36% (1,030 students). Data and a final report from ACHA should be received by the end of the summer.

B. External Reports

Every year IRP works with the relevant offices and databases on campus to prepare and review reports for many external organizations and agencies. Data for numerous topics are reported, including general institutional characteristics, admissions, financial aid, enrollment, graduation, retention, faculty/staff compensation and demographics, endowment, finance, tuition and fees, and voluntary support. Most reporting is in summary form; some require submission of unit-record data for individual students.

  • Publications
    o Common Data Set (CDS). The completed CDS is sent to many college guide publishers. IRP also posts it on its website, together with longitudinal summaries of the data and other data asked for regularly by publishers.
    o College Guides’ Supplemental Surveys. In addition to the CDS, U.S. News and World Report, Peterson's, College Board, Princeton Review, and many other college guides request much additional data.
  • Institutional Associations
    o American Association of University Professors (AAUP)
    o Associated Colleges of the Midwest (ACM)
    o College & University Professional Association for Human Resources (CUPA-HR)
    o College Sports Project
    o Evangelical Lutheran Church of America (ELCA)
    o Higher Education Data Sharing Consortium (HEDS)
    o Minnesota Private Colleges Research Foundation (MPCRF)
    o National Collegiate Athletic Association (NCAA)
  • State & Federal Government
    o Minnesota Office of Higher Education (MOHE)
    o Minnesota Career Information System (MCIS)
    o National Center for Educational Statistics (NCES): Integrated Postsecondary Education Data System (IPEDS).
    o US Department of Education: Equity in Athletics Disclosure Act (EADA)

Various other reports are submitted to institutional and departmental accreditation agencies, grant agencies, auditors and other organizations. These reports are either reviewed and supplemented by the IRP office and then sent, or a final copy is sent to the IRP office as the central repository for institutional reports. One major addition to the above data submissions was beginning participation in the College Sports Project, an initiative of The Andrew W. Mellon Foundation. This unit record data submission for all full-time students initially enrolled in 2005-06 required detailed extraction of data from the newly-designed Student Information System and coordination with the Athletics Department’s information on student athletes.

Both the Director and the Research Analyst participate in ongoing electronic discussions regarding the Common Data Set and various data collection issues with HEDS members.

C. Internal Data Analyses, Collaborations and Committee Involvement

IRP provides a variety of reports and data summaries for internal use by committees and managers. In addition, the IRP Director continues involvement in many campus committees and collaborations this year. Involvement with these groups provides access to existing data and information, an institution-wide perspective on various issues, and allows the IRP office more opportunity to offer its relevant services.

Data collections, analyses and reports:

  • Center for Integrative Research (CIR) Financial Aid Analysis. Worked with a group of students from the CIR in conjunction with the Financial Aid Director to design and administer a survey of current students regarding spending habits, debt accumulation, and sources of income/financial aid. The survey data will be analyzed next year. In addition, the group did an analysis of student financial aid and demographic data compiled by the IRP and Financial Aid directors.
  • Student Samples for Tenure & Promotion reviews. Worked with another new ARP Associate to conduct electronic random sampling for forming student evaluation groups for faculty tenure & promotion reviews.
  • External data comparisons. IRP provides external benchmarking information to the St. Olaf leadership in support of institutional planning. This year included providing detailed information from AAUP and CUPA faculty compensation studies, an AAUP/IPEDS Contingent Faculty report summarized data for Dean’s Council, and updating the baccalaureate origins of PhDs ranking data.
  • Internal data collections. IRP continues its service as data-provider, especially when it involves integrating data from two or more internal systems. This year data collections included a review of average class sizes across different terms for Dean’s Council, courses taken by Asian Studies majors for a departmental review, grades earned data for a study by a group of Statistics 316 students, chemistry placement exam data and St. Olaf chemistry grades earned for the chemistry department, a summary of tenure rates for Dean’s Council, and various age of faculty and tenure-track hiring history summaries for the Provost and President.
  • Advising on Data. The IRP staff frequently works with other campus offices (e.g., Admissions, Advancement, Communications, Financial Aid, Government & Foundation Relations) on collecting and verifying data.

Campus Collaborations:

  • Teagle Foundation grant. The IRP Director continues to serve on the steering committee for this three-year grant, together with the St. Olaf Director of ARP, as well as administrators and faculty from Carleton, Macalester, and Grinnell. A primary focus at St. Olaf in the second year of the grant was administering the Collegiate Learning Assessment (CLA) to the sophomore cohort who had taken the assessment in the previous year, as well as working with the Collaborative Assessment for Liberal Learning (CALL) faculty teams to look at data collected in the previous year.

Committee Membership:

  • Inquiry in Support of Student Learning (ISSL). The work of this committee was folded into much of the work of the CALL teams and implementation of the CLA (see above), as well as working with the ARP Director on the required interim report to the North Central Association’s Higher Learning Commission.
  • Institutional Review Board (IRB). The IRP Director has continued her work with the IRB this year, primarily reviewing applications for human subjects research and working with the IRB Director and student assistant to redesign the IRB application form.
  • Staff Development Task Force. The IRP Director served on this committee to help organize, advertise and host campus-wide staff development workshops, seminars and presentations.
  • Registrar’s Academic Advisory Group (RAAG). This group is a resource-sharing/learning group that meets monthly to understand more about other offices within the institution that work with the Registrar’s office.
  • Information Systems Integration Team (ISIT). This team was formed a couple of years ago and is now headed by IIT Director Roberta Lembke. Roberta invited the IRP Director to join this team in April 2007. The team addresses issues of information-sharing, effective workflow processes, and cross-divisional initiatives to increase efficiencies for the institution.

D. Data Coordination

A responsibility of IRP is to help establish and maintain the integrity and consistency of data that is stored, transformed, and reported from several different computer systems. Most of this activity is on-going:

  • Policy and Procedures. The IRP Director continues her oversight of policy and procedures to ensure data integrity and appropriate relationships between the faculty database in the Provost's Office and the Lawson financial system in the Human Resources Office. Joining the ISIT committee (see above) will enable her to work more closely with the other data systems on campus as well.
  • Student Information System. The IRP Director continued to work with IIT and Registrar staff as the new student record system was brought completely online this past fall. She continues to provide input to the IIT and Registrar staff about the new system.

E. Public Communication

Another function of the IRP office is to respond to questions about data from faculty, staff, and students. Requests are quite varied; this year they included requests for data for Manitou Messenger articles, student requests for data for class projects, Regents’ Student Committee, Office of Student Leadership and the Admissions office. IRP also reviews and updates data for use in many campus and admissions publications and websites. Summer 2006 involved major revisions to the college Viewbook, with much input and fact-checking from the IRP staff.

Most broad communication from IRP on campus takes place in the form of e-mail "IRP Notes" that are sent to all faculty, staff, emeriti, and student leaders. Topics of the 2006-07 notes are:

06/01/06 St. Olaf Parent Survey: Satisfaction with services, information resources
09/25/06 St. Olaf Parent Survey: Satisfaction with students' academic experiences and development
04/26/07 Faculty Compensation: AAUP report

These IRP Notes, as well as all other IRP reports and data summaries are posted on the IRP website. Updates to web pages are ongoing. In addition, the IRP Director made Informed Workplace seminar presentations on the Campus Climate and Parent Surveys.

Information sharing also takes place outside the campus with queries and studies initiated by members of HEDS. The IRP Research Analyst responds regularly to various e-mail queries from HEDS institutions. We receive summary reports from every query that we answer, and each summary is sent to the relevant office or department on campus. This year St. Olaf responded to approximately 75 requests, covering such diverse topics as academic suspension, measuring geographic diversity, dependency status changes, faculty development fund, solicit-able alumni and census date

Finally, the IRP Office responds to various ad hoc requests from off-campus. This year these included inquiries from:

  • Manhattanville College Strategic Planning committee and the University of Sioux Falls, both conducting studies on faculty workload
  • University of Virginia-College at Wise collecting peer institution data for a five-year program review report
  • Beloit College Department of Philosophy and Religious Studies completing a self-study in preparation for an external review
  • Lawrence University updating its dashboard indicators and peer comparisons
  • a student from Lake Forest College conducting research about commuter students for a journalism class
  • Tufts University researching the collecting of sexual orientation and transgender information on college campuses.

F. IRP Office Activities

Internal activities of the IRP Office for 2006-07 included:

  • IRP/ARP Self Study and External Review. IRP staff spent a lot of time this year preparing its portion of the ARP/IRP Self Study document, planning the schedule and details of the external review team’s site visit, and working with the reviewers for their final review report. See below section Looking Ahead for more detail on the results of this review.
  • Director, Professional Development. Susan Canon attended the HEDS summer conference in Minneapolis and the January Forum in Santa Fe, NM. At the summer conference Susan was elected to the HEDS Board of Directors for a three-year term. This prompted her to attend the NEAIR conference in Philadelphia in November in order to also join a HEDS Board half-day meeting after the conference. She also attended the AIRUM fall conference in Bloomington, MN. At the January HEDS Forum, Susan coordinated a joint presentation with colleagues from Carleton College and Macalester College, as well as St. Olaf’s Director of Academic Research, regarding experiences administering and using data from the Collegiate Learning Assessment (CLA). She also participated in a HEDS Board retreat for strategic planning in April 2007 in Philadelphia as well as numerous conference calls. On campus, Susan participated in several Informed Workplace sessions and CILA lunch conversation groups.
  • Research Analyst, Professional Development. Joy Johnson also attended the HEDS summer conference in Minneapolis in June 2006. She continued to develop her query skills in Microsoft Access and practiced survey development by creating a survey for field testing ear medicine in dogs in a three-hospital study and a veterinary satisfaction survey. She also diversified her work load by taking on additional responsibilities, such as doing the 10th day enrollment statistics for IPEDS. Joy also participated in several Informed Workplace sessions.

G. Looking Ahead

Our highest priority this summer is to respond to the recommendations from the external program review team’s report (see above section in IRP Office Activities). Working closely with the Assistant Provost and the Director of ARP, the IRP staff will plan which recommendations to implement or modify as appropriate, and which ones may not fit with our goals at this time.

The staff must maintain the reporting of required data and information consistently and accurately to external and internal constituencies. Additionally, the staff will continue to provide analysis of survey data, work on specific requested projects, and contribute valuable exploration of institutional and comparative data. IRP is uniquely positioned in the institution to coordinate information-sharing and work on research that crosses all divisions of the college. Therefore, there are several projects of significant size and potential benefit to the College that remain high on our list of potential projects. Depending on decisions based on our program review, we will hopefully be able to focus on a few of these in the coming year or two.

  • Strategic or Key Indicators. For a number of years IRP has tried to encourage the use of a quarterly or annual report tracking key strategic indicators about the college. This report could include data from admissions, financial aid, advancement, enrollment, graduation, and faculty and administrative staffing. Benchmarking data could be provided on many indicators as a systematic way to monitor the health of the college in various ways, to provide information for planning purposes, and to keep college constituencies well informed. The program review report recommended this be done.
  • Alumni Outcomes Tracking Database. Several years ago the former IR Director proposed the creation of an alumni database to systematically collect information on alumni achievements after graduation (graduate degrees, employment, etc.) and connect that information to St. Olaf data about admissions, demographics, majors, grades, etc. The database could be used as the source of many different investigations, such as (a) an often-requested study of the relation between entering students' major interests, their graduation majors, and their initial post-baccalaureate pursuits and (b) a longitudinal retention study to determine how different factors contribute to students finishing or leaving St. Olaf and, for those who transfer or drop out, whether they complete their studies at other institutions. Although some preliminary discussions have taken place, no work has been done to further this project.
  • Need-sensitive admissions analysis. With the completion of at least four admissions cycles using need-sensitive admissions, IRP could work with the Treasurer, Registrar, Admissions, and Financial Aid offices to determine the impact of this change for the institution, both financially and on the profile of the student body.
  • Study Abroad/Financial Aid analysis. This project needs to be revitalized and completed.
  • Institutional Review Board. Although not directly a function of either ARP or IRP, the program review team did make some recommendations regarding the college’s IRB process. The IRP staff will work with the IRB Chair/ARP Director to implement these recommendations as appropriate. As noted above, we have already designed a new application form and we intend to refine the application process over the summer for implementation this fall.