St. Olaf CollegeInstitutional Research and PlanningSt. Olaf College

2002-2003 Annual Report

Institutional Research and Planning (IRP)

Lynn Steen, Director
May 2003

 

The Office of Institutional Research and Planning (IRP) was established in 1998 to provide relevant and accurate information about St. Olaf College in order to monitor strategic characteristics of the College, to provide public web-based information about the College, and to aid senior administrators and program managers in assessing progress and making decisions to meet institutional goals. From June 2000 through May 2003 the IRP office has had a staff of 2.25 FTE-a part-time faculty director, a full time associate director, and a full time research associate. It reports to the Assistant Provost.

The work of the office is focused on five major kinds of activities:

A. Overseeing a system of regular surveys of students, faculty, staff, and alumni.

B. Preparing and validating college data for submission to external agencies.

C. Providing a variety of reports and advice for internal use by committees and managers.

D. Helping guide the college data procedures to ensure consistency and utility.

E. Maintaining an informational website for communicating with various constituencies.

Important information gleaned from these activities are communicated regularly to faculty, administrators, staff, and student leaders via e-mailed IRP Notes that summarize more detailed presentations posted on the IRP web site (http://www.stolaf.edu/offices/irp). All "hard" data concerning such things as admissions, enrollment, majors, athletics, graduates, and faculty are posted without access restrictions; results of opinion surveys and occasional other special reports are restricted to on-campus readers.

 

This annual report highlights IRP activities in each of the five major areas, followed (in section F) by a summary of internal activities required to maintain the office itself. A final section (G) outlines personnel changes planned for next year as well as potential new projects.

 

A. Surveys

A chief activity of IRP is to conduct surveys-of students, faculty, staff, parents, and alumni-and to communicate insights from these surveys to various college constituencies. Most surveys (e.g., the HERI (Higher Education Research Institute) Faculty Survey from UCLA and the National Survey of Student Engagement (NSSE) from Indiana University) are externally prepared and administered, while a few (e.g., last year's Parents' Survey) are locally developed. Externally prepared surveys, while generic in character, offer the advantage of comparative benchmarks that, when properly used, can be of help in improving the College's educational program.

 

Most surveys stretch over two years--one for administration, the next for interpretation. Thus this section includes both surveys conducted in 2001-02 whose analysis and interpretation took place in 2002-03 as well as surveys conducted during 2002-03 whose results will not be available until next year.

Cooperative Institutional Research Program (CIRP). In August 2002 St. Olaf participated in the national survey of incoming students conducted by the Higher Education Research Institute (HERI). (St. Olaf participates biennially.) This spring we posted the data for this survey on our the IRP web site and e-mailed an "IRP Note" to faculty, administrators, staff, and student leaders highlighting trends at St. Olaf as compared with national norms.

HERI Faculty Survey. After receiving data from the HERI survey of faculty conducted in the fall of 2001, two IRP Notes were prepared and distributed comparing these results with data from a Higher Education Data Sharing (HEDS) consortium peer group and with the results from a previous HERI survey in 1999. Detailed comparison tables were posted on the IRP web page.

National Survey of Student Engagement (NSSE). St. Olaf participated in the 2001 NSSE survey as a pilot study and reported results on campus last year. In June 2002 the Board of Regents undertook a two-hour discussion of the NSSE data and its implications. This year, in the spring of 2003, following a plan to undertake NSSE every other year, St. Olaf students (225 first-years, 225 seniors) again participated in this survey. Response rate was 65%, much better than in 2001. In addition George Kuh, director of NSSE, worked with IRP director Lynn Steen to enhance the quantitative dimensions of the survey, and Jo Beld, director of the office of Academic Research and Planning (ARP) used parts of the NSSE survey to provide data for a review of the American Conversation program.

Recent Graduates. The IRP office conducts an annual survey of the previous year's graduates for the Minnesota Private College Council (MPCC) and the St. Olaf Center for Experiential Learning (CEL). This two-part survey collects information about the primary activity of our graduates (graduate school/employment/volunteer work) as well as usage of the CEL and its internship programs. Based on data from prior surveys, the IRP staff prepared a summary of the primary activities for the classes of 1997 to 2001 and wrote two IRP Notes conveying key results of this study. We also prepared final reports on the Class of 2001 for MPC and CEL; a preliminary report on CEL usage from the class of 2002 has also been completed.

Parent Survey. Over the summer of 2002 the staff prepared a full report and summary IRP Note on an internally designed survey of parents of students in the classes of 2001-2005 that was conducted in the spring of 2002.

Alumni Surveys. IRP staff prepared two reports and related data tables on HEDS surveys for alumni classes of 1990 and 1996 that were conducted in the spring of 2002. Both reports are posted on the IRP Web page.

Faculty Activities Survey. The IRP Office, together with Provost and Dean of the College, encouraged faculty participation in a pilot study from the University of Delaware on "out-of-classroom" activity. A web version of the survey form was designed and information about the survey was distributed to 173 tenured and tenure-track faculty. Due to a low response rate (31%) the IRP Office decided not to submit our data to the Delaware office.

 

B. External Reports

Most of the reports listed below recur every year, with the exception of the report prepared for the NCA accreditation visit.

College Guides. Completed surveys for U.S. News and World Report, Peterson's, College Board, Princeton Review, and other college guide surveys. A newly revived survey was the Minnesota Career Information System (MCIS) from the Minnesota Department of Children, Families and Learning, which provides data for a computerized career information system to Minnesota schools and agencies as well as the Internet System for Education and Employment Knowledge (ISEEK).

Common Data Set. Completed the 2002-03 Common Data Set (CDS) with appendices, posted them on the web, and updated the web summaries that IRP has prepared to make the CDS data easier to access.

Basic Institutional Data Forms. Prepared 20 pages of data for the accreditation team visit from the Higher Learning Commission of the North Central Association of Colleges and Schools.

Instructional Costs. Prepared St. Olaf data for the National Study of Instructional Costs and Productivity being carried out by the University of Delaware. This pilot project required considerable effort to link our academic and financial records, an effort which should prove valuable as we refine our new student data system. We should receive comparative information from the study sometime next year.

ELCA. Completed and submitted reports to the ELCA on admissions, financial aid, enrollment and retention.

HEDS. Completed various data reports for the Higher Education Data Sharing (HEDS) collaborative: Admissions, Freshman Financial Aid, EADA Athletics (DOE), Endowment (NACUBO), Graduation & Retention, Strategic Indicators, Faculty Compensation, Faculty Demographics, Voluntary Support of Education (CAE), FTE & Student Faculty Ratios, Applications, Deposits, Tuition & Fees, Finance (NCES).

IPEDS. Submitted surveys on Institutional Characteristics and Completions for the federal government's Integrated Postsecondary Education Data System (IPEDS) in Fall 2002. Winter data collection included Employees by Assigned Positions, Faculty Salaries and Fall Enrollment. Spring 2003 surveys completed were Student Financial Aid, Finance, and Graduation Rate data.

• Salary Reports. Compiled and submitted the AAUP, IPEDS, CUPA and ACM reports on faculty salary and benefits,

Staff & Employee Reports. Prepared data, in conjunction with the HR staff, for the ELCA Faculty, Staff & Administration report. Also supplied HEDS with supplemental demographic information on the AAUP faculty salary report.

Various other reports are submitted to ACM, ELCA, department accreditation agencies, auditors and other organizations that are either reviewed and supplemented by the IRP office and then sent, or are simply copied to the IRP office.

 

C. Internal Reports

Retention by Ethnicity. Reviewed and revised retention and graduation rate data from the registrar's office and prepared data tables summarizing the information by ethnicity. Several reporting errors were discovered and corrected. These tables, beginning with cohort of first-time degree-seeking students who entered in 1991, were subsequently posted on the IRP web site and an IRP Note summarizing the data was distributed.

Earned Doctorates. Prepared an IRP note with supporting data on the IRP web site on doctorates earned by St. Olaf alumni in comparison to all baccalaureate colleges for the period 1991-2000.

Science Graduates and Faculty. Researched and responded to questions from the Development Office and the Associate Dean for Natural Sciences and Mathematics regarding science graduates and faculty. Required downloading student records for graduates in the period 1991-2002 and summarizing majors by gender; also summarized faculty FTE by rank and gender for the period 1994-2002.

Advising on Data. The IRP staff frequently advises other campus offices (e.g., Admissions, Communications, Education) on collecting and verifying data.

College Rankings. Prepared two detailed memoranda on St. Olaf's status vis-a-vis the US News and World Report annual rankings of colleges--one in the fall for the Provost and one in the spring for the President.

Financial Reporting. Developed several reports on college finances and financial aid, general ones for posting on the IRP web site and more detailed ones for use by the Associate Deans.

 

D. Data Coordination

A chief responsibility of IRP is to help establish and maintain the integrity and consistency of data which is stored, transformed, and reported from several different computer systems. Much progress has been made during the past year, but most of this activity is never-ending:

Policy and Procedures. Susan Canon continues her oversight of policy and procedures to ensure data integrity and appropriate relationships between the faculty database in the Provost's Office and the Lawson financial system in the Human Resources Office.

Institutional Review Board. Susan Canon was appointed to the newly formed Institutional Review Board, which is charged with reviewing applications for all human subjects research being conducted by St. Olaf faculty, staff and students. Work on this board includes an on-going review of the policy on surveys that was approved by the Cabinet in the fall of 2001, including approval forms and guidelines for approval.

Student Records. Susan Canon continues to advise on the redesign of the student record system with a goal of making consistent and transparent the relation between student enrollment records and faculty instructional records.

Staff Data. This fall IRP requested a download of administrative/staff FTE data from the Lawson system in the Human Resources Office and prepared a summary for the IRP web page. Working with this data highlighted the need for review and revision of some of the classification codes used in the Lawson system.

Longitudinal Retention Study. Began collecting data for an on-going study to track each entering class to determine how different factors (e.g., GPA, SAT scores, ethnicity, finances) contribute to students finishing, leaving, or dropping out of St. Olaf. Includes collecting data from Admissions, Registrar and Financial Aid databases.

 

E. Public Communication

A chief function of the IRP office is to respond to questions about data from faculty, staff, and students. Requests are quite varied; this year they included requests from The Manitou Messenger staff regarding racial/ethnic enrollments and graduation data, a 10-year analysis of majors by gender in the sciences to support the Science Center initiative, a request from the Dean's Council to summarize majors offered at 22 peer institutions an historic review of mathematics majors, and a request for data for gender-related data from the committee on the Status of Women. IRP also reviews and updates data for use in various campus and admissions publications.

 

Another type of information sharing takes place with queries and studies initiated by members of HEDS--the Higher Education Data Sharing collaborative. Joy Johnson responds regularly to various e-mail queries from HEDS institutions. We receive summary reports from every query that we answer, and each summary is sent to the relevant office or department. This year St. Olaf responded to approximately 60 requests, covering topics such as policies for skateboarding on campus, Latin honors, gender and careers, outside evaluators, academic calendar, contact hours, etc. Also this year we conducted a HEDS survey of our own regarding career and graduate school services offered to recent graduates and alumni. Data was collected and summarized for the Dean of the College and the Director of Center for Experiential Learning (CEL) as part of a review of the services offered by St. Olaf.

 

On campus, most broad communication from IRP takes place in the form of e-mail "IRP Notes" that are sent to all faculty, staff and student leaders. Topics of the 2002-03 notes are:

09/03/02 Report on 2002 Survey of St. Olaf Parents

09/05/02 Faculty Survey--Highlights & Comparisons

09/15/02 US News College Rankings, 2002-03

09/25/02 Retention and Graduation Rates

11/03/02 ACM/GLCA Faculty Salary Comparisons

11/05/02 Staff & Administrative FTE

11/22/02 HERI Faculty Survey

12/16/02 Recent Graduates Survey

01/28/03 Incoming Freshmen Survey (CIRP 2002)

02/07/03 Ph.D.s awarded to St. Olaf graduates

02/11/03 Follow-up to Recent Graduates Survey

02/17/03 Majors at Peer Institutions

04/01/03 Financial Aid

04/08/03 Retention and Graduation Rates by Ethnicity

05/01/03 IPEDS Financial Indicators

 

Finally, the IRP Office responds to various ad hoc requests from off-campus. This year these included:

  • As part of the postsecondary education research project of the Federal Reserve Bank of Minneapolis, the IRP office provided summary data on the number of graduates who remained in Minnesota after graduation for the classes of 1990, 1992, 1994, 1996, 1998, and 2000.
  • Provided Lutheran enrollment information for a St. Olaf graduate doing a master's thesis at Harvard.
  • Proofed and corrected data on print materials from Neuger Communications for Admissions brochures.
  • Supplied copies of class evaluations and information on campus climate surveys (staff and student) to a St. Olaf graduate for use in planning a high school evaluation system.

 

F. IRP Office Activities

Internal activities of the IRP Office for 2002-03 include:

Job Classification. Completed the PDQ and wrote a new job description for the position of Director.

Computer Upgrades. Software on Joy Johnson's computer and the student computer was upgraded to Office/Windows 2000. Susan Canon's computer is scheduled for an upgrade over the summer in conjunction with upgrades in the Provost's Office and associated faculty databases.

Electronic Data Files Inventory. Joy Johnson worked with students over the summer of 2002 to create an electronic inventory of all IRP data and reports, both electronic and paper. She is now in the process of designing and cross-referencing an Excel data base of this inventory.

Professional Development (SC). Susan Canon attended the HEDS summer conference in Chicago as well as the January conference in Santa Fe. Susan serves on the HEDS Technology Committee and was just asked to renew her term for three more years. She also attended the AIRUM fall conference in Bloomington and a Women in Higher Education conference in Minneapolis sponsored by the American Council on Education (ACE). On campus, Susan participated in several information sessions for supervisors as well as two web development workshops offered by the Academic Computing Center.

Professional Development (JJ). Joy Johnson attended two SPSS training courses in July (one week each) and an ACC Excel training workshop.

Statistical Research. A continuing request for additional technical support was partially addressed by an arrangement whereby newly hired faculty statistician Paul Roback would have some time assigned during 2003-04 to work on special assignments for IRP and ARP.

 

G. Looking Ahead

The chief change in the IRP office is that Associate Director Susan Canon was promoted to Director effective June 1, 2003. Lynn Steen, having completed a planned three year term as Director, returns to his former position as Special Assistant to the Provost. In that capacity he will carry out special studies both for the Provost and for the IRP office.

 

In addition to continuing all regular work of the office as outlined above, several projects of significant size and potential benefit to the College remain high on our agenda. Most represent unfulfilled potential--information that has been partially gathered but remains underutilized:

Alumni Outcomes Tracking Database. Create an alumni database that systematically collects information on alumni achievements after graduation (graduate degrees, employment, etc.) and connects that information to St. Olaf data about admissions, demographics, majors, grades, etc. This database could be used as the source of many different investigations, such as (a) an often-requested study of the relation between entering students' major interests, their graduation majors, and their initial post-baccalaureate pursuits and (b) a longitudinal retention study to determine how different factors contribute to students finishing or leaving St. Olaf and, for those who transfer or drop out, whether they complete their studies at other institutions.

Strategic Indicators. Design and implement the use of a quarterly or annual report tracking key strategic indicators about the college including data from admissions, financial aid, advancement, enrollments, graduation, and faculty and administrative staffing. Benchmarking data could be provided on many indicators as a systematic way to monitor the health of the college in various ways, to provide information for planning purposes, and to keep college constituencies well informed.

National Survey of Student Engagement (NSSE). Because NSSE is so strongly associated with monitoring academic rather than institutional characteristics, it is likely that responsibility for conducting and analyzing this increasingly important survey may be transferred to the office of Academic Research and Planning (ARP).

Recent Surveys. The Fall 2001 HERI College Student Survey, the Spring 2002 HEDS Senior Survey, and the Fall 2002 CIRP first-year student survey all need to be analyzed, posted, and reported to the college community. At the same time, it would be valuable to conduct a cross-survey study of those items that are similar across different populations (incoming students, current students, faculty, parents, alumni).

Personnel Reviews. Conduct staff reviews (none have been done in several years).

External Review. It has been six years since the college last conducted an external review of its institutional research activities. Especially with the forthcoming change in leadership in the IRP office, it would be an opportune time to carry out a follow-up review to help establish priorities for services the office could or should be providing.

Web Site Index. As the IRP web site grows in size and complexity, it is important to prepare a site map or index that can help it function effectively as a useful and "user-friendly" data warehouse.

Inventory of External Reports. This project, long in development, needs to be completed and mounted on the IRP web site to provide a single place where all information about reports and responsible offices can be found.

Survey Schedule. For various reasons, including changes made in anticipation of the recent North Central reaccreditation review, the carefully planned IRP survey schedule has been disrupted with various exceptions. Now is a good time to re-examine this schedule to develop a revised plan for the next five years.

Peer Institutions. Persistent questions from the College Council and others suggest a need to either reaffirm or revise the list of peer institutions developed three years ago.