2007-2008 Academic Program Review Guidelines:
Purpose and Process

2007 - 2008 Academic Program Review Guidelines: Purpose and Process (PDF)

The University of Missouri’s Executive Guideline 25 describes a policy for the review of academic units at five-year intervals. Each undergraduate, graduate, professional, certificate, and extension program is reviewed to reinforce a continuous program improvement.

At UM-St. Louis, faculty in each academic unit prepare a Self-Study Report in response to the following questions:

What are the mission and goals of the unit and how do they support the UM-St. Louis Mission and Action Plan?
What evidence have faculty analyzed to assure that each program has met its goals?
What are the programs’ goals for the next five years?
What strategies are in place for the unit to assess, review, and revise its goals annually?

The review process consists of the following steps:

The department conducts a self-study and submits a report to the Office of Academic Affairs.
A Campus Review Team is appointed consisting of three senior faculty representatives (including colleagues recommended by the unit) and an external reviewer; it may also include a representative from the Office of Academic Affairs.
Academic Affairs forwards the Self-Study Report to the Campus Review Team and academic leadership (dean, vice-provosts, associate provosts, provost).
The Campus Review Team conducts a one-day site visit to verify the information in the Self-Study Report.
The external reviewer and faculty team members submit separate reports with recommendations to the unit and the Provost. The unit may respond to the reports.
The dean meets with each chair to discuss the reports and review the department’s goals; they may invite the provost and/or other academic officers.
The chair submits an Executive Summary of the review to Academic Affairs; after approval it is forwarded to the UM Office of Academic Affairs.

The Self-Study Report

To promote continuous program improvement, the University of Missouri Office of Academic Affairs recommends that units consider the quality principles identified by William Massy in Honoring the Trust: Quality and Cost Containment in Higher Education (Anker Pub., 2003). A summary of these is included in the appendix.

During the self-study process faculty identify appropriate measures of quality, apply those measures to evaluate current programs, and use that data to determine program revisions. Preparing the Self-Study Report provides an opportunity to assess the department’s strengths and weaknesses, consider the consistency of your goals with those of the campus and the University, and apply measures to evaluate progress toward your goals for teaching, research, and service.

Effective self-studies rely upon evidence from the past to improve future performance. They describe the unit’s current operations and changes since the last review by presenting evidence that evaluates the mission, instructional programs, use of resources, faculty productivity, administrative organization and governance, attention to and engagement with students, and external funding. Evidence may be conveyed with a) direct assessments such as pre/post tests, portfolio evaluations of student and faculty products, discipline-specific performance assessments, or b) indirect assessments obtained from surveys, questionnaires, or focus groups of alumni and employers. In short, the self-study process includes a review of all the major processes carried out in the unit and their effect on student learning.

Five-year institutional data are available from Institutional Research including:

Degrees awarded by college, department and level.
Degrees awarded by college, department, degree program, and level, including BSEd degrees by emphasis area.
Enrollment by college, department, course level, term, and fiscal year.
Student credit hours by college, department, course level, term, and fiscal year.
Majors by term, college, department, and degree level.
Department expenditures by general revenue, instruction, and research.
Grade data by term, fiscal year, college, department, and course level.
The most recent Delaware data.

The five-year review web site also has research expenditure data based on the accounting records, but please contact the Office of Research Administration for award data. Data on budgets can be obtained from senior fiscal officers.

For comparative purposes, it is useful to identify a similar program on a comparator or aspirational campus and benchmark your outcomes with theirs. Studying other institutions may also guide the unit to consider alternative sources of evidence, identify innovative curricular and learning strategies, or parlay its own strengths into alternative goals and aspirations.

Organizing the Self-Study Report

I. Overview

Introduce the unit by beginning with one or two paragraphs that include:

A brief history of the academic unit;
The unit’s mission and goals;
The unit’s facilities, faculty, and staff;
How the unit contributes to the campus’ identity regionally and nationally;
Academic programs that comprise the unit; and
A brief appraisal of how the programs have operated, changed, or developed since the last five-year Program Review. Be sure to include the work already completed for UMSL’s re-accreditation process.

II. Assessment of Academic Programs

The questions below may help faculty focus their inquiries as the self-study proceeds, but they are not intended to limit the scope of the process or the report. How this information is presented may be unique to each department.

A. The Departmental Context
The Self-Study Report should begin with summary information (with details in the appendices) about the department; consider answers to the following questions:

Is the expertise, size, diversity, and research of the faculty consistent with and does it support the programs’ goals?
How are faculty successes in scholarly and service activities evaluated (annually, pre-tenure review, promotion and tenure, post–tenure review)?
How does the department foster scholarship among the faculty (incentives and support) and evaluate the effectiveness of the support?
How would you characterize the quantity and quality of faculty members’ scholarly activities for the five-year period?
Are you satisfied with the five-year record of securing grants, contracts, and gifts and how these efforts reflect scholarship within the discipline?
How do you evaluate and improve teaching for full-time and part-time faculty, teaching assistants, and undergraduate tutors?
How effectively do you use information technology (including My Gateway) instructionally, administratively, and in research? What evidence do you have that any online instruction is effective?
How do faculty regard their role as contributors to the department, campus, metropolitan area, and professional organizations? Explain how faculty research informs those activities.
Does the administrative structure and policy-making processes meet the department’s goals.

B. Programs
The Self-Study Report must document how learning goals are reviewed for each program, identify the evidence that the goals are being met, and show how this information is routinely used in program and curricular planning.
For each undergraduate and graduate degree and graduate certificate program the report should address:

The mission, goals, and expected learning outcomes of the program.
How the program is designed so that courses build on each other to meet the program goals.
How the undergraduate curriculum builds upon and/or includes general education.
How the undergraduate programs and graduate programs are related and what distinguishes them. When courses meet both programs’ goals, how do student outcomes and assessments differ?
How the program assesses and builds on the prior knowledge of incoming students.
How students are recruited, engaged, and retained in the program. Explain how you’ve handled classes or programs with a large dropout rate.
How instructional activities, including classrooms, labs, research and creative projects, service-learning, community-based activities, practica and internships, and study abroad support learning outcomes and meet the goals of the program.
How co-curricular activities in or outside of the unit support student learning and engagement.
How the coursework builds to a synthesizing experience such as a senior project, capstone experience, thesis/dissertation, or other exit requirement; provide evidence of the effectiveness of the experience.
The effectiveness of the advising system for students; explain how the unit’s advising responsibilities are integrated into the program and address how the undergraduate program meets the unique needs of transfer students. Include a review of websites and handbooks to assure that correct and consistent information is available to students and advisors.
How student progress is assessed; identify the procedures and timelines used to provide feedback to students. How does the program ensure that students’ knowledge develops to meet the program outcomes? What percentage of graduates meets all learning outcomes?
How the unit relies upon reviews of students’ products (capstone experiences, theses, exit tests, etc.), course evaluations, and employer surveys to make programmatic changes.
An analysis of the unit’s resources (personnel, facilities, scholarships, assistantships) that support the program;
How the program’s outcomes compare to programs on comparable or aspirational campuses;
Include, when relevant, the most recent professional accreditation/certification date, summarize the recommendations made by the accrediting body, and describe how the program is addressing concerns and tracking progress.

III. Five-year Plan

Prioritize the unit’s goals and outcomes for the next five years.

Describe implementation strategies to attain the goals. Include a timeline and assign responsibilities.
Identify how progress toward the outcomes is assessed and reviewed and revised (if necessary) annually.
Describe any costs or savings expected with the planned changes.

IV. Appendices or Displays

During the Site Visit the Campus Review Team may want to review supporting materials in binders. Appendices may include but are not limited to:

Current curricula vitae for all faculty;
Most recent syllabus for each course in each program.

Preparing the Academic Program Review Materials for Submission

Limit the Self-Study Report to 25 pages by focusing on the topics outlined above that are most salient for the department.
Add a cover page identifying the department, college, and date of the site review.
Include a table of contents to identify sections and number the pages consecutively.
Incorporate relevant supporting materials as appendices in separate files.
Please submit all materials as electronic files (Word Documents) no later than March 3, 2008 (or four weeks prior to the Site Visit), to Associate Provost Judith Walker de Felix (jwdfelix@umsl.edu) and to the Dean of the College. She will post the Self-Study Report and supporting materials on the Accreditation Website to make them available to members of the campus community:

An Executive Summary based on the Self-Study Report and Reports of the External Reviewer, and the Campus Review Team is due in the Office of Academic Affairs by August 25, 2008. Formats for these reports follow.

UM-St. Louis Program Review

Guidelines for Reports from the External Reviewer and the Campus Review Team

Please submit all reports electronically to Associate Provost Judith Walker de Felix.

Suggested Format for the report from the External Reviewer:

The report is written based on:

The unit’s Self-Study Report;
Meetings with faculty, staff, students, and partners affiliated with the unit;
Meetings with the Campus Review Team;
Meetings with the Dean(s) and members of the Provost’s Office.

Include attention to these and other issues noted during the campus visit:

The evidence provided to document the programs’ effectiveness;
The extent to which the programs are focused on national trends;
The degree to which the unit’s future goals are consistent with trends in the field;
The choice of department selected for comparative purposes: please suggest other comparable or more comparable programs if necessary;
Recommendations about programs, resources, curriculum, and internal operations.

Your report:

Has a flexible format adapted to the unit under review;
May be limited to five pages;
Identifies the Reviewer and his/her academic affiliations;
Should be submitted within two weeks of the site visit.

Suggested Format for the report from the Campus Review Team:

This report is written based on:

The Self-Study Report;
Meetings with those in the department, institute, or center and its affiliates;
Meetings with the external reviewer;
The Report from the external reviewer.

The report should include:

An explanation of the procedures followed by the Campus Review Team;
An evaluation of the evidence included in the Self-Study Report;
The strengths and weaknesses of the unit’s program(s);
Recommendations about programs, resources, curriculum, and internal operations.

Your report:

Has a flexible format adapted to the unit being reviewed;
May be limited to five pages;
Includes the names and campus affiliations of the Campus Review Team;
Should be submitted within two weeks of receiving the report of the external reviewer.

2007 – 2008 Executive Summary
Academic Program Reviews of UM-St. Louis Departments and Centers*

Units prepare and submit an executive summary of the Academic Program Review by August 25, 2008. This summary may include elements of observations and recommendations from the Campus Review Team and external reviewer and meetings with academic leaders.

Unit:

College:

Identify the mission of the unit and each of its programs:

Summarize the strengths and distinguishing characteristics of the programs:

Address how continuous improvement efforts ensure curricular quality:

Describe procedures in place that provide evidence of learning outcomes, and assess how those procedures are responsive to trends in student enrollments, degree completions, employment and graduate school placements:

Summarize faculty productivity in scholarship and professional and community service:

Identify program accreditations and dates (if any):

Summarize the long-range goals and strategies included in the Five-Year Plan:

This summary was revised in March 2003 and November 2006 to reflect changes in Executive Guideline 25 (see 20.035 Program Assessment and Viability Audit in the UM Collected Rules and Regulations).

*For uniformity, please limit the executive summary to 3 single-spaced pages written in the third person and use Times New Roman 12-pitch font. Submit electronically as a Word document to Associate Provost Judith Walker de Felix.

Appendix

Quality Principles

a) Define quality in terms of outcomes. Exemplary units define outcomes related to the unit's mission and then focus their efforts on achieving those outcomes. Does your unit have a mission statement? Do you consider it annually? What outcomes are important for the unit to achieve and how does this unit compare to others? That is, how good are your programs and your research? How good is the service the unit provides?

b) Focus on how things get done. Exemplary units think carefully about how things are done. They search out impediments to achieving their goals and mitigate them to the extent possible.

c) Work collaboratively. Exemplary units demonstrate collegiality in their efforts to support teaching and research. Members share information and help one another solve difficult problems because such teamwork makes the unit a "learning organization."

d) Base decisions on evidence. Exemplary units define indicators to track the quality of their programs and services. Exemplary units review the literature and consult with outside experts to identify trends in the field, and then ask how the trends are likely to impact their missions. They marshal facts about their performance relative to peer units and use these facts to develop realistic goals and strategies

e) Strive for coherence. Exemplary units’ programs and services are consistent with the abilities and interests of the faculty and are compatible with campus goals. They make good use of interdisciplinary resources and view learning through the lens of the participants' entire experience. Exemplary programs build upon one another to provide the desired depth and breadth.

f) Learn from best practice. Exemplary units identify and analyze good practices in comparable units and institutions and then adapt the best to their own circumstances. They compare good versus average or poor‑performing practices within their own department, assess the causes of the differences, and seek ways to improve the lesser performers.

g) Make continuous improvement a priority. Exemplary units strive to improve the quality of their efforts on a regular basis. They work with related units to determine how they can improve their services and foster education and scholarship on campus. They continuously check to see if their efforts match institutional priorities.

Source: William Massy (2003). Honoring the Trust: Quality and Cost Containment in Higher Education (Bolton, MA: Anker Pub.)

2007-2008 Academic Program Review Guidelines: Purpose and Process