Office of Academic Affairs
Policies and Documents
Office of Academic Integrity
Colleges and Schools
Centers and Institutes
Academic Support Units
Information About UMSL Faculty
Information For UMSL Faculty
Information About UMSL Students
Information For UMSL Students
Assessment at UMSL
Five Year Program Reviews
Assessment of Faculty
Student Assessments and Processes
The University of Missouri's Executive Guideline 25 describes a policy for the review of academic units at five-year intervals. The policy requires an examination of the activities of undergraduate, graduate, professional, and extension programs with attention given to teaching, research, and creative activities. The policy states that recommendations from the program reviews should be considered in developing operating and capital budget requests and in planning for reallocations when necessary and appropriate.
Specifically, the purposes of the program review processes are:
- To initiate a continuous improvement model as units engage in self-study with the goal of improving its programs;
- To support long-range planning for the unit and the institution.
At UMSL, administrative units are required to participate in the five-year review process as well. Each unit prepares a self-study document, which is distributed to members of a Campus Review Team. The Team includes three representatives (including one or more colleagues recommended by the unit) and an external reviewer from another institution. An administrative representative may also participate. The reviewers conduct a one-day site visit and prepare a report that is submitted to the unit and to the Chancellor. The unit may submit a written reply in response to the reviewers report. These documents provide the discussion points for a meeting of the leader of the unit with the Chancellor and other administrative officers as appropriate. The unit then updates the report each year until the next five-year review.
Introduction to Quality Processes
Academic Quality Processes can be defined as follows (adapted from Massy, 2003a, p. 159):
Academic quality processes are organized activities dedicated to improving and assuring education and research quality. They systematize a university's approach to quality instead of leaving it mainly to unmonitored individual initiative. They provide what higher education quality pioneers David Dill and Frans van Vught call ...a framework for quality management in higher education...drawn from insights in Demings approach, but grounded in the context of academic operations (van Vught 1994, Dill 1992).
Academic quality processes should not be confused with the units normal activities. One might say that quality processes plan and govern the improvement and monitoring of the daily activities.
Quality processes are applied to well-defined focal areas. These include the units normal activities as well as special initiatives such as those identified in the campus Action Plan or Systems Strategic Initiative. Staff discuss and agree on no more than 10 programs or initiatives that the unit will focus on over the next several years. The focal areas, then, become the targets for the units quality improvement efforts.
The Improvement Cycle
Improving performance in any focal area requires serious reflection followed by a commitment to improvement. Asking how well ones current activities are aligned with the quality principles below is a good first step. Good alignment provides cause for celebration and examples of good practice that can be applied elsewhere. Poor alignment presents an opportunity for improvement.
Its often effective to begin by reflecting on processes where performance is believed to be good. Celebrations of success build confidence in ones command of the principles and their application, and comparing poorly aligned processes with ones own best practice can help motivate improvement. Some find the list of principles a bit daunting at first. While many of the ideas are familiar, others arent. The following questions may help one apply the quality principles in a practical way.
What are we trying to do? Traditional thinking equates quality to inputs like size and qualifications of staff-e.g., "Quality improvement requires more and better staff members." But inputs are means to ends and its the endswhat one is trying to accomplishthat ultimately matter. This leads to the first principle: Define quality in terms of outcomes.
How are we doing it? Quality gurus say, "All work is process." In other words, to do something you must engage in some kind of process even if its ad hoc. It stands to reason, then, that paying attention to process can improve quality. In other words: Focus on how things get done.
Who is responsible for doing it? Tasks can be assigned to teams or left to individual initiative. While individual initiatives are salutary, teams usually outperform "lone wolves" when it comes to sustaining and improving quality. Hence the principle: Work collaboratively.
How do we know were succeeding? Its hard to consistently produce quality without feedback on how youre doing. The feedback should be rooted in evidence rather than anecdote, and evidence is important in applying the other six principles as well. Therefore: Base decisions on evidence.
Applying Quality Principles in the Self-Study
Start your self-study by discussing with staff the quality principles related to your unit
a) Define quality in terms of outcomes
Desired outcomes relate to the unit's mission, not some abstract notion of mission. Exemplary units define the kinds of outcomes they want and then focus their efforts on achieving those outcomes. Do you have a mission statement for the unit? Do you consider it on an annual basis? What outcomes are important for the unit to achieve and how do those outcomes compare to peer institutions? How good are your programs? How good is the service that the unit provides?
b) Focus on how things get done
Excellent units think about how things are done and how they can support integrated processes. They search out impediments to achieving their goals and mitigate them to the extent possible.
c) Work collaboratively
Outstanding units demonstrate collegiality in their efforts to support the campus mission. Members share information and help one another solve difficult problems because such teamwork makes the unit a learning organization.
d) Base decisions on evidence
What indicators do you use to track the quality of your services? Who are your peer institutions and what are you doing that is similar and unique to peer institutions? What peer institutions can you reasonably expect to catch up with? Exemplary units review the literature and consult with outside experts to identify trends and then ask how the trends are likely to impact their focal areas. They marshal facts about their performance relative to peer units and use these facts to develop realistic goals and strategies
e) Strive for coherence
Are your services consistent with the abilities and interests of campus personnel? Are they compatible with campus goals? Are they making good use of resources? Ideal units view services through the lens of stakeholders entire experience. Resulting initiatives build upon one another to provide the desired depth and breadth.
f) Learn from best practice
With their staff, leaders analyze good practices in comparable units and institutions and then adapt the best to their own circumstances. They compare good versus average or poor‑performing practices within their own units, assess the causes of the differences, and seek ways to improve where necessary.
g) Make continuous improvement a priority
Commendable units strive to improve the quality of their efforts on a regular basis. The unit works with other related units and stakeholders to determine how they can improve their service to them. They continuously check to see if their efforts match institutional priorities.
Draft Self-Study Template for Administrative Units
I. Describe Quality Processes
Use this section to discuss the processes that the unit uses to carry out its mission and evaluate its activities. You may use the following sections or organize your description in a way that better meets your needs:
A. Mission, objectives, and future direction
Items reviewers may expect:
Do you have a mission statement that you consider annually? What outcomes are important for the unit to achieve and how do those outcomes compare to similar organizations in other universities? How does the Unit decide its priorities? Are the units processes well thought out and based on clear objectives, matching staff expertise? How does the unit collaborate with other parts of the university? How do you regularly consider quality assurance?
B. Supporting constituents
Items reviewers may expect:
How do you make policies regarding stakeholders? How involved are staff in policy decisions? Are the units administrative processes "user friendly"? To what degree are the unit's policies coherent; aligned with its mission and objectives; well understood by faculty, staff, and students; and acted on effectively?
C. Evaluating and stimulating improvement of services
Items reviewers may expect:
How are reviews of programs or projects conducted? Are these reviews achieving their purpose?
D. Relation to campus goals
Items reviewers may expect:
What are your goals in recruiting constituents? How did you select the goal(s)? How does your process compare to your peer institutions? What are relevant statistics related to your programming and services? How do these compare to national benchmarks? What reflections do you have on these data, and how have these reflections influenced policy decisions? Is there follow‑up with individual participants to monitor progress toward their goals, including interventions if necessary?
E. Fostering the campus mission
Items reviewers may expect:
What efforts are underway to support the campus mission? What indicators document that those efforts are having the anticipated outcome? What is the process for determining those outcomes? How does the unit respond to the campus action plan and other campus initiatives?
II. Improvement Cycle
Describe what the unit will do to maintain its strengths and enhance areas that need strengthening. In the self-study, you may use the four questions on page 2 or the following categories:
A. Overall Performance
How does the unit gauge the overall quality of its activities? What criteria and evidence are used and why? Does the unit distinguish between immediate- and long-term performance indicators and if so, how? [2 pages]
B. Performance in the Focal Areas
How does the units work in each focal area measure up against the quality and evidentiary principles? What practices does it consider to be exemplary and what does it judge as needing improvement? Why? (As suggested above, it may prove fruitful to identify exemplary practices at the start, and then determine whether/how the principles contributed to these successes and how the lessons learned can be applied more broadly.) [10 pages]
C. Potential Initiatives
What initiatives might the unit take to address practices that need improvement? What would each initiative be expected to accomplish? What actions would be required and how long would they take? Who would be responsible for these actions? How would the initiatives success be gauged? If the initiative proved successful initially, how would the unit refine it to work better in the future? [~3 pages]
D. Commitment to Improvement
Which initiative does the department consider to have the highest priority? Are staff prepared to work on it? If so, who will participate and who will take responsibility? What if any additional resources and/or cooperation from people outside the department will be required? Why does the unit believe these will be forthcoming? What additional initiatives, if any, are being considered for early action? [~ 1 page]
Submission of the Five-Year Review Materials
Academic Affairs will facilitate the reviews for the Chancellor. Each review consists of a self-study, external review, and an opportunity for the unit to respond to the external reviews. Copies of the documents are maintained for the accreditation review. The Chancellor may also use the reviews for his planning and evaluation purposes.
The Self-Study should be completed and submitted by June 6, 2006, unless prior arrangements have been made.
- Present a cohesive and coherent report based on discussions of the topics that are most salient for the unit. Relevant and referenced supporting materials may be included in appendices.
- Number the pages, include a table of contents, and identify sections; prepare a cover sheet identifying the unit and date.
- An electronic copy may be submitted to the accreditation workspace. Please contact your Vice Chancellor or Judith Walker de Felix for access to the site. Alternatively, you may submit 9 copies to the Office of Academic Affairs and one to the Chancellor.
Please notify Academic Affairs of your suggestions for the membership of the external review team by May 1, 2006. The campus members should include one faculty member with expertise in your field plus two staff (or additional faculty) members with an interest in your unit. The external reviewer should come from a UM campus or a comparator institution.
Resources for Further Reference
Barnett, R. (1992). Improving higher education: Total quality care. London, England: The Society for Research Into Higher Education and The Open University Press.
Dill, D. D. (1992). Quality by design: Toward a framework for academic quality management. In J. Smart (Ed.), Higher education: Handbook of theory and research (pp. 37-83). New York, NY: Agathon Press.
ECS (2003). Accountability Next Generation Models: Quality Improvement Model. Denver, CO: Education Commission of the States. Proposes to apply education quality processes and academic process review to k-12 education.
JHHEG (2004). Frequently asked questions on academic process review. The Jackson Hole Higher Education Group, Inc.*
Massy, W. F. (2000). Energizing quality work: Higher education quality evaluation in Sweden and Denmark. Technical Report: National Center for Postsecondary Improvement, Stanford University. The Jackson Hole Higher Education Group, Inc.
Massy, W. F. (2003a). Honoring the trust: Quality and cost containment in higher education. Bolton, MA: Anker Publishing Company (Chapters 6-8).*
Massy, W. F. (2003b), Access to what? Putting Quality Into National QA Systems. Keynote address on Quality and Standards: The National Perspective to the 2003 conference of the International Network of Quality Assurance Agencies in Higher Education (INQAAHE), Dublin, Ireland (April 14-17).
Massy, W. F. (2003c), Auditing higher education to improve quality. Chronicle of Higher Education (June 20).*
Massy, W. F. (2004), Academic audit for accountability and improvement. In J. C. Burke (Ed.), The Many Faces of Accountability: Holding Higher Education Responsible for Performance. Chapter 9San Francisco: Jossey-Bass, forthcoming.*
Massy, W. F., and French, N. J. (2001). Teaching and learning quality process review: What the program has achieved in Hong Kong. Quality in Higher Education, 7(1) (April), 33-45.
TLQPR Review Team (1999). A Campaign for quality: Hong Kong teaching and learning quality process review. Hong Kong: University Grants Committee of Hong Kong.
van Vught, F. (1994).The new context for academic quality. Enschede, Holland: University of Twente, Center for Higher Education Policy Studies. Paper for Symposium: University and Society, Vienna (June 9-10).
University of Missouri (2005). Academic quality improvement: A handbook for departments. Columbia, MO: Self.
WASC (2002). Evidence guide; A guide to using evidence in the accreditation process: A resource to support institutions and evaluation teams. Alameda, CA: Accrediting Commission for Senior Colleges & Universities, Western Association of Schools and Colleges (available on the Web).
* An asterisk indicates introductory materials on academic process review. Note that academic process review may be referred to as academic audit or (in Hong Kong) as Teaching and Learning Quality Process Review.