Office of Academic Affairs

2005- 2006 Review Guidelines for Administrative Units

Policy

The University of Missouri’s Executive Guideline 25 describes a policy for the review of academic units at five-year intervals.  The policy requires an examination of the activities of undergraduate, graduate, professional, and extension programs with attention given to teaching, research, and creative activities. The policy states that recommendations from the program reviews should be considered in developing operating and capital budget requests and in planning for reallocations when necessary and appropriate.

Specifically, the purposes of the program review processes are:

  1. To initiate a continuous improvement model as units engage in self-study with the goal of improving its programs;
  2. To support long-range planning for the unit and the institution.

At UM-St. Louis administrative units are required to participate in the five-year review process as well.  Each unit prepares a self-study document, which is distributed to members of a Campus Review Team. The Team includes three representatives (including one or more colleagues recommended by the unit) and an external reviewer from another institution. An administrative representative may also participate. The reviewers conduct a one-day site visit and prepare a report that is submitted to the unit and to the Chancellor. The unit may submit a written reply in response to the reviewers’ report. These documents provide the discussion points for a meeting of the leader of the unit with the Chancellor and other administrative officers as appropriate. The unit then updates the report each year until the next five-year review.

The Review

Introduction to Quality Processes

“Academic Quality Processes” can be defined as follows (adapted from Massy, 2003a, p.  159): 
Academic quality processes are organized activities dedicated to improving and assuring education and research quality. They systematize a university’s approach to quality instead of leaving it mainly to unmonitored individual initiative. They provide what higher education quality pioneers David Dill and  Frans van Vught call “...a framework for quality management in higher  education...drawn from insights in Deming’s approach, but grounded in the  context of academic operations” (van Vught 1994, Dill 1992).  

Academic quality processes should not be confused with the unit’s normal activities. One might say that quality processes plan and govern the improvement and monitoring of the daily activities.  

Focal Areas 
Quality processes are applied to well-defined focal areas. These include the unit’s normal activities as well as special initiatives such as those identified in the campus’ Action Plan or System’s Strategic Initiative. Staff discuss and agree on no more than 10 programs or initiatives that the unit will focus on over the next several years. The Focal Areas, then, become the targets for the unit’s quality improvement efforts.

The Improvement Cycle 
Improving performance in any Focal Area requires serious reflection followed by a commitment to improvement. Asking how well one’s current activities are aligned with the quality principles below is a good first step. Good alignment provides cause for celebration and examples of good practice that can be applied elsewhere. Poor alignment presents an opportunity for improvement.  

It’s often effective to begin by reflecting on processes where performance is believed to be good. Celebrations of success build confidence in one’s command of the principles and their application, and comparing poorly aligned processes with one’s own best practice can help motivate improvement.  Some find the list of principles a bit daunting at first. While many of the ideas are familiar, others aren’t. The following questions may help one apply the quality principles in a practical way.  

What are we trying to do?  Traditional thinking equates quality to inputs like size and qualifications of staff—e.g., “Quality improvement requires more and better staff members.”  But inputs are means to ends and it’s the ends—what one is trying to accomplish—that ultimately matter. This leads to the first principle: Define quality in terms of outcomes

How are we doing it?  Quality gurus say, “All work is process.”  In other words, to do something you must engage in some kind of process even if it’s ad hoc. It stands to reason, then, that paying attention to process can improve quality. In other words:  Focus on how things get done. 

Who is responsible for doing it?  Tasks can be assigned to teams or left to individual initiative. While individual initiatives are salutary, teams usually outperform “lone wolves” when it comes to sustaining and improving quality. Hence the principle: Work collaboratively

How do we know we’re succeeding?  It’s hard to consistently produce quality without feedback on how you’re doing. The feedback should be rooted in evidence rather than anecdote, and evidence is important in applying the other six principles as well. Therefore:  Base decisions on evidence.

Applying Quality Principles in the Self-Study

Start your self-study by discussing with staff the quality principles related to your Unit

a) Define quality in terms of outcomes
Desired outcomes relate to the unit's mission, not some abstract notion of mission. Exemplary units define the kinds of outcomes they want and then focus their efforts on achieving those outcomes. Do you have a mission statement for the unit? Do you consider it on an annual basis? What outcomes are important for the unit to achieve and how do those outcomes compare to peer institutions? How good are your programs? How good is the service that the unit provides?

b) Focus on how things get done
Excellent units think about how things are done and how they can support integrated processes. They search out impediments to achieving their goals and mitigate them to the extent possible.

c) Work collaboratively
Outstanding units demonstrate collegiality in their efforts to support the campus’ mission. Members share information and help one another solve difficult problems because such teamwork makes the unit a learning organization.

d) Base decisions on evidence
What indicators do you use to track the quality of your services? Who are your peer institutions and what are you doing that is similar and unique to peer institutions? What peer institutions can you reasonably expect to catch up with? Exemplary units review the literature and consult with outside experts to identify trends and then ask how the trends are likely to impact their focal areas. They marshal facts about their performance relative to peer units and use these facts to develop realistic goals and strategies

e) Strive for coherence
Are your services consistent with the abilities and interests of campus personnel? Are they compatible with campus goals? Are they making good use of resources? Ideal units view services through the lens of stakeholders’ entire experience. Resulting initiatives build upon one another to provide the desired depth and breadth.

f) Learn from best practice
With their staff, leaders analyze good practices in comparable units and institutions and then adapt the best to their own circumstances. They compare good versus average or poor‑performing practices within their own units, assess the causes of the differences, and seek ways to improve where necessary.

g) Make continuous improvement a priority
Commendable units strive to improve the quality of their efforts on a regular basis. The unit works with other related units and stakeholders to determine how they can improve their service to them. They continuously check to see if their efforts match institutional priorities.

Draft Self-Study Template for Administrative Units

I. Describe Quality Processes
Use this section to discuss the processes that the unit uses to carry out its mission and evaluate its activities. You may use the following sections or organize your description in a way that better meets your needs:
A. Mission, objectives, and future direction
Items reviewers may expect:
Do you have a mission statement that you consider annually? What outcomes are important for the Unit to achieve and how do those outcomes compare to similar organizations in other universities? How does the Unit decide its priorities? Are the Unit’s processes well thought out and based on clear objectives, matching staff expertise? How does the Unit collaborate with other parts of the university? How do you regularly consider quality assurance?
B. Supporting constituents
Items reviewers may expect:
How do you make policies regarding stakeholders? How involved are staff in policy decisions? Are the Unit’s administrative processes "user friendly"? To what degree are the Unit's policies coherent; aligned with its mission and objectives; well understood by faculty, staff, and students; and acted on effectively?
C. Evaluating and stimulating improvement of services
Items reviewers may expect:
How are reviews of programs or projects conducted? Are these reviews achieving their purpose?
D. Relation to campus’ goals
Items reviewers may expect:
What are your goals in recruiting constituents? How did you select the goal(s)? How does your process compare to your peer institutions? What are relevant statistics related to your programming and services? How do these compare to national benchmarks? What reflections do you have on these data, and how have these reflections influenced policy decisions? Is there follow‑up with individual participants to monitor progress toward their goals, including interventions if necessary?
E. Fostering the campus’ mission
Items reviewers may expect:
What efforts are underway to support the campus’ mission? What indicators document that those efforts are having the anticipated outcome? What is the process for determining those outcomes? How does the Unit respond to the campus Action Plan and other campus initiatives?

II. Improvement Cycle
Describe what the unit will do to maintain its strengths and enhance areas that need strengthening. In the self-study, you may use the four questions on page 2 or the following categories:
A. Overall Performance 
How does the unit gauge the overall quality of its activities? What criteria and evidence are used and why? Does the unit distinguish between immediate- and long-term performance indicators and if so, how? [2 pages]
B.  Performance in the Focal Areas 
How does the unit’s work in each focal area measure up against the quality and evidentiary principles? What practices does it consider to be exemplary and what does it judge as needing improvement? Why?  (As suggested above, it may prove fruitful to identify exemplary practices at the start, and then determine whether/how the principles contributed to these successes and how the lessons learned can be applied more broadly.) [10 pages] 
C. Potential Initiatives 
What initiatives might the unit take to address practices that need improvement? What would each initiative be expected to accomplish? What actions would be required and how long would they take? Who would be responsible for these actions? How would the initiative’s success be gauged? If the initiative proved successful initially, how would the unit refine it to work better in the future? [~3 pages] 
D.  Commitment to Improvement 
Which initiative does the department consider to have the highest priority?  Are staff prepared to work on it? If so, who will participate and who will take responsibility? What if any additional resources and/or cooperation from people outside the department will be required? Why does the unit believe these will be forthcoming? What additional initiatives, if any, are being considered for early action? [~ 1 page]  

Submission of the Five-Year Review Materials

Academic Affairs will facilitate the reviews for the Chancellor. Each review consists of a self-study, external review, and an opportunity for the unit to respond to the external reviews. Copies of the documents are maintained for the accreditation review. The Chancellor may also use the reviews for his planning and evaluation purposes.

The Self-Study should be completed and submitted by June 6, 2006, unless prior arrangements have been made.

  • Present a cohesive and coherent report based on discussions of the topics that are most salient for the unit.  Relevant and referenced supporting materials may be included in appendices.
  • Number the pages, include a table of contents, and identify sections; prepare a cover sheet identifying the unit and date.
  • An electronic copy may be submitted to the accreditation workspace. Please contact your Vice Chancellor or Judith Walker de Felix for access to the site. Alternatively, you may submit 9 copies to the Office of Academic Affairs and one to the Chancellor.

Please notify Academic Affairs of your suggestions for the membership of the external review team by May 1, 2006. The campus members should include one faculty member with expertise in your field plus two staff (or additional faculty) members with an interest in your unit. The external reviewer should come from a UM campus or a comparator institution.

Resources for Further Reference

Barnett, R. (1992). Improving higher education: Total quality care. London, England: The Society for Research Into Higher Education and The Open University Press. 

Dill, D. D. (1992). Quality by design: Toward a framework for academic quality management. In J. Smart (Ed.), Higher education: Handbook of theory and research (pp.  37-83). New York, NY: Agathon Press. 

ECS (2003). Accountability – Next Generation Models: Quality Improvement Model. Denver, CO: Education Commission of the States. Proposes to apply education quality processes and academic process review to k-12 education. 
JHHEG (2004). Frequently asked questions on academic process review. The Jackson Hole Higher Education Group, Inc.* 

Massy, W. F. (2000). Energizing quality work: Higher education quality evaluation in Sweden and Denmark. Technical Report: National Center for Postsecondary Improvement, Stanford University.  The Jackson Hole Higher Education Group, Inc.

Massy, W. F. (2003a). Honoring the trust: Quality and cost containment in higher  education. Bolton, MA: Anker Publishing Company (Chapters 6-8).* 

Massy, W. F. (2003b), Access to what? Putting “Quality” Into National QA Systems.  Keynote address on Quality and Standards: The National Perspective” to the 2003 conference of the International Network of Quality Assurance Agencies in Higher Education  (INQAAHE), Dublin, Ireland (April 14-17). 

Massy, W. F. (2003c), Auditing higher education to improve quality. Chronicle of Higher Education (June 20).*  

Massy, W. F. (2004), Academic audit for accountability and improvement. In J. C. Burke (Ed.), The Many Faces of Accountability: Holding Higher Education Responsible for Performance. Chapter 9San Francisco: Jossey-Bass, forthcoming.* 

Massy, W. F., and French, N. J. (2001). Teaching and learning quality process review:  What the program has achieved in Hong Kong. Quality in Higher Education, 7(1)  (April), 33-45. 

TLQPR Review Team (1999). A Campaign for quality: Hong Kong teaching and learning  quality process review. Hong Kong: University Grants Committee of Hong Kong. 

van Vught, F. (1994).The new context for academic quality. Enschede, Holland:  University of Twente, Center for Higher Education Policy Studies. Paper for Symposium:  “University and Society”, Vienna (June 9-10). 

University of Missouri (2005). Academic quality improvement: A handbook for departments. Columbia, MO: Self.

WASC (2002). Evidence guide; A guide to using evidence in the accreditation process: A  resource to support institutions and evaluation teams. Alameda, CA: Accrediting Commission for Senior Colleges & Universities, Western Association of Schools and  Colleges (available on the Web).

 *   An asterisk indicates introductory materials on academic process review. Note that academic process review may be referred to as “academic audit” or (in Hong Kong) as “Teaching and Learning Quality Process Review.”