Assessment of learning outcomes should be designed to evaluate the efficiency and effectiveness of student learning outcomes. It involves observations of the characteristics of students, curriculum, programs and units to make informed decisions to guide continuous improvement of the learning process. It should be an inquiry-driven process and performed continuously with an eye on refining the degree assessment process. The assessment process should follow the principles of continuous refinement, implying that it should follow the cycle of observation, refinement, and implementation.
The goal of the assessment process is not just to find opportunities for improvement in a program but also to reinforce its strengths. It should be looked at as a process to improve the overall quality of the program, especially the student experience. Other issues to be addressed include the evolution of the programs, the attempts to seek equity, and the change in students. The assessment of learning outcomes for each degree program should be informed by the overall mission of the unit. Further, course goals should align with the program learning outcomes.
The assessment of student learning outcomes for each degree program should be performed periodically by the faculty involved in the degree program. It is to be expected that each program is evaluated at least once every three years. For programs with a small number of students, the assessment can be performed by assessing the entire population of students, by using a census. Such programs should use the entire population of students to assess each learning outcome. For programs with a large number of students, it may not be feasible to perform assessment of each student and hence, a sampling of the students can be used for assessment. For example, it will be difficult to assess a capstone project/paper for each student in a program with over 200 students. A typical sample size can be at least 10 students or 10% of the student population, whichever is greater.
Other issues affecting the sample size can be the length and complexity of the artifacts (tests, projects, composition) used in assessment, and the number of faculty members who are charged with the task of assessment (the assessment team). Typically, programs with long artifacts (ones that take a substantial time to review) may use a smaller sample size while the programs with short and simple artifacts may use a larger sample size.
Selecting a Sample Size
For each student learning outcome, the assessment team decides whether the entire student population or just a representative sample will be assessed. The team should choose an appropriate sample size taking into account the size and complexity of the artifact being assessed, the student population size, and the faculty panel workload.
Sampling Based on Percentage
For a large program with a limited faculty, the assessment team can decide to use a percentage of overall student population as a representative sample. As an example, for programs with more than 100 students, the team can choose 10% of the student population, with at least 10 students as the sample.
Sampling Based on Artifact Size
Programs that have to assess the learning outcomes from long or complex artifacts may choose a smaller number of students. For example, if the student learning outcome is assessed by a capstone project and paper (senior thesis) with at least 50 pages, it makes sense to select a smaller percentage of students for the assessment. Again, the team should be mindful that there is at least a minimum number of students that have been selected in the sample.
Sampling Based on Faculty Panel Size
Programs with a limited number of faculty members and a large student population need to strike a balance for optimized faculty workload. A small panel of faculty members (say 3) may not be able to read long reports from every student in the program. Similarly, it will be unreasonable to expect the panel to listen to a 10-minute presentation for 200 students. The size of the faculty panel should be used to decide the percentage of student population in the sample.
Sampling Procedure
The representative sample of the student population may be selected by a number of different methods. The common recommendations are described below.
Simple Random Sampling
As the name implies, this method randomly selects a specified number of students from the overall population.
Systematic Sampling
Systematic sampling is slightly different from random sampling. The students can be sorted on some criterion, for example alphabetically based on last name. Then, every nth student in the list is selected in the sample.
Stratified Sampling
In this method, students are divided into homogeneous groups and then, a random number of students are selected from each group. This method can help with equity assessment by selecting underrepresented groups of students. For example, if the program has only a few students from a gender or race, this method can help with the selection of those students for assessment.
Cluster Sampling
Here, the student population is divided into clusters, for example sections of a course. Then, a cluster is randomly selected for assessment.
The above methods for selecting a sample size and representative samples are just recommendations and a program should choose the method that works best for their assessment.
For more information, please refer to this sampling guide as a resource.
