December 2002 |
|
Vol. 16 : No.
12< > |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
* A plus sign (+) indicates a possible combination whereas a minus sign (–) indicates no possible combination. We do not include asynchronous (e.g. cassette tapes, self-paced manuals) and synchronic (e.g. teleconferencing) distance learning resources that are not computer-mediated in this typology. We do not claim that the findings reported here are relevant to choosing among learning management systems. Rather, our purpose is more limited in scope, and analytic in nature. This paper reports on a study of student class participation, satisfaction, and communication with the teacher in one specific type of blended learning in a university class. Our analysis considers both access (the primary location where students access MyGateway) and use (low use or high use) of the learning portal (MyGateway) as key independent variables in predicting learner satisfaction, participation, and communication within distance learning classes. We focus on access because of the longstanding assumption that one of distance education’s most promising features is the way it makes learning resources flexible and available to learners regardless of location (Valore & Diehl, 1987). Access has always been an issue for distance learning, from early reliance on the postal system, distributing books, self-paced manuals and other learning materials like audio-cassettes or albums, to later reliance on the general availability of analog technologies like television and telephony. In each case, access to learning resources was important. The same concern for access to learning resources appears relevant to interactive video classes, especially those integrating learning management system portals such as MyGateway into the curriculum. For instance, roughly equal access (between the host and remote sites) to studio classrooms equipped with interactive video feeds does not ensure equal access to Internet connectivity, especially in rural areas (Irons, Jung, & Keel, 2002). On the other hand, computer-mediated communication via the Internet can also provide a base for engaged students, even at remote sites, to increase their sense of involvement by using discussion groups, chat rooms, and other asynchronous technologies (Bielema, 1996). We use access location to the learning portal, MyGateway, as an independent variable in our analysis of student satisfaction. We also focus on use because, flexibility
of access aside, the ability of learners to apply technology in
support of their learning activity depends largely on their ongoing
response to using it (Storey, Phillips, Maczewski & Wang, 2002).
If using a particular technology is not relatively straightforward
and demonstrably effective in ways that are relevant to the
student’s learning activity, users (in this case students) will not
continue using the technology when presented with other options.
Thus, we predict that such students will agree with negative
statements about MyGateway and not agree with positive statements
about MyGateway.[3]
Our concern with use is different from a concern with usability per
se. We are focusing on the results of sustained exposure (high use)
to a learning management system in a blended learning class, as
opposed to limited or incidental exposure (low use). Further, our
concern is for how students perceive the learning management system
to affect their participation in the class (learning activity),
satisfaction with the class, and communication with the teacher as
compared to a traditional face-to-face class in which a learning
management system is not
used. MethodologyA stratified random sample (10 percent of the courses using MyGateway) was selected according to levels of faculty use of the learning management tool, MyGateway (N = 45). The courses were designated low use if faculty had logged 9 days or less in Control Panel accesses; high use if faculty had logged 10 or more days of Control Panel accesses by the middle of the semester (sample median was 8, mean 11). Questionnaires were completed in the class during the evaluation period at the end of the semester. Completed surveys by undergraduate and graduate
students numbered 666 with a 70% response rate. Students responding
in high use courses numbered 318, while 348 students in
low use courses responded. Given our interest in access, the
survey also asked students about the computing and network resources
available from their primary access location. QuestionnaireThe questionnaire for the current study consisted
of 46 items, with 16 of those items designed for the data analysis
reported here using a seven point Likert scale (see Table 2). The
Likert items asked students to indicate their agreement on a
seven-point scale ranging from (1) strongly agree to
(7) strongly disagree. Table 2. Questionnaire lead-ins and question text
Computers and Access for StudentsStudents were asked about their primary computer
access location and capabilities available to them. Over 60 percent
(61.6%) of the respondents reported owning a new computer (e.g.,
Pentium). Slightly fewer students in low use courses than
those in the high use courses indicated having the newer
computers. Only 9.8 percent of students reported that they do not
own a computer. Table 3. Primary Access Location
If a student’s primary access was from off-campus, they were asked specifics about the type of Internet connection: dial-up telephone via modem, dial up with DSL, cable modem, or other. See Table 2 for connection details of both groups. Again, the two groups of students were similar,
with cable modems used a little more frequently by the high
use students than those in the low use group. Table 4. Type of Internet Connection to University Servers
Blended Learning: Location and UseWe analyze the relationship of access location and use to learner satisfaction with MyGateway by: 1. distinguishing students who primarily use it on campus from those who primarily use it off campus 2. distinguishing low use from high use classes. An independent samples t-test was used to measure
each of these independent variables’ (access location and use)
predictive power in relation to specific items from a survey
instrument. The items used in the full survey were intended to
assess the learning portal from a larger array of concerns. The
items studied herein were developed to measure the degree to which
two independent variables (access and use) are significant
predictors for increased learner satisfaction (questions 33,
35, 36, 37, 38, see Table 2), increased learning activity
(questions 9, 11 – 16, see Table 2), and increased
student/teacher communication (questions 8, 10, 17, 32, see
Table 2) in the courses studied. We examine each of these below and
assume that as the degree of satisfaction with a technology
increases the more learning activity occurs, along with increased
student/teacher communication.
Access Location and MyGateway Student EvaluationsOur first concern in analyzing access location is to estimate its ability to predict the responses offered by students using MyGateway. To the extent that students have to travel to campus to use computers with access to the network, we expect those students to experience disadvantage in learning activities, lower satisfaction, and poorer student/teacher communication. So, our first three hypotheses test these relationships: H1: Access to learning resources over the Internet from off-campus locations increases student participation in learning activities in blended learning classes. H2: Access to learning resources over the Internet from off-campus locations increases student satisfaction in blended learning classes. H3: Access to learning resources over the Internet from off-campus locations increases student/teacher communication in blended learning classes. Specifically, we measured the relationship of
access location (either on campus or off-campus) to students’
responses to items involving learning activity, student
satisfaction, and student/teacher communication. Table 5. Access Location Items (independent samples t-test)
Question 35 was the only item from the survey that indicated a significant relationship (p < .05) between the access location of students and their response. Question 35, a student satisfaction item, stated: “It takes more effort to complete a course that uses online resources like MyGateway.” Judgments about the success or failure of those efforts are not predicted by access location. The type of off-campus connectivity available to the student does not affect the relationship, i.e. whether the connection was dial-up or broadband (DSL or cable modem). [4] Students accessing MyGateway from home, regardless of connectivity, were statistically more likely to disagree with Question 35. Alternatively, those students accessing MyGateway at campus locations were more likely to agree with Question 35. Thus, we find only limited support for the hypothesis that students who commute to campus to use MyGateway resources experience lower satisfaction. Qualitative comments from students on the personal benefits of using MyGateway support the notion that flexible access to materials, support staff, and teachers was a source of satisfaction. One comment in particular was instructive on this point:
Especially because [the university] is a
commuter campus, it is useful to be able to access course
information, contact instructors and stimulate “community”
interaction (via discussion boards). Degree of Use and MyGateway Student EvaluationsIn addition to access location, the survey items allow us to analyze differences between students in classes with low use of MyGateway and students in classes with high use. Our working hypothesis is that increased exposure to classes using MyGateway results in higher degrees of learning activity, satisfaction, and student/teacher communication. To the degree that the hypothesis is not supported we would expect lower acceptance of blended learning courses. H3: Increased use of the learning portal increases student participation in learning activities in blended learning classes. H4: Increased use of the learning portal increases student satisfaction in blended learning classes. H5: Increased use of the learning portal increases student/teacher communication in blended learning classes. The hypotheses, simply stated, claim that the
more people use the learning portal, MyGateway, the higher their
participation in learning activities, satisfaction with their
classes, and student/teacher communication when compared to
traditional classes—rather than the other way around. Use of MyGateway in Relation to Learning ActivityEvery learning activity item measured by the
MyGateway survey indicates that students in low use classes
are more likely (p < .05) to disagree with positive
statements about learning activities in MyGateway classes. Thus,
students in classes with high use of MyGateway were more
likely to review lecture notes for clarification (Question 9),
discuss ideas with other students (Question 11), actively
participate in the course (Question 12), work on assignments with
other students (Question 13), complete assignments on time (Question
14), access other online materials related to course content
(Question 15), and spend more time studying for the course (Question
16). Table 6. Learning Activity Items (independent samples t-test)
It is notable that even though students agreed
that they were spending more time studying for the course, they
nonetheless recognized the benefits of greater use of MyGateway in
their courses. Students responding to open-ended questions on the
benefits of using MyGateway reinforced the answers to the survey
items. The two most often mentioned benefits were the availability
of grades and course materials. The ability to easily communicate
with instructors and other students was also emphasized, along with
convenience of web access. Specific pluses mentioned included help
in staying connected to the class, getting instructor feedback
quicker, increased organization and opportunities for
clarification. Use of MyGateway in Relation to Student SatisfactionTwo items related to student satisfaction with
courses using MyGateway indicate that students in low use
classes are more likely (p < .05) to disagree with
positive statements about their satisfaction with MyGateway classes.
Thus, students in classes with high use of MyGateway were
more likely to agree that they are very satisfied with the course
(Question 33) and that they would like to have MyGateway used in
other courses (Question 36). Interestingly, responses to Question
35, which are significant when measuring access location are not
significant when measuring use. Even though students who travel to
campus to use the Internet think more effort is involved, their
responses are not affected by degree of use. In comparing high and
low use groups, MyGateway has no discernable effect on re-enrollment
(Question 37) or student retention (Question 38). Table 7. Satisfaction
Items (independent samples t-test)
|
|
Lead in Statement Headings and Survey Item Text |
|
|
Sig.
|
Mean Difference |
Std. Error Difference |
|
In general: |
|
|
|
|
|
Q33 |
I am very satisfied with this course because it used MyGateway |
5.77 |
700 |
.000 |
.83 |
.14 |
Q35 |
It takes more effort to complete a course that uses online resources like MyGateway |
-.52 |
697 |
.604 |
-7.65 |
.15 |
Q36 |
I would like to have "MyGateway" used in other courses |
2.86 |
715 |
.004 |
.37 |
.13 |
Q37 |
I am more likely to take another course because it uses MyGateway |
1.28 |
703 |
.201 |
.20 |
.16 |
Q38 |
I am more likely to complete my degree because of the advantages of using MyGateway in my courses |
1.23 |
692 |
.220 |
.19 |
.16 |
Three items related to student/teacher
communication in courses using MyGateway indicate that students in
low use classes are more likely (p < .05) to
disagree with statements about their degree of communication
with faculty in MyGateway classes.
|
Lead in Statement Headings and Survey Item Text |
|
|
Sig.
|
Mean Difference |
Std. Error Difference |
|
Because
of using MyGateway in this class |
|
|
|
|
|
Q8 |
Seek clarification when I did not understand something |
4.506 |
700 |
.000 |
.62 |
.14 |
Q10 |
Receive instructor comments on assignments quickly |
5.603 |
700 |
.000 |
.83 |
.15 |
Q17 |
Communicate with my instructor |
4.859 |
701 |
.000 |
.71 |
.15 |
|
In general: |
|
|
|
|
|
Q32 |
I prefer communicating with my instructor via [personal] email, outside MyGateway |
.714 |
702 |
.475 |
.11 |
.16 |
Thus, students in classes with high use of
MyGateway were more likely to agree that they were able to seek
clarification (Question 8), receive instructor comments on
assignments quickly (Question 10), and communicate with their
instructor (Question 17). Additionally, students did not express
(Question 32) a preference for using their personal email to
communicate with the instructor, implying that the MyGateway
learning portal’s communication resources are sufficient.
The data presented above indicate that increased
use of the Blackboard Learning System implemented as a learning
portal, MyGateway, results in higher estimates of learning activity,
higher degrees of satisfaction, and higher student/teacher
communication than in courses not using the portal. Thus, the data
from the current survey lend support to the contention that the more
students are exposed to learning portals like MyGateway the higher
their estimation of its positive affects. We suggest that the
findings here are not only applicable to the MyGateway
implementation of the Blackboard Learning System. Rather, we contend
that similar findings are likely in any learning portal
implementation when students get accustomed to using it (i.e., use
it in direct support of the course work), especially when the course
consists of a blend between traditional co-present classrooms and
asynchronous, computer-mediated learning portals like MyGateway.
Barbian, J. (2002) “Here’s Proof” online learning magazine 6,6, pp. 27-31.
Bielema, C. L. (1996) Factors affecting implementation of interactive, computer-mediated instructional techniques for instructors and learners at a distance (Doctoral dissertation, University of Illinois, Urbana-Champaign) Dissertation Abstracts International, 57(11), 4703A.
Hara, Noriko & Kling, Rob (1999) “Students' Frustrations with a Web-Based Distance Education Course” First Monday 4, 12. Retrieved from http://firstmonday.org/issues/issue4_12/hara/index.html
Irons, L., Jung, D. & Keel, R. (2002) “Interactivity in Distance Learning: The Digital Divide and Student Satisfaction” Educational Technology & Society 5, 3, pp. 175-188. Retrieved from http://ifets.ieee.org/periodical/vol_3_2002/jung.pdf
Storey, M.A., Phillips, B., Maczewski, M. & Wang, M. (2002) “Evaluating the usability of Web-based learning tools,” Educational Technology & Society 5, 3, pp. 91-100. Retrieved from http://ifets.ieee.org/periodical/vol_3_2002/storey.pdf
Valore, L. & Diehl, G.E. (1987) The effectiveness and acceptance of home study. Washington, DC: National Home Study Council.
Larry Irons combines industry and academic livelihoods in his professional career. He is Managing Partner of I.C. Technologies, a consulting services company focused on using performance support, eLearning and communities of practice to achieve client goals. Larry has researched distance learning since the early 1990s, participating in several industry/academic/government-sponsored projects. His consulting and research interests both focus on how communication technologies are designed and used. Larry received his Ph.D. in Sociology from Washington University in St. Louis in 1992.
Larry can be contacted at: larryi@ic-t.com
Robert Keel has taught sociology for the past 26 years. He currently is a lecturer in the Department of Sociology at the University of Missouri-St. Louis. For the past 8 years he has worked to develop Internet-based class resources for his classes at UM-St. Louis. He also works as a faculty liaison and specialist in information technology for the UM-St. Louis Information Technology Services department.
Robert can be contacted at rok@umsl.edu.
Cheryl Bielema, Instructional Designer, manages the Faculty Resource Center, University of Missouri-St. Louis, consulting with and training faculty in the integration of technology. She collaborates with the Center for Teaching and Learning, jointly sponsoring teaching and technology workshops for faculty and staff. Bielema worked previously at the University of Illinois, Urbana-Champaign, in distance learning as well as in UI Extension. Her teaching experience is in Curriculum and Instruction and Educational Technology. Her Ph.D. is in Human Resource Development, University of Illinois.
Cheryl can be contacted at bielema@umsl.edu.
Endnotes
[1] A word on terminology is needed here. We use the term student when referring to specific characteristics of participants in the current survey who were, after all, university students. The term learner is used when discussing analytic findings with general implications for blended learning. The term “blended learning” is used as an umbrella concept to encompass various preexisting categories that provide less analytic utility as the multiple modalities depicted in Table 1 become increasingly common. Such terms as e-Learning, online learning, web-based learning, and distance learning are increasingly ineffective categories with efforts to distinguish between them rather futile.
[2] Heidi J. Larson provides a useful overview of the Blackboard Learning System on the Training Media Review web site, http://www.tmreview.com/review.asp?id=876&mem=true
[3] Storey, Phillips, Maczewski & Wang (2002) asked students and faculty to compare Blackboard to WebCT’s learning management system on issues of navigation, customization, student management and content creation and noted an overall preference for Blackboard. Further, it is interesting to note that the authors contend that most “usability principles were violated by the tools [Blackboard/WebCT] we evaluated and negatively impacted students and their attitudes towards these tools” (p.92).
[4] It is important to note that the failure of connectivity (i.e., dial-up vs. broadband) to predict the outcomes expected from our hypotheses may result from the fact that the MyGateway system classes do not typically employ rich media as part of the blended learning situation. Thus, as classes increasingly use rich media, such as streaming video and animation, it will remain important to consider access location and connectivity when studying blended learning classes.