My Gateway (i.e., Blackboard™) was introduced on the UMSL campus Fall Semester 2000. From an initial adoption by approximately 150 faculty, using the web course management tool in 270 courses, the implementation rate has grown to 435 faculty using My Gateway in 600 courses during fall semester 2001.Their level of use varied from posting the syllabus and emailing students to utilizing the full range of features available (i.e., course documents, discussion board, external links, group pages, online quizzes, and student grade book).

Evaluations have been completed by participating instructors and by students in randomly selected courses each semester. This report summarizes the results of the most current evaluations. Additionally, implications for future support and training are suggested.

Faculty Instructional Uses for My Gateway

In a mid-semester faculty survey, 67 respondents indicated how they were using My Gateway in their courses. (Total faculty users at mid-semester was estimated at 350, based upon active course sites). Answers revealed the specific features faculty used, amount of time they spent preparing materials and using My Gateway, and what learning activities they had implemented. Table 1 summarizes their use of the various features.

Table 1: My Gateway Features Utilized in Fall Semester 2001

Perceived Barriers And Benefits

Faculty listed the following barriers in using online learning materials or activities (in order of their frequency):

1.       Technical problems with computers

2.       Perceived personal (real or imagined) lack of technical expertise with computers

3.       Not having a computer at home; or one that is inadequate

4.       Problems accessing My Gateway account

5.       Inability to use email efficiently

6.       Inadequate tech support

7.       Slow downloads from remote/home locations

8.       Lack of student motivation to master skills necessary for success in the cyber world

9.      Platform and software incompatibility

10.   Face-to-face interaction more stimulating

Benefits of using online materials and activities (in order of frequency):

1. Wide range of information (grades, assignments, instructor feedback) available to students at all times
2. Everyone gets same information
3. Collaborative learning experience
4. Enhanced instructor access
5. Creates venue for equality of student participation
6. Online information is always current
7. Expands classroom walls
8. Easy to create class notes for all
9. Provides multiple avenues for student inquiry
10. Enhanced ease of production of graphic materials for class

Improvement Suggestions

When questioned about support needed from the My Gateway support team, faculty respondents made these suggestions:

Ø       Provide easily accessible training for instructors: flexible schedules for in-service programs, employ varied learning/teaching methods (hands-on and demo), as well as books and online instruction

Ø       Email pop-up messages (e.g., if you’re having problems with so-and-so, try this)

Ø       Unit that could do an in-class My Gateway introduction and demonstration upon request

Ø       More adjunct support

Ø       Specific topics for how-to sessions included “using video clips,” “converting traditional course to online,” “using virtual chat,” “using external links,” and “uploading or linking to quiz files.”

Successful Learning Activities

In response to an open-ended question about the successful teaching techniques they had implemented, faculty listed the following items.

A variety of course documents are uploaded for access anytime and anywhere by students. Syllabi and lecture notes were mentioned most often, followed by PowerPoint slide files. Lecture notes are generally available before class, so students have them in hand for note taking. Other course documents mentioned:

Ø       Video lectures via WWW on which students write reports

Ø       “Hints on Reading”

Ø       “What it Takes to Be Successful in an Online Course”

Ø       Answers to homework problems

Ø       Link to an online learning center, accompanying the current textbook.

Discussion Forum topics extend, enhance, or summarize face-to-face classroom discussions, according to the respondents. The following ideas were listed:

Ø       Getting-acquainted topics increase comfort level with discussion

Ø       Small group journaling

Ø       Questions posted about campus lecture series (students attend and respond via Discussion Board)

Ø       Annotated bibliography is shared among graduate students as they read in specific areas

Ø       Case studies are discussed

Ø       Student projects are presented and discussed

Ø       Anonymous postings are “enabled” for class-related issues

Ø       Questions and answers about course content and assignments handled in a discussion forum for that purpose

Ø       Review of web sites shared via discussion

Ø       Students summarize a discussion forum or evaluate quality of other students’ input

Ø       Guest lecturers were invited to facilitate a discussion for specified time; they prepared a case study or notes related to their expertise (e.g., project management).

Accessing WWW resources is the third most-used instructional technique by these faculty members. Here are several innovative ideas:

Ø       Instructions for how to use search engines and potential search terms were part of a web resource assignment

Ø       Following September 11 tragedy, students were directed to web sites to search for the economic consequences

Ø       Individual students review an educational web site

Ø       Link to a web site to complete homework assignment

Ø       Access to readings too costly to provide in hardcopy

Ø       Finding and using resources other than textbooks teaches students, resourcefulness.

Quizzes are utilized for preview of class discussions and review before scheduled tests, in addition to graded assignments. The digital drop box permits sharing of preliminary drafts of papers among students and/or the instructor. Clinical logs are submitted electronically and receive immediate, personal response. Virtual chat has been used to supervise the field experiences of students.

My Gateway Is a Positive Force

These comments give clues to the progress of technology integration across our university.

Keep up the great work!

I don’t like how the spreadsheet view for grade book is set up

Rocky [Keel] has been exceptionally helpful…. I like the My Gateway system, and am looking for ways to increase its use…

Thanks for your support – always given in a timely manner

I love the My Gateway system. I resisted using a web site of my own because of the high “set up cost”…. The off-the-shelf assistance of My Gateway fits my skill set and schedule

Online grade book and course documents (the only two features I have been using) have been VERY useful

I think the My Gateway staff has been outstanding. I was very uncomfortable at first, but when I called I always got a quick & friendly response. This encouraged me to do more. The staff should be commended

Having access to computer technology in the classroom on a daily basis seems a bit of a problem in Marillac Hall

It has helped me keep sections of my course the same through having the materials copied from my section to those of adjuncts

The first time using Blackboard fully for a class takes a great deal of time, but it is so much easier the second and third times.

Assessment by Students

Courses were randomly selected in winter semester 2001, with participation by faculty and students divided between face-to-face surveys and online surveys. The final sample was 21 courses, comprised of 890 enrolled students. Sixteen of the classes were surveyed face-to-face with hard copy instruments and scantron sheets. The remaining classes were invited to complete an online survey. Final respondents numbered 522 for a response rate of 59 percent.

In fall semester 2001, all active faculty members on the My Gateway list were asked to involve their students in an end-of-course My Gateway assessment. They were given a sample announcement including the survey URL for posting on their course sites—ONLY the online survey was available. All five colleges were represented among the participating students in 53 different courses. Final number of respondents was 351; 28 additional surveys were posted after the data were run and are not included in the data analysis.

Who are the Students Responding

Students indicated the highest degree earned, if applicable. They were asked how many credits they had completed at UMSL, as well. Over 60 percent indicated no degree and are classified as undergraduates for the table below. Fifteen percent (15%) had previously earned an Associate's degree while 14 percent indicated they had a BA or BS. Those having completed advanced degree programs comprised 3.2 percent.

Table 2. Highest Degree Earned

Credits earned at the time of the survey were sub-divided into 15 semester-hour segments. Thirty percent (30.7%) appeared to be in their first semester at UMSL, indicating 0-15 credits; 13.2 percent had earned 16-30 credits; 12.8 percent, 31-45 credits, while 10.7 percent checked 46-60 semester hours. Those accumulating 60+ credits comprised almost 33 percent (32.5%)of the respondents.

Computers and Access for Students

Technology requisites are a computer and Internet access. Students were asked to check primary computer access and capabilities they had available. Types of Computer, Primary Access Location, and UMSL Dial-Up Traffic are the three tables that follow.

Over 60 percent of the respondents reported owning a new computer (e.g., Pentium), in the winter and fall semesters. Students owning older computer declined slightly from 4 percent in winter semester to 2 percent in fall semester. There was a slight decrease of those reporting a Macintosh, to 2.7 percent. “Other” computers represented 11.8 and 12.7 percent, respectively. An additional 5 percent indicated they didn’t know what type of computer they owned in fall semester. The students reporting no personal computer has varied from semester to semester, from a high of 16.5% in the first semester, to 9.5% and 14.4% in the last two semesters.

Table 3. Types of Computers Owned by Students

Students indicated their primary access location, off-campus via a modem (65.2%), computer or departmental labs (26.2%), student housing (4.7%), and WebTV (.6%). There was a slight decrease in reported use of dial-up modems for Internet access in winter semester compared to fall semester, with the concomitant increase in use of computer labs and other UMSL computing resources, from fall semester to winter semester. Overall, the total for UMSL computers was 34.2% WS, compared to 30.6% FS. If a student’s primary access was from the computer labs, they were questioned about wait time in getting to a computer. Their responses indicated limited wait times, with 87.3% (compared to 62.3%, FS01) typically waiting 0-5 minutes, and 8.3% (compared to 26.5%, FS01) having waited 6-15 minutes.

Table 4. Primary Access Location

If a student’s primary access was dial-up and modem, they were asked about the time they encountered a busy UMSL dial up. Reports were favorable, in that 70 percent experienced busy dial-up 0-10% of the time, and an additional 17.7% reporting busy dial-up 11-25% of the time.

Satisfaction with individual Internet access speed was a third aspect to be questioned. Nearly 75% indicated they were satisfied with speed of access, while the other 26% indicated dissatisfaction. Important to future educational applications using multimedia is the fact that 86% of the students responded that their personal computers were “multimedia capable.” Access speed may be adversely affected by the bigger, better and more diverse applications.

Table 5. UMSL Dial-up Traffic

Learning with and without My Gateway

  A series of statements sought to identify any learning differences between web-assisted courses (using My Gateway) and traditional courses (NOT using My Gateway). The areas where respondents indicated greater differences included “referring to a syllabus,” “receiving quick feedback on quizzes and exams,” “accessing online materials,” “communicating with instructor,” “receiving instructor comments quickly,” “assignments completed on time,” and “helpfulness of quizzes.”

Areas where there was less difference between face-to-face courses and web-assisted courses appeared in “work with fellow students,” and in “discussions about concepts improved learning.” These results may indicate that the instructors did not have specific learning objectives related to online collaborative assignments or threaded discussion requirements.

Table 7. Learning with My Gateway

Note. The last column may help readers interpret the change over this one-year period. Students were instructed to “skip items that do not apply.” The N/A column lists the number of students who skipped each question.

Overall Impact of Online Materials and Learning Activities

In summing up the online learning experience, students were asked if the online environment had contributed to his/her learning. There has been growth in the positive responses from 65 percent in FS 2000 to 81 percent one year later in FS 2001. Additionally, stronger advocacy for online teaching and learning is revealed by the increased percentage of students indicating they would like My Gateway used in other courses – it was the majority opinion by 82 percent in FS 2000 and has grown to 91 percent of students agreeing in FS 2001.

Respondents were asked to assess their rate of participation and effort in using My Gateway compared to participation and effort in traditional courses. A majority (55%) indicated that their effort was about the same as that in traditional classes, in each of the semesters. However, participation seems to have increased -- 35 percent indicated  “greater to much greater” participation in the first semester of My Gateway availability, while one year later, nearly half (48%) marked “greater participation” in those courses using My Gateway. Students corroborated this in their comments about benefits of using My Gateway [below].

Perception of faculty enthusiasm was positive, as students indicated whether or not “the instructor was excited about using My Gateway – 81% (FS 2000) vs. 90% (FS 2001) agreed or strongly agreed. Students themselves indicated satisfaction with My Gateway – 82% were satisfied or greatly satisfied in the first semester’s surveying, while the current satisfaction rating is 88.5% for our students.

Student Experiences with My Gateway

Students described more specific experiences in using My Gateway. They responded to open-ended questions: "what are the benefits of using My Gateway," and "how can the use of My Gateway be improved for the course?"

Benefits of Using My Gateway

The following benefits are listed in order of their frequency:

Ø       Fast and easy access to course materials from anywhere

Ø       Ease of viewing grades

Ø       Great way to keep in contact with instructor; email provides prompt communication

Ø       It helped to read classmates ideas on Discussion Board and encouraged those of us uncomfortable speaking in class

Ø       I’m better prepared with instructor’s lecture notes; helped reinforce face-to-face lectures

Ø       Quick turn around on grades for quizzes and assignments

Ø       Allowed me to work more at my own pace, at home and didn’t necessitate trips to campus

Ø       Encouraged my [active] participation and application of subject matter

Quotes by our Students

Using electronic tools makes me more comfortable with technology—I’m constantly learning new things

Enabled me to interact even though I don’t live in St. Louis. Previously I have felt more isolated from colleagues taking courses. This has opened up more interaction for me

Classes that do not use My Gateway I tend to participate less in. The classes that use My Gateway I have had a grade b or better. I can tell if I am doing well in the class or if I need to participate more in class or studying.

Improvement Suggestions

For this listing, students were asked if they had experienced any problems using My Gateway. Their responses are categorized according to implications for students support services or potential topics for faculty instructional workshops.

Student Support Services

The following identified problems might be solved through training or specific help documents:

Ø       How to copy and paste

Ø       Drop Box – how it works

Ø       Discussion Board “options” to collect and manage postings

Ø       Explanation of NEW student email and My Gateway – now accessible in My Gateway frame

Ø       Free software (i.e., PPT reader or Adobe Acrobat reader) needed

Ø       Right click to “open frame in new window”

Quote by one of our students:

A quick lesson on how to use individual features at the start of a course would be helpful.

Home Computer Use Problems

These problems were directly related to use of home computers:

Ø       Slow loading or downloading of documents via Internet Service Provider

Ø       My Gateway is slow to load via DSL

Ø       When trying to access My Gateway, message was “connection is busy” or got a busy signal (is it my phone or UMSL’s lines?)

Ø       Computer not showing text boxes

Ø       Slow page loading through UMSL dial-up

Ø       Do not own a home computer

Student Email

Ø       Easier toggling between the systems

Ø       Couldn’t access my email account

Ø       Want to use preferred email address

Instructional Improvements Suggested

Ø       All instructors use My Gateway (or) use more of the features available

Ø       More interaction from instructors on the Discussion Board

Ø       Teachers use announcements more effectively; more frequently

Ø       Teachers more accessible by email, rather than office visits or telephone

Quote by one of our students:

The only thing I wish is that all teachers would know how to use the system, so we can use [My Gateway] for every class.

There are those students who don’t like using My Gateway. The objections included:

Ø       It’s impersonal to use the computer

Ø       I prefer listening and taking notes in class

Ø       Don’t allow online quizzes

Ø       Instructor spent too much time working out the bugs with My Gateway

Ø       I don’t like My Gateway simply for the reason that I did not check it enough. I don’t understand the system that well. I was never able to receive my [student] email.

Students and instructors are discovering the learning pay-offs as well as the trade-offs of technology use. Students mentioned greater involvement and contact with their peers and with instructors. Instructors are creating additional learning activities using web resources and asynchronous interactions among students. The comments were increasingly more positive about access to course materials, including grades, and the availability of outside resources.

Both groups advocated more strongly in the latter surveying for all instructors to learn how to use My Gateway AND use most or all of the features therein. As one student wrote, “To check grades, assignments, get lecture notes, communicate with professor and other students, place opinions on things studied in class. There are so many benefits to this system, I don’t have the time to name them all.”

Questions about the survey findings may be directed to Dr. Cheryl Bielema, Instructional Development Specialist, 119 South Classroom Bldg., University of Missouri, St. Louis. PH: 314-516-7134; EM: bielema@umsl.edu. Evaluation team included Robert Keel, Laura Hofer, and Cheryl Bielema.

Information Technology Services

February 7, 2002