XplanaZine

You are invited to participate in a new survey, which responds to the fact that new ways to assess quality in the development and implementation of online courses must continuously be developed. The goal of this survey is two-fold: to provide a checklist of current standards, and to gauge shifts in attitudes and ideas about the importance of items. In doing so, emerging trends and practices will reveal themselves, broken down geographically, demographically, as perceived by the various decision-makers in the online course development, implementation, and administration process.

The following instrument has been designed to be responded to by individuals involved in online education, training, and lifelong learning. It is available online and you are invited to go to the site and respond to it now. Taking the survey will require approximately 15 minutes. http://intercom.virginia.edu/SurveySuite/Surveys/Course_Benchmarks

The instrument, “Online Course Quality Benchmarks,” was developed after a thorough literature search, and the analysis of numerous best practices studies, including the Institute for Higher Education Policy’s Benchmarks for Success in Internet-Based Distance Education for assessing e-learning published in 2000. Other sources include the Sloan-C’s effective practices for quality online education. The survey detailed below was also developed after the implementation of smaller and more focused online surveys dealing with online course effectiveness and learner satisfaction.

All questions should be answered on a scale of 1 - 5

1 – Definitely No
2 – Sometimes No
3 – Neutral
4 – Sometimes Yes
5 – Definitely Yes

Learning Goals
Because the mediation that occurs in face-to-face environment is not as easily effected in an online course, it is very important to identify and describe the desired learning outcomes. If that does not happen, it is possible that students will become enmeshed in peripheral items. Consistency of outcome, and an ability to classify and remember specific fact-based knowledge will be a primary goal. Another is the goal of assuring that deeper learning is taking place, which would be evidenced in tasks requiring synthesis, creative problem-solving, and conceptual analysis.

1—Does the course include a list of overall learning objectives?
2—Does the course syllabus or website describe how the learner will achieve the objectives?
3—Are the learning objectives clear?
4—Do the learning goals include synthesis of course-spanning material?
5—Do the learning goals include mastery of declarative knowledge involving specific content?
6—Are there learning goals for each unit?

Content
The instructor or course facilitator in an online course must plan the delivery of content in an online course. In a face-to-face environment, content can be modified, challenged, and spontaneously delivered in response to questions. In an online environment, planning content is important in order make certain that the depth, breadth, and level match the needs and abilities of the learners, in addition to making certain they have access to information and activities needed to achieve learning goals.

7—Does the online course include the right amount of content to be able to achieve all the learning objectives?
8—Does a person other than the instructor review course materials to be assured of the quality of course materials?
9—Is the content of the course easily covered in the time frame required by the course?
10—Is the course content at the right educational level for the learner?
11—Is the course content variable so that students wishing enrichment or additional reading can easily find it, either in the online course itself, or in easily accessible libraries or resource centers?

Instructional Design
While informal instructional design has always played a part in face-to-face learning, in previous delivery-modes, there has been a reliance on an apprenticeship model in which master teachers share their pedagogy and approaches. Although shadowing and mentoring are important, the online environment requires the construction of a learning space far in anticipation of the actual activity. In order to be effective, instructional design tasks have to be formalized and there should be an ongoing awareness of how and why decisions are made. Best practices will evolve over time, always in response to learner needs.

12—Is the course designed in such a way that the learning goals are made clear?
13—Are specific activities developed that center on each learning goal?
14—Are learners presented with more than way (or activity) to achieve a learning goal in order to accommodate multiple learning styles?
15—Are activities designed in a way require faculty to respond with guidance and coaching?
16—Do instructors have a chance to modify the course content or activities?
17—Do graphics reinforce course content and help students make connections and/or illustrate learning goals?
18—Does the course management software have an integrated function that allows students to do more than one thing while logged into a single site, such as engage in a discussion board, online chat, send e-mail, submit work, access course documents, participate in group projects, contact the professor?
19—Does the course management software work well over a modem connection?
20—Can a student log into the course management software from an internet café, military computer lab, or office where there are strict firewalls?

Interactivity
Many studies have demonstrated that student success in an online course depends on the level and quality of interaction. Interactivity in an online course can take many forms, ranging from automatic responses generated by applets, to human interaction in discussion areas or via e-mail. The guidance provided by interactivity not only keeps a learner focused on overall objectives, it also is a factor in motivation.

21—Is the learner engaged through the opportunity for input to automated programs?
22—Is the learner engaged through interaction with other students in a virtual classroom or learning community?
23—Is the learner engaged in one-on-one communication or interaction with the instructor?
24—Does the instructor have an opportunity to modify or customize interactive tasks, such as discussion board topics?

Navigation
Learners are frustrated when they find themselves lost in the course and who cannot find help to make their way in an ordered, understandable manner. Navigation is not simply a matter of providing links. It also involves efficiencies and ease of access. In a course requiring more than three links where students are accessing via modems on phone connections, their connection time is likely to be slow. Course management software that requires frequent logins and multiple screens to access a single point is one such problem. This introduces another level of frustration. Navigation is a key component, but often overlooked in course management and design.

25—Can easily users determine how to find the syllabus, required work, e-mail, discussion board, content and course activities in the course?
26—Are there navigation buttons to library or other online resources?
27—Is there a navigation button for help with the course management system?
28—Is the course designed so that the pages load rapidly after the navigation buttons are clicked?
29—Is there a navigation button that shows learners how to submit their work?
30—Can access to key information or activities be accomplished in three clicks (screens) or less?
31—Is there a course map?
32—Are the icons and/or clear labels intuitive enough that a learner can learn by doing, rather than having to read user manuals or excessive documentation?

Motivational Components
Without taking motivation into consideration, online courses are likely to fail. Motivation does not simply involve the learner. It also involves the instructor, who, if not properly motivated, will not follow through in tasks required for student success. What motivates some learners will demotivate others. For that reason, it is important to understand the learners’ needs and backgrounds and develop motivational components with those elements in mind.

33—Does the course engage the user through novelty, humor, game elements, testing, adventure, unique content, surprise elements, etc.?
34—Has a learner needs assessment been conducted to assure that course activities give the learner an opportunity to make connections to his or her own interests or goals?
35—Do course activities clearly seem relevant to achieving learning objectives?
36—Do course activities help prepare learners to better deal with situations they will encounter in the real world?
37—Is the instructor charged with providing timely feedback?
38—Is the instructor required to provide substantive feedback, such as guidance, suggestions, revision requirements, corrections of fact, encouragements to look at more than one side of an issue?
39—Are students guided and encouraged to directly respond to other learners in threaded discussion boards?
40—Has information been gathered about the learners to assure that the motivational components are effective, culturally appropriate, and not offensive?
41—Are required texts and course materials of high quality? Are they up to date?
42—Is the instructor required to participate in training that includes how to provide positive motivation to learners?
43—Are incentives or rewards provided for excellence in online instruction?

Use of Media
The use of streaming media in online learning can be invaluable, particularly in subject matter areas that have a performative aspect. This includes language, film studies, medicine, engineering, sports, and other areas. It can also be a good source of illustrative content. However, badly implemented and designed, media can be distracting and so cumbersome that access is not realistically possible. The effective use of media – from both instructional design and technology points of view – must be determined.

44—Does the course appropriately and effectively employ graphics, animation, music, sound, video, etc.?
45—Is the gratuitous use of these media avoided?
46—Does the software used for streaming media accommodate different connection speeds, including modems?
47—Does the media encourage interactive learning?
48—Does the media feature a “talking head” or a tape of a professor delivering a lecture in a typical traditional classroom?
49—Is the music or soundtrack annoying?

Evaluation
In order to determine whether or not a learner is achieving goals and objectives, and if student learning is consistent across the curriculum, it is necessary to develop methods of assessing and evaluating their progress. Overuse of evaluation can be demotivating, however. Finding the right balance is very important.

51—Are learners evaluated following the completion of a simulation?
52—Are learners evaluated or assessed after the completion of writing activities?
53—Do learners receive grades or assessments after completing a research activity?
54—Must learners demonstrate mastery of each section’s content before proceeding to later sections?
55—Do learners have section quizzes?
56—Is there a final exam?

Aesthetics
Aesthetics have to do with the overall look and feel of the online course. Although aesthetics may play a role in traditional face-to-face learning, usually the appearance of a presentation, or the aesthetics of the overall learning experience are not as important as in the online environment, where distracting designs and colors can have a negative impact on the student. Aesthetics include issues of semiotics, signs, symbols, and arrangement – all the non-textual elements that generate meaning and interpretive possibilities.

57—Is the interface design attractive and appealing to the eye and ear?
58—Are the graphics stimulating?
59—Is the course evaluated by more than one person in order to determine whether or not graphics or design could be offensive or culturally inappropriate?
60—Is the course navigation element designed so that it does not create a distraction?

Record Keeping
Without clear and consistent record-keeping, students and instructors quickly become frustrated and errors are made. Privacy laws must be respected, and security is a must. The online environment requires administrators and designers to re-think record keeping.

61—Are student performance data recorded, such as time to complete, grades on activities, question analyses, and final scores?
62—Are scores or grades communicated directly to the student?
63—Are grades posted in an online gradebook?
64—Is the site secure?

Tone or Ethos
The tone of a site relates a great deal to the tone of a written document or a media presentation. It encompasses the “voice” that comes through in the situation. Generally stated, “ethos” has to do with credibility, and is closely related to ethics, character, and believability. Both are vitally important in the online environment. If tone and ethos are problematic, the validity of the information, the approach, and the instructor can be seriously compromised.

65—Is the tone of the writing in the online course appropriate for the audience?
66—Does the tone avoid being condescending, trite, pedantic, etc.?
67—Is the tone consistently positive and encouraging?
68—Does the tone promote a sense of fairness?
69—Is the tone (or “ethos”) of the course a consistent one that encourages intellectual risk-taking and intellectual investigation?
70—Does the tone or “ethos” create a supportive environment for making connections between real-world situations and the course content?
71—Does the tone encourage the learner to become emotionally invested in a positive attitude about online learning?