In today's educational context, the delivery of education
via the Internet or Web based learning is not a new concept. In fact, this
medium of education delivery is much sought after by students and many
academics now accept and use this environment. The opportunity now exists
for educators to look more closely at the effectiveness and appropriateness
of a Web based learning environment. Educators need to equip themselves
with instruments that allow the effectiveness of Web based learning to
be evaluated.
This paper describes the development of a new Web based learning environment
evaluation instrument. Apart from demographics and general background information,
this instrument is divided into four main aspects. The first three are
adapted from Tobin's (1998) clusters of emancipatory activities, co-participatory
activities and qualia. The fourth aspect of the instrument focuses on information
structure and the design of online material.
Background
Using the Internet as a medium, web based learning can be viewed as
an innovative approach for education delivery. The Web learning environment
is potentially a powerful teaching and learning arena in which new practices
and new relationships can make significant contributions to learning. We
now have a shift in learning, unlike traditional face-to-face teaching
and learning, attention needs to be paid to online teaching and learning.
Designing and delivering instruction on the Web requires thoughtful analysis
and investigation of how to use the Web's potential (Khan, 1997). Before
we can investigate the effectiveness of web based learning environment,
we need to revisit the role of teachers and students in a face-to-face
learning environment and compare this to web based learning environment.
Clearly the role of the teacher either in a classroom face-to-face teaching
or online setting is to ensure that some type of educational process occurs
amongst the learners involved. In the traditional classroom setting, the
teacher role generally imparts knowledge to learners (Relan & Gillani,
1997). In the Web environment, the role of the teacher becomes that of
an educational facilitator (Sherry & Wilson, 1997). As a facilitator,
the teacher provides guidance and allows students to explore the course
material as well as related materials without restriction. Students in
face-to-face classroom setting see and work with one another and get to
know each other well through the learning process. For learning to be successfully
delivered online, the learning process on the Web needs to be well facilitated.
Roles of the online teacher
>
Collins and Berge (1996) categorise the tasks and roles of the online
teacher into four areas: (1) pedagogical, (2) social, (3) managerial, and
(4) technical. They described pedagogical as function and task that revolves
around educational facilitation. Social function is described as the promotion
of friendly social environment which is needed in the process of online
learning. The managerial aspects of online learning involves setting agenda,
objectives, rules and decision making norms. The technical aspect focuses
on the teachers' proficiency with the use of the technology. The teacher
in an online course must ensure that they themselves are comfortable with
using this technology to transfer knowledge to students.
Roles of the online learner
A successful learner should be active in an online learning environment.
The roles of the online learners include (1) knowledge generation, (2)
collaboration, and (3) process management (Palloff & Pratt, 1998).
In online learning environment, the learner is responsible for actively
seeking solutions to problems confined within the knowledge area being
studied with guidance from the teacher. They are expected to view problems
and questions presented by the teacher and those of other students. Students
in the online learning environment are also expected to learn collaboratively
and cooperatively (Khan, 1997). Students are expected to work together
in order to generate deeper levels of understanding of the course material.
Students are also expected to share the resources and other materials that
they are finding with other learners. In the role of process management,
students are expected to participate with minimal guidelines, interact
with one another and speak up when the discussion are moving into an uncomfortable
zone. Students must be willing to speak out when they are offended or simply
have an opinion on something (Palloff & Pratt, 1998).
Web based learning applications
Finder and Raleigh (1998) explain four ways in which Web applications
could be used in a course: (1) Informational, (2) Supplemental, (3) Dependent,
and (4) Fully Developed. Informational use is described where course information
such as course outline and assignment descriptions are available on the
Web. Supplemental use includes students use of the Web to complete part
of the course and the learning materials generally includes links to related
sources. Dependent use is described as that most learning materials exist
on the Web and students use the Web to complete course assignments. Fully
Developed use is viewed as using the Web to deliver the entire course.
This is deemed as offering the course totally online where students and
teacher may never meet face-to-face. The Web based learning environment
instrument described in this paper focused on the four Web applications
mentioned above. Tobin (1998) in his work on Connecting Communities Learning
has described a framework which can be used for evaluation of learning
environments in interactive environments. This new web based learning environment
instrument, known as WEBLEI, is built on the work of Tobin (1998).
The Web based Learning Environment Instrument (WEBLEI)
This instrument is designed to capture students' perception of web based
learning environment. Apart from demographics and background information
sections, there are four core aspects in the instrument. The first three
aspects are adapted from Tobin's (1998) work on Connecting Communities
Learning (CCL) and the final aspect focuses on information structure and
the design aspect of the web based material. Each of these aspects is explained
in the following section.
WEBLEI Scale I: Emancipatory activities
Tobin (1998) listed three main categories of convenience, efficiency
and autonomy for emancipatory activities.
- Convenience is achieved when students can access the learning activities
at convenient times.
- Efficiency is described as not having to attend on campus classes and
therefore allowed for efficient use of time.
- Autonomy is described as allowing students to decide when and how to
access the curriculum. (Tobin, 1998:151)
The following items are included in evaluating Emancipa
tory Activities:
- I can access the learning activities at times convenient to me.
- The online material is available at locations suitable for me.
- I can use time saved in travelling and on campus class attendance for
study and other commitments.
- I am allowed to work at my own pace to achieve learning objectives.
- I decide how much I want to learn in a given period.
- I decide when I want to learn.
WEBLEI Scale II: Co-participatory activities
According to Tobin (1998), "co-participation implies the presence
of a shared language which can be accessed by all participants to engage
the activities of the community, with a goal of facilitating learning".
Included under the co-participatory activities are six categories of flexibility,
reflection, quality, interaction, feedback and collaboration.
- Flexibility is described as allowing students to meet their goals.
- Reflection is noted as asynchronous interactions which encouraged reflective
interactions.
- Quality is linked to the learning reflected in the level of activity
undertaken by the students.
- Interaction is described as enabling students to interact with each
other asynchronously.
- Feedback is described as the availability of feedback from students
and the teacher.
- Collaboration enabled students to collaborate in a variety of activities.
(Tobin, 1998:152)
The focus of this aspect is on the learning activities that the students
will participate in. This includes structuring of activities in which students
use their existing knowledge to apply to the present subject and build
new understandings from the present subject. This co-participatory aspect
is aligned with Laurillard's (1993) analysis of how learners 'come to know'
through (1) active learning, (2) feedback, and (3) reflection.
The following items are included in evaluating Co-Participatory Activities:
- The flexibility allows me to meet my learning goals.
- The flexibility allows me to explore my own areas of interest.
- I am encouraged to explore concepts beyond my regular web based lessons.
- The asynchronous nature of the interactions enables me to reflect and
respond when I had formulated an appropriate response.
- This mode of learning enables me to interact with other students and
the tutor asynchronously.
- I communicate with other students in this subject electronically (via
email, fax, bulletin boards, chat line).
- I have the autonomy to ask my tutor what I do not understand.
- The tutor responds promptly to my queries.
- The tutor addresses my queries adequately.
- The tutor sends me comprehensive feedback on my assignment.
- I have the autonomy to ask other students what I do not understand.
- Other students respond promptly to my queries.
- In this learning environment, I have to be self-disciplined in order
to learn.
- I regularly participate in self-evaluations.
- I regularly participate in peer-evaluations.
- It is easy to organise a group for a project.
- It is easy to work collaboratively with other students involved in
a group project.
WEBLEI Scale III: Qualia
Tobin (1998) explained qualia by describing knowledge which is considered
"as embodied in neural networks as vectors of electric charge that
reflect life experiences of individuals". According to Churchlands
(1989) and Churchlands (1996) "neural network theory conceptualises
knowing in terms of electronic loadings on a matrix of neurons that tightly
couples qualia and cognitive ways of knowing".
Tobin (1998) described six categories of qualia. They are enjoyment,
confidence, accomplishments, success, frustration and tedium.
- Enjoyment is associated with academic success and mastery of technology.
- Confidence is associated with successful learning and support for learning.
- Accomplishments are described as allowing student to display their
course accomplishments regularly and publicly.
- Success has two dimensions - use of technology and conceptual aspects
of the program.
- Frustration is associated with the use of technology and the conceptual
aspects of the program.
- Tedium is associated with posting and responding to reviews on a regular
basis. Tobin (1998:155)
The following items are included in evaluating Qualia:
- I felt a sense of satisfaction and achievement about this learning environment.
- I enjoy learning in this environment.
- I could learn more in this environment.
- The technology resources enhance learning.
- I was supported by positive attitude from my peers.
- I was able to access the materials without much difficulty.
- I had no difficulty using the technology.
- I am confident in using the technology.
- I have no problems going through the materials on my own.
- I was in control of my progress as I moved through the material.
- It was easy to move about in the material.
- The web based learning environment held my interest throughout my course
of study.
- I felt a sense of boredom towards the end of my course of study.
- I felt isolated towards the end of my course of study.
WEBLEI Scale IV: Information structure and design activities
Information structure and design deals with how the web based learning
materials is structured and organised, and whether the materials presented
follows accepted instructional design standards, such a stating its purpose,
describing its scope, incorporating interactivity, or providing a variety
of formats to meet different learning styles. Included in this section
are relevance and scope of content, validity of content, accuracy and balance
of content, navigation, and aesthetic and affective aspects.
The following items are included in evaluating information structure
and design activities:
- The learning objectives are clearly stated in each lesson.
- The scope of the lesson is clearly stated.
- The organisation of each lesson is easy to follow.
- The structure keeps me focused on what is to be learned.
- Expectations of assignments are clearly stated in my subject.
- Activities are planned carefully.
- The subject content is appropriate for delivery on the Web.
- There is a logical sequence of presentation of the subject content.
- The presentation of the subject content is clear.
- The quiz in the web based materials enhances my learning process.
- The material shows evidence of originality and creativity in the visual
design and layout.
- The graphics used in the material are appropriate.
- The colours used in the material are appropriate.
- The multimedia technology (eg animation, graphics, sound, video) contributes
to the affective appeal of the material.
- The links provided in the material are clearly visible and logical.
- The links provided are relevant and appropriate to the document.
- The links provided are reliable ie no inactive links.
- The 'Help' system included in the material is context sensitive.
- The web based learning approach can substitute traditional classroom
approach.
- The web based learning approach can be used to supplement traditional
classroom approach.
Conclusion
This paper has described a new instrument which assesses student perceptions
of four core aspects of the Web based learning environment, namely, emancipatory,
co-participatory, qualia, and information structure and design. The availability
of this instrument will allow researchers and developers to evaluate the
use of the Web as a learning environment.
References
Churchland, P. M. (1989). A neurocomputational perspective: The nature
of mind and the structure of science. Bradford Book. Cambridge, MA: MIT Press.
Churchland, P. S. (1996). Toward a neurobiology of the mind In R. Lilnas
and P. S. Churchland (eds.), The mind brain continuum. Bradford
Book. Cambridge, MA: MIT Press: 281-303.
Khan, B. H. (1997). Web based instruction. Educational Technology
Publications, Englewood Cliffs, New Jersey.
Laurillard, D. (1993). Rethinking university teaching: A framework
for t
he effective use of educational technology. London: Routledge.
Palloff, R. M. & Pratt, K. (1998). Effective teaching and learning
in the virtual classroom. World Computer Congress: Teleteaching 98,
Vienna/Austria and Budapest/Hungary, August.
Relan, A. & Gillani, B. B. (1997). Web based instruction and the
traditional classroom: similarities and differences In B. H. Khan, Web based
instruction, Educational Technology Publication, Inc., Englewood Cliffs,
New Jersey, 41-46.
Sherry, L. & Wilson, B. (1997). Transformation Communication as
a Stimulus to Web Innovations In B. H. Khan, Web based instruction,
Educational Technology Publication, Inc., Englewood Cliffs, New Jersey,
67-73.
Tobin, K. (1998). Qualititative perceptions of learning environments
on the world wide web In B. J. Fraser & K. G. Tobin (eds.), International
handbook of science education, Kluwer Academic Publishers, United Kingdom:
139-162.
| Author: Vanessa Chang, School of Information Systems, Curtin University of Technology, GPO Box U1987, Perth Western Australia 6845. Email: changv@cbs.curtin.edu.au
Please cite as: Chang, V. (1999). Evaluating the effectiveness of online learning using a new web based learning instrument. Proceedings Western Australian Institute for Educational Research Forum 1999. http://www.waier.org.au/forums/1999/chang.html
|
[ Proceedings Contents ]
[ Schedule ]
[ Abstracts ]
[ WAIER Home ]
Last revised 30 May 2006. This URL: http://www.waier.org.au/forums/1999/chang.html
Previous URL 30 July 2001 to 16 May 2006: http://education.curtin.edu.au/waier/forums/1999/chang.html
Previous URL from 2 Sept 1999 to 30 July 2001: http://cleo.murdoch.edu.au/waier/forums/1999/chang.html
HTML: Roger Atkinson [rjatkinson@bigpond.com] and Clare McBeath [c.mcbeath@curtin.edu.au]
