 |
Department of Computer
Science
|
Experiences using a web-based tool for course support
Scott Hazelhurst
Department of Computer Science
Postscript version for printing
1. Introduction
The University has identified the use of information technology as a
strategic issue in teaching, research and administration. The
wide-spread use of the internet has raised the possibility of internet
technology being used in the delivery and support of courses offered
by the University. Various suggestions have been made - with varying
degrees of enthusiasm - on how this technology can be used.
In 1998, the Senate Committee on Student Development granted me R2000 to
explore the use of a web-based tool for the delivery and support of
course material. The objective of my work was to explore how web-based
tools can be used, and what the advantages and costs
are.
The tool was used in presenting the course Data Abstraction and
Algorithms II in 1998. The course was taught twice:
- Over a ten week period to the Computer Science II class
(approximately 100 students), lectured by me and with support by
about five student tutors and demonstrators;
- Over a seven week period to the Higher Diploma in Computer
Science class (approximately 15 students), lectured by Dr Vashti
Galpin and with support by one student tutor.
The tools was used in different ways by each lecturer. Other
lecturers expressed interest in using the tool, but for reasons of
time pressure did not.
Section 2 discusses the tool that was used, the
features and costs. Section 3 discusses the
experiences of using the tool, and Section 4
concludes with some recommendations. In summary the conclusion and
recommendations that this report makes are:
- There is potential in the use of web-based tools for course
support;
- We should not expect any significant cost savings, at least in
the short-term;
- We should have a plan with sound educational objectives;
- The University should choose a particular tool for some pilot
studies (some important technical criteria for this tool are
suggested in the report);
- Resources should be placed into supporting these pilot
studies -- these resources are modest but are necessary if we want
to do things properly;
- We should have a central resource for information on such tools.
2. The tool used: WebCT
The tool that was used is called WebCT, developed by WebCT Educational
Technologies, a spin-off company from the University of British
Columbia. A number of other tools are available, but when I made the
choice I did not have time to do a proper comparative survey.
Note: This report is based on the use of WebCT. While the
discussion does focus on WebCT, I believe that many of the later
comments are generic in nature. While my experience of WebCT was
positive, this report is not meant as an endorsement.
There is a large amount of documentation and published papers
concerning the use of WebCT, most of which can be found on the
web1.
Section 2.1 gives on overview of the features of
WebCT; Section 2.2 discusses setup and system
requirements; and Section 2.3 describes how we used
WebCT.
2.1 Overview of features
Some of the features that WebCT provides are:
2.2 Set-up and requirements
WebCT runs on a server machine (either Unix or Windows
NT)2. The machine need not
be dedicated to WebCT use. The users (whether the instructors or the
students) interact with WebCT using a standard web-browser on any
machine (Netscape 3 or above). Unless limitations are placed on the
machine, a user can access the server from anywhere on the internet
To set up WebCT on the server requires reasonable but not highly
expert knowledge of the system running on the server. The typical
system administrator should be able to do this easily. From then on,
all interaction is done using the web. One person acts as
``Administrator''. Using a web interface the administrator can create
a new course and give the login and password to the lecturer who will
be responsible for the course. Using a web interface the lecturer can
create the course material, create accounts for the students and
tutors, and perform any administration relevant to the course. The
individual students access the account through the web.
The WebCT tool provides facilities for the production of course
material. Thus no skills beyond using the web browser and WebCT are
necessary. However, for productive use it is desirable that the bulk
of the course material is produced in a reasonable format (preferably
HTML) and then uploaded to WebCT. Many standard word processors (e.g.
LATEX, Microsoft Word, ...) have facilities to produce the
document in HTML format and there are a number of other products of a
range of sophistication that can be used to produce HTML code directly
(e.g. FrontPage, emacs).
The learning curve is reasonable so a lecturer can start to produce
reasonable material with minimal extra skills, and as they learn more
(e.g. HTML, Java etc) they can get as fancy as they like.
WebCT is provided on a time and user-based licence. There is no cost
for downloading the WebCT system, experimenting and creating the
course material. The charge is based on the number of student accounts
created (a student registered in three courses concurrently would need
three accounts). We had a four month, 200 person licence, which cost
us US$300. A year's licence that would more than cover the needs of
the Department of Computer Science costs US$1250. A single-server,
unlimited user, 12 month licence costs US$3000. The WebCT
documentation indicates that discounts for multiple licences may be
possible, and that for renewing customers prices will not increase by
more than 5% per year. I tried to find prices for competing products
but without much success.
While these costs are not trivial, the costs of the software are quite
manageable for us (and we might be able to press some vendors for
`developing country' prices) and do not constitute a barrier.
The costs in using such technology come elsewhere:
- The cost of the hardware and operating system for the server machine;
- The cost of the hardware and operating system for the user machines; and
most importantly
- The cost of human resources.
These costs require serious consideration and are discussed later in
the document.
2.3 Our use of WebCT
The most important part of the course material on WebCT were the
lecture notes for the course which had previously been produced by a
lecturer in the Department3. The original document was
in Microsoft Word, and was converted semi-automatically into HTML
format. A simple glossary and index were created for the material and
few self-tests were added.
WebCT was used for a few tests, administered on-line to students in
their laboratory groups. The lecturer evaluation was conducted on-line
using this facility.
Laboratories, tutorials assignments and extra material were placed on-line.
We ran WebCT on a fairly old Unix workstation that is used as a
general compute server in the mathematical sciences. The performance
of the server did not appear to be a problem. The lecturers used
Netscape on Unix machines, Macintoshes, or PCs running Windows 95; our
students ran Netscape on in the lab on Macintosh PowerPCs, and those
who have internet access at home could have used any machine.
This section reports some of our experience with WebCT. Unfortunately,
most of our experience is anecdotal as we did not have the resources
to conduct a proper study.
- Technically the system was easy to set up and maintain. It was
reliable during use. We ran into some problems in the labs as the
memory of the machines was not sufficient to run both WebCT and the
programming environment at the same time conveniently.
- There is a significant cost in the development of course material.
- It was reasonably easy to take existing notes and convert it into
WebCT form. However, it did take time (a student could be paid to do
this). It will be easier to convert material where the material was
created knowing that it would be converted into HTML format. More
tools will become available that convert conveniently from one
format into another.
- There is only a small benefit to be had in having the bulk of
the lecture notes on-line as for most purposes the printed notes are
more convenient. Having the material on-line provides the ability
for the students to find links between material, use the glossary
and indexing facilities, and to view the material in a non-linear
way. It is also convenient from the lecturer's point of view when
correcting mistakes.
- It was useful to place the labs, tutorials and assignments
on-line as it made it easier for the students to access. This is
especially true for labs and assignments were often I wanted to
provide programs or extracts of programs to students. By publishing
the material this way, it was easy for students to use the material.
- While I believe there will always be a place for traditional
linear presentation of material, I felt that the benefit of
hypertext presentation was shown, even for the simple examples I
used. For example, one of the central concepts of the course being
taught is abstraction. It is difficult to get the point across using
linear presentations of examples, whereas it is often natural to do
using hypertext. The underlying principle is that different students
at different times need different views and paths through the
material: linear presentations cannot provide this.
- There is clearly lots of opportunity for using multimedia and
having interactive material. I didn't do much of this.
- One of the most useful features I found was student tracking --
being able to see which students had seen what. For example, when I
handed out the assignment I could see how soon students started
working on the assignment. This let me give convincing feedback to
the class. Another example was some tutorial material I prepared in
response to questions from the class about difficult material. I
could see exactly who had and hadn't spent time looking at this
material. Thus when students came to see me I could use my time
productively: those who had used the material I would try to help;
those who hadn't were sent off to use WebCT, saving me many hours of
work and plenty of aggravation.
The underlying principle here may be that a successful tool will
support the staff as much as its supports the students.
- The use of the testing facilities worked very well. I only used
the multiple choice testing features. The tests were easy to
prepare4. It is possible for WebCT to present the
questions in different order to different students or to choose
from a bank of questions so that people sitting next to each other
don't see exactly the same test.
It is also possible to annotate each answer with an explanation of
why it's right or wrong. Once marked the student gets back their
test, with the mark and for each question whatever explanation has
been provided about the answers. This made the tests a more valuable
learning experience for the students.
The tool enables you to set a time limit for each test as well as to
restrict when the test can be taken. The students reacted very well
to the testing facilities and liked getting the feedback very
quickly. I have some security concerns, but I believe that these can
be addressed in the long term. The real problem is building up a
good database of questions.
- The use of the anonymous survey facility for the lecturer and course
evaluation worked very well5. From a lecturer's point of view it is convenient to
run, and the answers (both multiple-choice and open-ended) are very
nicely tabulated and presented. The presentation of the open-ended
questions' answers is much more useful than having 100 slips of
paper because it can be neatly printed out and so patterns are easier to
see. The one drawback of the system is that you cannot get
comparative figures that ADC can provide. An advantage of the system
is that once the database of questions is drawn up it will be easy
to get feedback in the middle of the course.
- The use of multimedia will extend the types of questions that
can be asked. This will require more work, but it also may make
multiple choice type questions more useful in wider domains.
- The use of the student presentation facility worked well. We
used these presentations for submission of assignment material (this
is useful for assignments that comprise different parts, like
programs, reports, etc.). The markers for the assignment can view
the presented material when marking the assignment.
4. Conclusion and Recommendations
Overall, my experience was positive and I believe that Wits as an
institution should put some resources into developing web-based
education, based upon our strategic plans and the educational
objectives we see we will gain.
Before making specific recommendations, I have a few
general points.
- There is a tremendous amount of hype about web-based
education. Just because there is a lot of hype doesn't mean that we
shouldn't take it seriously; however there are dangers in the
bandwagon effect, especially when we don't have lots of
resources.
- We must always be clear about what our objectives are and be
critical in what we do. We should have a plan with educational
objectives. While moving to web-based education for PR reasons is
quite legitimate, but can only be justified if we get more out of it
than we put into it.
- I do not believe that we shall see any savings by moving to
web-based education. We should see the use of this technology as
enabling us to do things better, in new ways, or expanding what is
possible.
- The development of course material is not cheap. My experience
was that on a cost-benefit analysis I do not feel I got a good
return for the time I put into developing the WebCT material, and
ultimately would not have been worth doing if I did not have broader
goals in mind.
I do believe there are real benefits to be gained in providing
better courses and a better learning environment for students. But
these benefits will require that courses are planned with the use of
this technology in mind, and the resources can be justified only
where the extra (human resources) costs can be amortised over
several offerings of the course.
- The 1980s was the decade of computer-based education (CBE and a
host of other acronyms). There were some successes with CBE and some
expensive failures. It is true that web-based education provides
certain things that CBE didn't. But it will be a grave failing to
ignore the experience of CBE, and there is lots of literature to
draw upon.
- My comments focus on courses where web-based tools are
used to supplement an existing course. I have not really explored
stand-along courses (i.e. courses which primarily depend on
web-based techonology, but I believe that the above warning apply
even more strongly. Moreover, I believe that Wits should focus on
its strengths as a residential university with good tutorial and
laboratory support.
On a small scale, Departments with reasonable technical expertise such
as Computer Science or Electrical Engineering can adopt this
technology with very modest investments and the new technology would
not put any significant extra burden on technical staff.
Central resources will be needed for effective wider use, but
these would be modest in relation to the objectives (if the objectives
are met by the technology). I believe there is enough evidence to
support some pilot studies, and that the cost of these pilot studies
is justifiable based on the possible benefits.
From the user side, this assumes that staff and students have
access to computers. New entry-level machines (whether Pentiums,
Macintoshes or whatever) will be able to meet the needs adequately. In
the short-term there may be some problems with older machines but
natural attrition will deal with this.
The real costs will be in the human resources required to develop and
maintain educational resources. There are many examples of successes
using this technology as enthusiastic, bright people put lots of
resources in doing fun things. But this doesn't mean that this is the
most productive way of using resources, or sustainable in the longer
term.
There is potential for the use of this technology and we should take
some steps to evaluate how the technology can be used effectively.
Having some central resources to minimise the duplication of resources
is very desirable.
We should pick a
web-based technology for adoption in the University. This would not be
prescriptive in that people should be allowed to adopt other
technologies if they want, but the University should provide some
support for the chosen tool. Some subsidy would be given to people
using the tool (through the purchase of site licences), and someone
should be given the task of gaining experience in using the tool so
that academic departments that adopt the tool would have technical
support.
I have had positive experience with one tool, but clearly a study
needs to be done to make a choice. A number of other Universities have
done similar studies so we can draw upon these and we need not
reinvent the wheel. There are educational criteria in terms of
functionality and ease of use that can be found in the
literature. There are also technical criteria that we should bear in
mind. Some are:
We may need to tie ourselves to a particular vendor for the web-based
tool, but we do not want to tie ourselves to particular hardware or
operating systems platforms. There is some effort in standardising
this type of system (the IMS project) and whatever platform is chosen
should have a commitment to IMS.
We should have a resource centre
for case studies which contains the international literature on the
subject as well as local experience. Someone should be mandated to
keep this material and to keep in touch with initiatives such as the
IMS project. We need to share information so that we don't duplicate
efforts.
We should identify courses that could
benefit from web-based technologies. These should be courses that have
reasonably large registrations, where computer labs are available, and
where the course is likely to stable for at least three years. How
much resources we put in should depend on what importance the
University places in this. I would suggest the purchase of a
reasonable server machine, the cost of licence fees for selected
projects and some technical support for a selected range of courses
for a three year project
- The first year would be the development of course material;
- The second year would be refining and adding to the course
material and doing a prototype study;
- The focus of the third year should be a proper study to evaluate
the benefits and costs of the approach.
The objectives of these studies are to build capacity in the area and
to learn how to use the technology effectively.
Although there is potential for
distance education, I recommend that we leave this for the future
until we have more experience. Rather we should concentrate on what we
do well, better. There may be room for some pilot experiments - for
example some of the admissions tests could be done remotely, but we
should focus our energies appropriately.
Vashti Galpin and Philip
Machanick gave a variety of help, Gloria Aikhorin cleaned up the web
pages, and the Department of Computer Science contributed financially.
This is an appendix to the report I wrote on 7 February 1999 for the
Senate Committee on Academic Development.
On 17 and 18 June 1999 I attended the First Annual WebCT Conference on
Learning Technologies held at the University of British Columbia,
Vancouver, Canada.
The conference was very well attended with well over 500 very
enthusiastic delegates from many different countries attending. Most
of the delegates seemed to come from the equivalents of the Academic
Development Centre, though there were also a number of subject matter
lecturers and representatives of publishing companies. I met
delegates from the Universities of Stellenbosch and Pretoria and
the South African distributor of WebCT.
It is clear from the presentations and posters that the use of
technology such as WebCT for the delivery of course material has
grown dramatically all over the world. Compared to previous
technologies (such as television and traditional computer-based
education) there is a qualitatively much deeper and broader
penetration and use of the technology.
Many exciting and interesting case studies were presented at the
conference. A number of universities now have extensive investments in
courses with a significant component of web-based material. There was
a remarkable range of different types of courses and ways in which
the web was used in the courses. Some courses were completely
web-based, and some used the web to support existing courses.
A minority of the courses appeared to be intended for distance
learning.
One of the things that I was interested in was the model which the
universities adopted for the introduction of the technology. My
subjective impression is that at most institutions the initial
introduction was `bottom-up' having been initially used by a few
individuals before wider adoption. There were some case studies
of `top-down' approaches where the university executive had been
responsible for the introduction of the technology and encouraging
departments to use it.
Two developments that appear positive are:
- The adoption by leading book publishers such as Pearson's
(Addison-Wesley, Prentice-Hall) and others of this technology. I
attended a talk given by a representative of Pearson's which
described how they would start to supplement text books with
web-based course material. The costing model appears attractive.
- Work on integrating the web-based teaching technology with the
back-end functions to integrate these new systems with student
management systems (registration and mark management), which should
bring a number of administrative benefits.
Two negative aspects came out:
- I saw relatively little analysis of the effects of the use of
the technology. The only reported evaluation I saw was surveys of
staff and students. I was particularly interested in whether there
had been any proper evaluations of the effectiveness of the
introduction web-based systems, and I saw none presented at this
conference.
- The question of costs needs thought. As discussed in my original
report, although the costs of the hardware and the licences needed
to run web-based systems are not negligible they are reasonable. The
real costs are in the development of course material. A figure
quoted in one of the talks I attended was the cost of the
development of a full web-based course (which I presume was a
replacement for a traditional 39 hour course) was US$50 000. In
discussion after the talk, a number of other delegates said they
felt this was a reasonable estimate. Costs of courses which
supplement existing material are much cheaper but still expensive.
No studies were presented about the cost-effectiveness of using
web-based system. I asked a number of people about their
experiences, and none said that the use of the technology had saved
costs.
In summary, I was positively impressed by what has been
achieved. However, we must take care not to be swept away by the hype
that surrounds the technology since the benefits and costs have not
been properly evaluated. My experiences at this conference
have reconfirmed the recommendations I made in my report. In
particular:
- There is potential in the use of web-based tools for course
support;
- We should not expect any significant cost savings, at least in
the short-term;
- We should have a plan with sound educational objectives;
- The University should choose a particular tool for some pilot
studies (some important technical criteria for this tool are
suggested in the report);
- Resources should be placed into supporting these pilot
studies -- these resources are modest but are necessary if we want
to do things properly;
- We should have a central resource for information on such tools.
In addition, I think it is important that we use the pilot studies to
do proper evaluations of the use of the technology in our environment.
As a concrete proposal, I suggest that:
- The University purchase a reasonable licence (US$1000-3000)
for WebCT, and a well-configured Pentium-based Linux
(R15000-R20000). As a short-term measure the Department of Computer
Science would be prepared to host this server. The question of its
permanent home needs thought, and depends on what licensing model
the University adopts in the long run. My view is that ADC would be
a suitable home for central servers, though some individual
departments may wish run their own servers.
- Some short training courses be offered in the University for
interested academics.
- Any department which wanted to use the central server be allowed
to do so (up to the limit of the server) free of charge on condition
that they run an evaluation of their use of WebCT.
I have the proceedings of the conference - unfortunately it was
damaged in transit so is not suitable for putting in the library but
if anyone wants to look at them, they are welcome to contact me.
Experiences using a web-based tool for course support
This document was generated using the
LaTeX2HTML translator Version 98.1p1 release (March 2nd, 1998)
Copyright © 1993, 1994, 1995, 1996, 1997,
Nikos Drakos,
Computer Based Learning Unit, University of Leeds.
The command line arguments were:
latex2html -split 0 -ascii_mode -no_navigation -no_images -no_subdir -show_section_numbers -no_auto_link complete.tex.
The translation was initiated by Scott Hazelhurst on 1999-10-28
Footnotes
- ...
web1
- http://www.webct.com/webct or
http://homebrew.cs.ubc.ca/webct
- ...
NT)2
- WebCT Educational Technologies will host a course on
their machines, but given the slowness of internet lines this
clearly is not a practicable solution for us.
- ... Department3
- From Data Structures to
Algorithms in C++ by Philip Machanick
- ... prepare4
- By which I mean that once you know what questions
and answers you want to ask, it is easy to upload the questions. I
am not suggesting it is easy to create the multiple choice questions
in the first place.
- ... well5
- In fact it is not quite
anonymous as you can see who has and hasn't responded. With the
connivance of a system administrator and a lot of effort the
lecturer could work out the identities of some of the authors of
some of the answers. But no more effort is required to match
handwriting on the ADC surveys with exams, so I don't see this as a
real issue.
Scott Hazelhurst
1999-10-28