- Usability Report: The Evolution
evolved from a three-page website designed for students at
Sookmyung Women's University (SMU) in Seoul, Korea
1, to a highly interactive site with
more than 50 pages, designed to help ESL learners improve their
English reading, writing, listening and speaking skills as well
as motivate them to study English. Versions 3
and 4 of this site were developed in
partial fulfillment of the course requirements for MDDE 615 Human
Factors in Education Technology offered by
Athabasca University. Version 5 of this
website was also used in the development of a thesis: Web
Design for Effective Online Training and Instruction.
final version of this website, version 6,
has been the result of survey feedback from MDDE 615
colleagues and the application of thesis conclusions based
on a web mining analysis study and an extensive review of
usability design principles in the research
1, 2, 3,
4, 5, and final
All links for version six of ESLtown.com are functioning,
though some of the earlier version links may be broken.
Designed for Sookmyung Women's University students participating in the
General English Program (GEP II). Student photos were scanned and and
added to phrase and dialog contributions. Student feedback
was encouragingly positive.
3-frame version was designed for use in GEP III. A Learning English With the Internet
survey conducted near the end of the course indicated a high interest in
online learning, especially in ESL games, pen
pals and listening activities.
A 4-Frame version was designed as a prototype for MDDE 615 and for use
in GEP III. Feedback from course colleagues indicated that the interface
was cluttered, advertising annoying and the site
sluggish, especially the home page.
Home page clutter was lessened. Flash welcome message was redesigned to
be more eye catching. Inner pages were redesigned to 3-frames, images
shrunk, tool bar better placed and over all site
speed increased by 50%. Lessons also added.
tracking codes were added to each page to monitor page hits, paths, time
spent, links selected, and other usage stats.
Six new discussion boards and a Java chat room with meeting times were
Web mining results indicated a high percentage of visitors entered via
non-frame pages and left right away. Frames and Flash components in
navigation pages were removed. Navigation pages were also redesigned to
load in less than 10s over
- Three Frames are Usable for Lab Classes
In the following 1999,
Winter semester, SMU had clearly jumped on the Internet
Bandwagon, opening a cyber university and purchasing
hundreds of new computers to fill new computer labs.
was also requested that the Internet become an important
part of GEP III, and viewed as the resident computer guru
in the GEP department, I was asked to initiate the
development of Internet related learning materials. A new 3-frame version
of my ESL site was developed along with handouts to
instruct students how to get a Yahoo Internet-based email
Many of the
instructors in GEP had success getting students online,
success that has paid off, as most students at SMU
currently have email accounts with Yahoo, Hotmail or
Hanmail (the hottest Korean service for Internet-based
email). The 3-frame version, which was mostly text-based
and had minimal graphics, proved to be very fast and
usable on the SMU network. Students in particular enjoyed
the chat sessions at the end of our computer lab classes,
especially since none of them had never chatted online
with a foreigner.
A Learning English With the Internet
conducted near the end of the 1999 Fall semester gave the following
results using the 3-frame version:
Interest - Students
were interested in studying English on the Internet.
Design & Content -
Students wanted help in finding email pen pals and they wanted to play ESL related games. They also wanted
to learn how to chat, send email, and search for information on the Web.
More specifically, the
survey results were as follows:
Do you like learning English with the Internet?
More than 80% (116/140) of the students responded positively to this question. 37% responded VERY positively (52/140).
If you were to study English using the Internet, what would you like to DO or LEARN the most?
Students were most interesting in: (a) finding a foreign pen pal 34/140;
(b) playing English related games 33/140; (c)
learning how to use chat lines 19/140; (d) listening to recorded information and filling in the missing words 13/140;
(e) learning how to send email 12/140; and (f)
learning how to search the Web for information (in
considering the validity of these results, it should be noted that students were allowed to select only one item from a list of
GoTo Search Term Survey
A survey of key words
related to ESL and English using the GoTo
Search Term Suggestion Tool gave the following results
esl chat room
Oxford English dictionary
english as a second language
dictionary online english
oxford english dictionary online
This GoTo survey indicated an interest in grammar exercises,
dictionaries, ESL quizzes, ESL games and ESL lessons.
A similar survey conducted in April, 2000 revealed a 68% increase on the
survey word "english" and a 32% increase on the word "esl"
and from a 200% to 500% percent increase on word like "esl
quizzes," esl games" and "esl reading." This
increase was striking due to the fact that other searches with English
key word unrelated to language learning showed no increase.
enough, in the Spring of 2000, online English learning
exploded with EF's Englishtown.com, the leading provider of
free online English learning. Curiously enough, it was
providing many of the services that my survey results
indicated were important to my ESL students.
- Interface Clutter Very Distracting
In the Winter 2000 session,
it became clearly evident that SMU hoped to establish
itself as one of Korea's foremost Internet leaders. More
computers were purchased and SMU increasingly began to
host Internet-based conferences. However, the SMU network
began to show growing pains. Bandwidth was becoming
scarce. At 9:00 a.m. when all workers and labs opened for
students, the SMU network slowed down to a crawl with
access speeds at less than 14 K.
and myself caught up in the glory of Flash multimedia,
RealAudio and other bleeding edge technologies, I created
a four-frame version of my ESL website, added more
graphics and even purchased two domain names to give it a
home (myenglishbuddy.com and eslenglish.com).
site navigation menu was designed to reflect the GoTo and
LEWI survey results with games, pen pals, listening and
chat activities as the first four menu items.
site was also developed as a prototype for MDDE 615 an AU
course in Human Factors in Education Technology. A
feedback survey using QuickPoll
by course colleagues generated the following positive
- 19 of 22 evaluators found that the homepage navigation menu
was clear and easy to use
Interesting - 18 of 20 evaluators found that they were
able to find something interesting to click on very
Navigation - 17 of 19 evaluators found the use of frames aided navigation and allowed
them to quickly grasp content
On the downside, the following criticisms were noted:
Too Much Clutter -
I9 of 20 evaluators disagreed that site structured and content seemed
Too Busy - 7 evaluators
thought the site was too busy and that "there was too much
Too Slow -
Although 14 of 24 evaluators agreed that the home page loaded fast enough to keep
their attention 5 disagreed.
Advertising Inappropriate -
9 evaluators didn't like the Global English
advertising and considered it inappropriate.
Flash Component Didn't Load Properly -
Although a number of evaluators reported that the
Flash component on the home page was the best part of the
site and that "the homepage Flash audio was clear, 6 evaluators reported "clicking sounds," while 7 where unable to load any part of the Flash component.
RealSlideshow Presentation Froze
- Although 8 evaluators found that the RealSlideshow presentation played very
well, 5 evaluators found that the images froze.
As a result of the QuickPoll
feedback, myenglishbuddy.com was redesigned back to three
frames and the easier-to-remember ESLenglish.com name
branded. The four-frame format advocated based on a model
proposed by Gillani and Relan (1997) was just too much for
users. Advertising was also reduced in size and the color
scheme redesigned to make the site less visually jarring.
The Home Page and Frames Page averages of the prototype
being well over the 47.8K limit recommended by Flanders (1999) were also
reduced in size, with an emphasis on reducing image saturation in .gif
files using ImageReady. The "Home Page" was reduced by more than 55% to 35K
from 78K, while the Frames Page was reduced by 33% to 54K from 81K (it
should be noted that the above home page total does not include the 60K
But alas, the
bleeding technology bug would not let go and I redesigned
my Flash component from 60K to 66K, perhaps for no other
reason than to prove I could do it. Yes, it looks kind of
neat, but in terms of usability, it should have been in
the trash bin a lot sooner.
Web Mining with AXS, HitBox and WebTrends
To develop a thesis I was
working on, I redesigned my ESL website (cut the audio on
the Flash) and most importantly included tracking codes on
most of the pages from
and HitBox to
monitor usage and access. I also downloaded 14-day trial
versions of WebTrends Logfile Analzer and NetTracker, as
well as a free copy of OpenWebScope to analyze my log
The heart of my thesis
was a series of 200 questions I had developed for
designing web-based materials based on learning theories
and practical design strategies as researched in the
literature (I like to refer to these questions as WeBIC
... Web-Based Instruction Checklist).
With WeBIC, I wanted to find out if my students could
learn better with web-enhanced learning materials compared
with learning materials designed using a modified form of
APA writing style.
the Fall of 2000, I was quite excited. However, I quickly
ran into some serious problems. SMU, in its never ending
quest to be recognized as an Internet leader in Korea,
purchased more computers to add to its network. However,
it has not yet upgraded its main line. Currently, with
from 5,000 to 8,000 students depending on whether you
count part-timers or not, there are over 3,000 computers
connected to one measly 1 GBps line to the Internet (in
absolute perfect conditions this is about 30Kps per
computer user). What this means is that each year, as more
and more students jump on the Internet bandwagon,
more and more end up fighting for the same bandwidth.
cold practical terms, what this meant to me is that my
carefully constructed thesis project nearly fell apart at
the seams as I had to quickly redesign by learning
materials to become more accessible. Access speed
is the Internet's nemesis, and it almost bested me.
6 Speed and Navigation are King
I discovered that my
three-frame online learning materials were navigational
nightmares to ESL learners who had only a few minutes to
figure out and master their design. I found that simpler
one-frame and two-frame page layouts worked better. I also
found out a fact that bears emphasis: ESL learners do
not read English. Just as usability gurus on the
Internet preach that users prefer to scan,
ESL learners will do everything but read to figure out how
to operate an interface. They will click buttons and
get lost, look at what others are doing (if in a computer
lab), and stare at a screen blankly waiting for something
to happen rather than click on an icon that says "go
to the next page" (commercial vendors targeting
international audiences in English should heed this
My Web mining results also indicated that from 25% to 30% of visitors
entered my site via a non-frame based page and were left with too few
navigational aids to stay much longer than a few seconds. A page with no
navigation also means less content and less interest.
short, because of the sluggish SMU network, the large
percentage of users entering via non-frames page, and the
fact that ESL learners especially need simple interfaces,
I redesigned my site so that each page would load in ten
seconds less, followed a simple two column Yahooish less
cluttered design, and provided simple consistent
navigational aids, as much as possible on every page. I
also made the bold step to purchase ESLtown.com as a final
home for this site. Although Englishtown.com will not have
to worry itself too much about my presence, it does make
me quite happy to realize that ESLtown.com is four letters
less ... and that four letters more in cyber space is
about four light years more in outer space.
impressions by users and students, have been quite
encouraging. It has also made me smile a bit while
colleagues panic unable to access their Hotmail accounts,
and my new site pops up rather reliably ready to be used.
Shortly after I decided to
abandon Flash and other bleeding edge technologies,
Nielsen reported Flash:
99% Bad (October 29).
think that it is important that all web designers
experiment with Flash and other bleeding edge technologies,
as well as various frame designs, but when it comes down
to creating usable web media, one-frame is best for
commercial sites with two frames a possible choice for
more educational sites that are not too worried about
users accidentally entering a non-frames page through a
edge technology bug has been cured. But a possible new
affliction looms ...
Web mining is another bug
that has recently caught hold of me. Unfortunately, most
web mining tracking codes like the ones I used from
and HitBox hog
bandwidth just like large graphic files, freeze downloads,
and provide results that are far from accurate. Sure the
nifty free AXS script is amazing for tracking outside
links and the hundreds of HitBox stats cool, but do I
really need this information?
the next year or two, I will go back to the basics:
content, speed, navigation and simplicity, and search for
a free tracking code that loads in a flash so I can learn
a little bit more about this new bug before abandoning it
with the others.
Flanders, V. (1999). How big can I make my page? Retrieved February 5, 2000 from the World Wide Web:
Gillani, B.B. & Relan,
A. (1997). Incorporating interactivity and multimedia into
web-based instruction. In B. H. Khan (Ed.), Web-based
instruction, (pp. 231-237). Educational Technology
Publications: New Jersey.