Web Usability Testing of Current and Prospective Students
This week we are conducting Web usability testing on the new site design with current and prospective students. We had two people from BarkleyREI on campus on Tuesday to test the new site with current students and we are testing prospective students at BarkleyREI’s headquarters in Pittsburgh throughout the week.
Before I get into it, I suppose I should explain what usability testing is and why we are doing it (and why it is really important). Web usability guru Jakob Nielsen describes usability testing as “a quality attribute that assesses how easy user interfaces are to use. The word “usability” also refers to methods for improving ease-of-use during the design process.”
The aim of our usability testing is to evaluate the new Web site design and information architecture with end users. Up until this point we’ve followed industry best practices in developing our site strategy, the site’s information architecture, and in designing the user interface for the site. So each step of the way we’ve made numerous assumptions and usability testing allows use to get direct input on how real users of the site think and respond to the design, content, and how we’ve organized the site. Does the site meet it’s intended purposes, what is its ease of use (in particular sections or as a whole), are users in each group able to complete expected site tasks, are we seeing any trends from one user test to the next.
We are testing 5-6 people in each user group (or audience) for a total of 10-12 user tests. This is more than enough of a sample size to begin to see trends, something that Jakob Nielson discusses on his site in an article entitled “Why You Only Need to Test with 5 Users.”
How User Testing Works
The actual usability testing is done using Morae usability testing software. All candidates are pre-screened against certain criteria before scheduled for testing and then screened again by the test administrator just prior to their assigned testing session. Since this is human testing, all participants must sign a consent form before the test begins since the entire session is videotaped and all screen movements are captured. Test participants are compensated for their time. A observer, who is located in an adjacent room, takes notes during each test session. Test subjects are asked their general impressions of the new site and if they are familiar with the current site, their feelings on that as well. Test participants are then asked to complete a series of tasks, these are tasks that people in a given audience group should be able to complete easily. For example, for prospective students, we might ask them where would they go to request information, apply, schedule a campus visit, find a list of majors, etc. Surprisingly, you can begin seeing trends as early as the first 2-3 tests.
After all the user tests are completed for both groups, a highlights reel and final report is produced. The data gathered during this process then allows us to make changes to the site that address any shortcomings.
Based on the preliminary current student test results (from what I observed), the new site tested very well. The general impressions of the site we very favorable and in most cases, not what current students would have expected from Saint Anselm College. It will be interesting to see the results of prospective student testing and how that compares. Surprisingly, the current Saint Anselm College site tested very well with its current design and organization of information, with a few known trouble areas (or complaints), both of which we are addressing during this redesign project, specifically, a more robust campus calendar and search engine.
I’ll report back on this blog on the final results for both the prospective and current student groups.
What About Faculty, Staff, and Alumni User Testing?
At this time, we are not planning to conduct user testing for faculty, staff, and alumni. Some of this has to do with available budget and time in our project schedule – usability testing is expensive and very time consuming. We’ve chosen to test two important audiences this go around – prospective and current students.
Alumni have their own Web site, AlumNet, which will only be reskinned with the new design template (the navigation will not change). The faculty and staff audience, since they are a smaller audience and an internal constituency, can always be “trained” on how to use the new Web site if need be, something we aren’t afforded with prospective students. If a prospective student can’t easily find how to apply, visit campus, or request information, there are real consequences for this and an impact on the college’s bottom line given our heavy reliance on tuition.
Comments
One Response to “Web Usability Testing of Current and Prospective Students”
Leave a Reply
Hi Doug,
I like your well thought out explanation of why user testing is important. Here is an article from Website Magazine http://bit.ly/32mqlQ
that reviews usertesting.com (which I am affiliated with) and other tools you may find of interesting.
Amanda