Abstract
This paper presents the results of two studies comparing traditional lab-based usability testing and remote web-based usability testing of Web sites. Two sites were tested: an employee benefits site and a financial information site.

The remote tests used an automated technique whereby the users participated from their normal work locations using their normal browser, and there was no real-time observation. Tasks were presented to the user in a small browser window at the top of the screen that was also used to capture their input. Results showed high correlations between lab and remote tests for the task completion data and the task time data. Subjective ratings of the sites did not correlate as well. The most critical usability issues with the sites were identified by both techniques, although each technique also uniquely uncovered other issues. In general, the results indicate that both the lab and remote tests capture very similar information about the usability of a site. Each type of test appears to offer its own advantages and disadvantages in terms of the usability issues it can uncover.

An Empirical Comparison of Lab and Remote Usability Testing of Web Sites (PDF, 1 mb)