Measuring Web User Experience

A hardworking Web developer who wants to please or appease the client, you've built the best site you could in the given circumstances. Perhaps you've even built a good site without the clear goals, defined vision, and essential content that were mysteriously absent in client meetings!

A couple months after completing the site, things tend to go back to normal. The remnants of quality all-nighters are gone. The empty cola cans and greasy fast food cartons have been thrown away and the rolled-up carpet under your desk that was used for a pillow has been flattened. That's when the same client comes strolling back into your life—looking well rested too, but also worried. The client asks very politely if her site is a success. You laugh for a second (using your inner voice so as to not offend the client) until you realize you don't know either. Or worse yet, you don't know where to even begin evaluating the success of a Web site.

In this article, I’ll discuss the importance of several tasks:

  1. Using log files
  2. Establishing goals and objectives
  3. Using formulas to quantify success
  4. Defining success through focus groups and surveys

Log Watching

The first place many Web developers and businesses look when evaluating a site's success is the log files. Log files are text files compiled by servers which constantly and diligently record numerous aspects of a visitor's trail on a Web site. From pages and images requested to the type of browser and IP address being used by viewers of the site, all aspects of the user's experience that are tallied in lifeless lines of information. It's not until a software application renders the information into a "human friendly" context that the information becomes understandable, but you still need to know your log file terminology.

Log File Terms

"A log file evaluation based on traffic can help if you can compare traffic to some other temporal reference point."

Establishing Goals and Objectives

Establishing the site’s goals before you begin development is a very good thing, especially when a crucial thing like getting content for a client's site often seems to slip when development starts.

When developing goals for your site, remember that a goal is generally a relative statement. You can't assign hard numbers to the goal, but they are great for laying down the groundwork for more specifics. An example of a qualitative goal for a Web site would be "to increase the sales through your site's affiliate," or to "have more users register for the sales promotion," or to "fill out the warranty registration form." With these goals established, you can begin to tell if your site is success or not.

"You can't assign hard numbers to a development goal, but they are great for laying down the groundwork for more specifics."

Determining how successful and by how much leads us to the next step: objectives. An objective is a quantitative measurement of a goal. Examples of objectives are: "a $200 increase in commissions," or "250 users sign up for the sales promotion," or "a thousand visitors to the site to fill out the warranty registration form." In other words, you assign hard numbers to your goals. With these hard numbers, you are in essence building a strong measuring stick.

Formulas and Feedback

Now that we've set up goals and objectives, the next step is to look at how we can gather numbers or other feedback to measure. Along with the log report that you've already analyzed, we can do a couple of quick number crunches.

Goal Percentage Formula:

Unique Number of Visitors performing task successfully

Total Visitors
Goal Success Percentage

For example, over the past 24 hours 30 people filled out a sample request form, but I had a thousand people view that page. That means my site was 3% effective. There's plenty of room for improvement, but that's not the impressive bit of knowledge. I now know I have to help my client meet a goal to get those sample request submission numbers up.

Also note that if your site is something beyond marketing “brochureware,” it's good to have more than one goal for a site. If you have more than one goal, don't forget to find the average of the goals to find a site average. That way, you can summarize the site's growth to your manager or client as a value of average site success.

Visitor Worth formula:

How much money your site made (or lost) last month a.k.a. profit (loss)

Total visitors
Value of each visitor

For example, if your site made two hundred dollars profit and you had six thousand visitors to view your catalog of Café Press t-shirts, the value of each visitor is a bit more than 3 cents. This formula is great if your business is strictly a "click and marketing" site without a physical store.

If you do have a physical store, your business needs to look beyond how much a site visitor is worth to you. You have to think of your Web site as multifunctional: able to perform marketing, brand awareness and selling (among other things). And that requires more factors about your business and a lot more math. Other methods of getting results from user experience aren't as math intensive.

Think of your Web site as multifunctional: able to perform marketing, brand awareness and selling (among other things).

Focus Groups and Surveys

Focus Groups

If you can't seem to increase that three percent success rate of the sample request form, then focus groups or usability lab tests are a way of seeing how visitors are fumbling through your site. Focus groups are ways of getting input, by having a small sampling of users from your target audience interacting with your Web site, helping you understand your product or your company's new tagline.

The beautiful thing about focus groups is that they allow users who aren't close to the project to give a review the work. Instead of accepting your boss or co-worker’s comments on their surfing habits as proof of building a site's navigation, you get unbiased feedback.

Online Surveys

This is one method that I wish more sites had. Creating and hosting focus groups and usability testing labs can get expensive. As an alternative, you can obtain feedback by putting up a simple online survey form that asks questions to users about your company and its services. True, it won't be an honest sampling of your audience, but you are providing a means for your site visitors to speak beyond the trail in the log files. Your log files may not lie to you, but they can't tell you as eloquently as a visitor typing something like this:

"Your personal Web site's puke green background color is a joke and you should consider a second career choice. Oh, and if you would print more Web Design mousepads I would buy them in a heartbeat for me and all my loved ones."

Since the form requires the visitor to type in a legitimate email address (that gets checked by a script on the server), I have a chance to follow up with her background color choice for my site as well as a pre-order form.