A recent study is so flawed that I just had to comment.

Catching up on my backlog, I read a BBC news report on a study published in the journal Behavior and Information Technology about the speed at which web users determine the quality of a particular site. The article, First impressions count for web [sic], draws the erroneous conclusion that visitors to a web site make up their minds about it within 50 milliseconds. If you think that this is completely false and misleading, believe me, it is.

I will not pretend that I have read the full study, as I am certainly not going to waste money on the publication just for this purpose, but according to the facts in the short BBC writeup, the researchers made at least five critical errors that invalidate any results:

1. An image shown for 50 milliseconds is seen for a longer time.

It is physiologically understood that the retina retains an image for a certain amount of time after that image is viewed. This is the characteristic of our eyes that allows us to watch movies without the perception of flicker. In order for the researchers to get feedback on a site image, the participants had to be aware that an image was about to be shown. When viewed, this image would definitely stand out against the static view before and after. The cones (color receptors) in our retinas hold the image longer than indicated.

Additionally, the amount of time that the image was shown, one twentieth of a second, is actually fairly long in the whole scheme of vision. This is 20% longer than a frame in a (modern) motion picture, and 50% longer than a full NTSC television frame (consisting of two interlaced fields). Perhaps more to the point, it is at least twice as long as an interlaced CRT frame and a minimum of three times as long as a non-interlaced frame on a CRT. In other words, it is clearly viewable, especially in contrast to solid black or white.

2. Web pages do not just emerge and appear fully formed.

When a user is loading a web page, it takes longer than 50 milliseconds to be fully displayed and, in most cases, the page arrives in pieces, not as one complete image. As each element arrives on screen, the brain has more data to process to supplement or dispel any preconceived ideas (including those determined a split second earlier). The study eschews this entire internal mental process.

3. An actual web user has expectations based on an action.

In the real world outside the laboratory, web pages arrive as a result of a user action, usually clicking on a link or entering an URL. Based on the action, and the motives behind it, the user has certain expectations of what he or she will see on the page. Additionally, every individual has a “frame” (per Marvin Minsky) for what a web page (or specific type of site) looks like, which forms the foundation of such expectations. Only the latter, in a very general sense, applies for a clinical study.

4. The study assumes that aesthetics is the only measure of quality.

This one seems pretty self-evident to me (pun intended). When going to a web site in a natural setting, I am looking for something, whether that be information, entertainment, stimulation, or some other desire. While I may be less inclined to pursue my goal on an ugly page, the quality of the site is determined by other factors as well. In a sterile environment with no motivation, the only measure is aesthetics.

5. Nothing indicates that a determination is made that quickly.

Even if one determines that the other errors are somehow irrelevant, the fact remains that there is nothing at all to prove that, as the story states, “Internet users make up their minds” as quickly as advertised. In fact, it is a much more logical conclusion that the viewed image is retained briefly on the retina, certain key elements are committed to short-term memory, and the actual determination is made, relatively speaking, much later, perhaps only when the research question needs to actually be answered.

To be clear, I have no doubt that most web surfers do not spend much time looking at a web page, so it is important to make a good impression, and all other things being equal, one should opt for aesthetics and simple layouts. However, this study does nothing whatsoever to prove that, and the conclusions, whether from the researchers or the story authors, are simply incorrect.

What this study does prove is that the human brain is able to formulate an emotional response to a visual image rather quickly. A interesting bit of research at this point would be to determine whether the concept of aesthetics as interpreted here is innate or learned. Of course, I doubt that we will be able to resolve the nature versus nurture debate over this issue.

I wonder aloud who funded this research and how much money it cost to do the study…