Evan X. Merz

Measuring the performance of a webpage is an extremely complex topic that could fill a book. In this article I'm going to introduce some of the commonly used tools for measuring web performance, and show you why you need to take multiple approaches to get a complete picture of the performance of any webpage.

Why measure webpage performance?

Why measure webpage performance? Performance is so critical to the modern web that this question seems almost comical. Here are just a few reasons why you should be measuring and monitoring webpage performance.

1. User experience impacts Google search rankings via Core Web Vitals.
2. Page load time correlates with conversion rate and inversely correlates with bounce rate.
3. Pages must be performant to be accessible to the widest possible audience.

Four approaches to measuring performance

No single tool is going to give you a comprehensive perspective on webpage performance. You must combine multiple approaches to really understand how a page performs. You must look at multiple browsing patterns, multuple devices, multiple times of day, and multiple locations.

In this article, I'm going to talk about four different approaches to measuring the performance of a page and recommend some tools for each of them. You must use at least one from each category to get a complete picture of the performance of a webpage.

The four approaches I'm going to talk about are...

1. One time assessments
2. Live monitoring and alerts
3. Full stack observability
4. Subjective user tests

Why do we need multiple perspectives to measure performance?

People who are new to monitoring web performance often make mistakes when assessing web performance. They pull up a website on their phone, count off seconds in their head, then angrily email the developers complaining that the website takes 10 seconds to load.

The actual performance experienced by users of your website is an important perspective, but there are dozens of things that can impact the performance of a single page load. Maybe the page wasn't in the page cache. Maybe the site was in the middle of a deploy. Maybe hackers are attacking network infrastructure. Maybe local weather is impacting an ISP's network infrastructure.

You can't generalize from a single page load to the performance of that page. I've seen many people fall into this trap, from executives to marketing team members. It makes everyone look bad. The website looks bad because of the slow page load. The developers look bad because of the poor user experience. The person who called it out looks bad because it looks like they don't know what they are doing. The data analysts look bad because they aren't exposing visualizations of performance that other team members can use.

So read this article and fire up some of the tools before sending out that angry email.

Please note that the tools listed here are just the tools that I have found to be effective in my career. There are many other options that may be omitted simply because I've never used them.

One time assessments

One time assessments are the most commonly used tools. One time assessments can be run against a production webpage at any time to get a perspective of the performance at that moment. One thing that's nice about these tools is that they can be used effectively without paying anything.

PROS:

1. Easy to use
2. Easy to quantify
3. Fast
4. Reliable
5. Affordable

CONS:

1. May lack perspective over time
2. Lacks perspective on actual browsing patterns, including subsequent page loads
3. Lacks perspective on other locations
4. May lack information on the source of an issue

TOOLS:

1. PageSpeed Insights
2. Chrome Audit/Lighthouse
3. WebPageTest
4. GT Metrix

Live monitoring and alerts

The performance of a webpage can degrade very quickly. It can degrade due to poor network conditions, an influx of bot visitors from a new ad, a influx of legitimate visitors during peak hours, or from the deploy of a non-performant feature.

When the performance does degrade, you need to know immediately, so you can either roll back the deploy, or investigate the other factors that may be slowing down the site.

Notice that price isn't listed as a benefit (pro) or drawback (con) of live monitoring tools. You generally need to pay something to use these tools effectively, but usually that price is less than $100 a month, even on large sites.

PROS:

1. Real-time notifications of performance changes
2. Easy to quantify
3. Can be configured to request from other locations

CONS:

1. Limited information
2. Lacks perspective over time
3. May lack information on the source of an issue
4. Fragile configuration can lead to false positives

TOOLS:

1. StatusCake
2. Pingdom

Full stack observability

Some tools offer insights into the full page lifecycle over time. These tools are constantly ingesting data from your website and compiling that data into configurable visualizations. They look at page load data on the client side and server side using highly granular measurements such as database transaction time, and the number of http requests.

If you wanted a single source of information to measure performance on your site, then these tools are the only option. They can provide one time assessments, monitoring, and backend insights.

One big problem with these tools is that they are quite complex. To use these tools effectively, you need developers to help set them up, and you need data analysts to extract the important information exposed by these tools.

PROS:

1. Includes detailed breakdowns that can help identify the source of performance issues
2. Includes data over time
3. Highly granular data

CONS:

1. Difficult to use
2. Expensive
3. Requires developer setup and configuration

TOOLS:

1. New Relic
2. Sentry

Qualitative user tests

The final approach for measuring the performance of a webpage is subjective. In other words, you must actually look at how your site performs when real people are using it. This can be as simple as opening a website on all your devices and trying to browse like a normal user, or you can set up time to interview real users and gather qualitative information about their experience.

I once worked at a company that required developers to attend in-person user tests every two weeks. This allowed every developer to see how users actually browsed and experienced their work. This may be overkill for most companies, but it's a perspective that can't be ignored.

PROS:

1. No additional tools are necessary
2. Exposes actual, real world user experience
3. Exposes issues raised from real browsing patterns including subsequent page loads

CONS:

1. It's easy to prematurely generalize from a small number of cases
2. Can be expensive and difficult to do well
3. Difficult to quantify
4. Not timely

TOOLS:

1. Web browsers on multiple devices
2. Google Analytics
3. User testing services

Conclusion

In this article, I introduced four different ways to measure the performance of a webpage. Each of them is necessary to get a full understanding of the performance of a page. I also introduced some of my favorite tools that can be used for each approach.

The four approaches are...

1. One time assessments
2. Live monitoring and alerts
3. Full stack observability
4. Subjective user tests

I hope that this prepares you to start wading into the complex world of measuring website performance.