Tell me if you’ve experienced this before. You run a speed test on a website, and the results don’t match what you personally experience. Perhaps you feel like your site loads fairly quickly, but the speed test results are sub-optimal. You may even have a client who’s wondering why they’re getting inconsistent results on tests they’ve run on a site you built for them.
It’s one of the most common (and frustrating) scenarios in web performance testing. Understanding why you’re seeing inconsistent results quickly becomes both confusing and overwhelming. In fact, it was the driving force in the creation of MachMetrics. We wanted to have control of the hardware that runs our tests and deliver consistent results. That’s why when you run a test with our service, your test is run on a real computer using a real browser. All of your images, javascript, and styling are being physically loaded on a real machine. Of course, you still won’t see the same loading time every test, so let’s look at some of the reasons why.
Outside factors may be affecting your speed test results
Unfortunately, there is no industry-accepted test that will produce the same results every time. At MachMetrics, our testing procedure is focused on showing you what the average user will experience. This is an important distinction because no two people will access your site within the same environment. Some visitors will be coming to your site via a blazing fast internet on an overpowered, flagship mobile device. Others will be coming to you from a slow, rural internet connection on a budget laptop. Every single visitor will be a different distance from the server.
Page speed test results will even vary slightly when the factors are unchanged. Below you’ll see an image showing a graph of a website’s test results over the course of a month. The website hasn’t had any changes made to it, (it’s a personal site I haven’t updated in some time), and you can see how each individual metric varies.
This variability could also come from loading third-party resources, such as ads or tracking snippets. Sometimes the ad loaded is a quick jpg, while other times it’s an animated gif. Sometimes the tracking server responds quickly, other times it’s under high load and you’re at its mercy.
There are also many different types of measurements, from server response to completely rendered. We go into more depth on each of them here.
While we report on all of them, the default one we display is the ‘page complete’, which is going to be the highest of most of these.
What’s important to remember is that we need to be focused on the average load time, not your load time.
The trend is your friend
One of the most important things you can take note of is the changes in different metrics and the overall trend. Is your website getting slower? Larger? Did the update to the server you just made really make your site faster? What effect does your recent CDN transition have? Did that WordPress plugin that you just enabled change your site performance?
You also need to keep in mind that repeat visitors are likely to have a different experience than first-time visitors. For example, if you implement a caching plugin into your WordPress site, many of your resources will be cached, drastically improving the load time for someone coming back to your site. To that end, with our service you can also monitor how your changes affect the trend for both first-time visitors and repeat customers.
This is the reason we stress the importance of a continuous monitoring service like ours at MachMetrics. It’s difficult to assess which way your website is trending when you only using spot testing. The advantage of continuous monitoring is being able to see the results over time. Another example of this can be found below.
After implementing a few optimizations to a client’s site, we were able to see exactly how the performance improved. We also send out a weekly summary of your portfolio of sites, so you get trend results delivered directly to your inbox. All of these features are geared towards addressing the inefficiencies of one-off speed tests.
Don’t stress over a score
One other frequent area of frustration stems from some page speed audits using a score to represent your website’s performance. Google Pagespeed Insights is a notable example of this. PSI offers great suggestions on how to improve your site’s performance. However, we’ve written before how going after a perfect score is a waste of time at best, and dangerous at worst.
PSI uses a Moto G4 on a slow 4G mobile network for its testing. As we mentioned above in the section covering outside factors, that may or may not be an accurate representation of what your visitors are using. If you personally are using a better phone or a faster network, you may wonder why you’re getting a low score yet experiencing a quick load time.
Additionally, as Google’s own John Mueller wrote,
Generally speaking, these scores can change over time even without changes on your site, they’re not physical measurements. Use these tools to find ways to improve your site for users, don’t see them as the final goal.
https://twitter.com/JohnMu/status/1092774805991051265?s=20
In short, if you deliver a snappy load time yet don’t have a perfect score, you’ve done your job.
Summary
We completely understand how frustrating these situations can be. They’re the reason MachMetrics was created! We wanted to provide a service that would allow our users to identify the trends and the way the average visitor experiences your site. It’s also why we went above and beyond simply testing your site. Being able to view charts and graphs depicting your site’s long-term performance is invaluable.
Ultimately, our hope is you’ll pick a speed testing tool and consistently use it. Bouncing from audit service to audit service only adds to the confusion when you’re getting different results and suggestions from each one. If it’s automated so you don’t have to remember to check it, that’s even better! You can’t improve if you’re not measuring – that’s half the battle.
In short, don’t be offended if you receive a load time (or score) that isn’t what you experience. Instead, consider it just a baseline measurement that’s used to report and notify you of the changes and trends of your site’s performance.