How our Expert Rating scores are calculated

The Expert Rating score is an aggregated rating made up of a number of individual review scores. Unless it’s a very niche model, we usually require a minimum of 12 scored reviews before we include a car in our Expert Rating index. Some cars have more than 60 reviews used in our calculations.

Our system uses a “weighted average” of all the reviews of a particular car published by the 25 sites we include in our index.
We factor in the age of each review (newer reviews carry more weight than older reviews) and we normalise the scores from each publication based to account for variations in how they each score their reviews.

It’s taken us a couple of years to develop and test the calculations to make sure they work, which is why we’ve only recently shifted from a simple average to our new weighted average system.

Sorry, that information is classified Ultra Top Secret and is above your security clearance.

What we can tell you is that it’s a complex mathematical algorithm plied universally to all reviews. We don’t manually alter any individual review scores, but every review is put through our super-secret algorithm.

Some titles review a lot of cars but they don’t provide scores (like the Daily Mail and the Sun). Other titles won’t score reviews for a short test and will only score vehicles on longer reviews. These reviews are still added to the database so you can find them and read them, but they don’t count towards the Expert Rating score.

Unlike some aggregator sites (Metacritic is one example), we don’t manually alter scores up or down, or create scores for unscored reviews. That works for their system, but we don’t think it’s the right solution for us.

It’s perfectly normal for Expert Rating scores to shift up or down over time, and there are a few reasons for that hpening.

Firstly, we are constantly adding new reviews to our database. For a new model launch, the big sites (Auto Express, Autocar et al) tend to get first crack, then the secondary sites. Finally, niche sites and regional titles get to review the cars. This process can take months. Then there will be additional versions (new engines, new trim levels) and facelifts, so the whole process repeats itself again and again.

Secondly, some sources will adjust their reviews scores over time. This may be as cars are re-evaluated against newer rivals, or it may be for a specific reason. For example, the 2021 Dacia Sandero was originally given quite high scores from most sources – right up until Euro NC crash-tested the Sandero and gave it a very poor two-star score. Several sites immediately downgraded their scores accordingly (and, in one embarrassing case, had to revoke a ‘Car of the Year’ award…).

Thirdly, the reviews are weighted for age. Older reviews carry less weight than newer reviews in the overall score.

Finally, our advanced algorithm automatically accounts for any changes in scoring scales or methodology from any of our sources.

We started out tracking 12 sites, concentrating on the largest and best-known automotive publications that review the most cars. Over the last two years, we have broadened that out to a current total of 25 carefully selected sites, including national newsper sites that have substantial and regular motoring content. We have also added a few specialist titles that target specific areas of the new car market (green cars, fleet buyers, performance cars).

As of August 2021, the list is:
Auto Express, Auto Trader, Autocar, Business Car, Car, Car Keys, Carbuyer, Carwow, Company Car Today, Daily Mail, Daily Mirror, Eurekar, Evo, Fleetworld, Green Car Guide, Honest John, Motors.co.uk, Parkers, The Car Expert, The Sun, The Sunday Times, The Telegrh, Top Gear, What Car? and Which?

We have a range of criteria for inclusion, but mainly we’re looking at the quality and quantity of reviews. We also consider the variety of cars reviewed by each publication and how the reviews are scored. We only take websites into account, rather than printed titles, so we can always link back to the original review source.

We also review our existing sites periodically and could remove some sites from the index if we don’t think they are contributing sufficiently to warrant their ongoing inclusion. By Christmas 2021, we hope to have added a few more sites to bring us to a total of 30. But any future expansion is dependent on review quality rather than simply adding more sources for the sake of it.

We only use UK sites in our Expert Rating data. A car’s pricing and specification can vary dramatically around the world, so a review from a US or Australian website won’t necessarily reflect that car’s position in the UK market.

As of August 2021, our aggregated ratings are available to licence. The data is already being used by Leasing.com, one of the UK’s leading leasing comparison sites. Leasing.com’s ratings are powered by the same review data as our own Expert Ratings, but we run them through a bespoke algorithm to provide a unique set of scores.

If you would like to enquire about licensing our ratings, please email editor@thecarexpert.co.uk.

We’re updating information in the Expert Rating section of the site daily, although that doesn’t mean we update every car every day. On average, each car will be updated about once a month so the information is about as current as you’ll see anywhere.

Ultimately, we want to include every new car on sale and we’re working as fast as we can to get there.

Broadly speaking, we’ve tried to get all of the most popular cars included as a priority. So that means there are more Fords and Fiats than Ferraris, although we’ve made sure that we have at least one or two cars from every major car manufacturer.

It can get tricky deciding whether to lump different versions of a model together or rate them separately. We look at this on a case-by-case basis.

For example, the Audi A4 has a multitude of different variants. We include the saloon, Avant (estate), A4 allroad and S4 versions all under the one general umbrella of “Audi A4”. But the Audi RS 4 is considered separately because it’s a very different vehicle.

Similarly, we consider electric models separately from petrol/diesel/hybrid models. A good example is the Vauxhall Corsa (petrol/diesel) and the Vauxhall Corsa-e (electric).

No, they are separate. We only use the overall review scores from each title. In many cases, these scores specifically take Euro NC safety ratings into account, so the Euro NC rating is indirectly included anyway. For example, Which? will heavily mark down any car with a Euro NC score of three stars or worse.

No, we don’t count user reviews even though some of the sites we track display owners’ ratings or carry out owner surveys (like the annual Auto Express Driver Power survey).

Owner reviews have some good points, but they can also be problematic. Most of the time, owner surveys are completed by people who’ve just bought a new car and are still in love with it, or by those who have had a bit of a nightmare and want to vent about it.
This tends to create a very polarised data set of very good and very bad reviews, rather than balanced and objective opinions that encompass a range of scores. You see the same thing across the internet in almost all situations – just look at Amazon for a good example.

Owner reviews are also much harder to verify and easier for PR departments (or salespeople, Russian bots, Extinction Rebellion or MI6) to influence, whereas reviews from recognised media sources and journalists are more reliable and trustworthy.

We are investigating ways of aggregating different user reviews to create an overall Owner Rating score to sit alongside the Expert Rating score, but so far we have found the data to be too sporadic and unreliable to provide a valuable score that readers can trust.

We are currently evaluating additional websites for inclusion in our index. We have a range of criteria for inclusion, but mainly we’re looking at the quality and quantity of reviews, the variety of cars reviewed and how the reviews are scored. We only take websites into account, rather than printed titles, so we can always link back to the original review source.

We also review our existing sites periodically and could remove some sites from the index if we don’t think they are contributing sufficiently to warrant their ongoing inclusion.

If you would like your site to be considered for inclusion in our index, please email editor@thecarexpert.co.uk.