A perfect 5.0-star rating should be a good sign. Statistically, it’s often the opposite: tiny review counts make perfect scores easy to manufacture and meaningless to compare. This guide shows how a simple adjustment — the same idea used by movie rankings — makes Amazon ratings genuinely comparable, and how to apply it without doing any math.
What a star rating actually measures
An Amazon star rating is an average of the ratings submitted for that listing. That sentence contains both problems. Average: averages of small samples are noise. Submitted: who submits is not random — early reviews skew toward people with a relationship to the seller, and review-gathering campaigns skew positive by design.
Neither problem means ratings are useless. It means a rating’s reliability depends almost entirely on its review count — the number printed right next to the stars, which most of us barely register.
The small-sample trap: why 5.0 with 20 reviews loses
Compare two headphones: one rated 5.0 from 12 reviews, one rated 4.6 from 20,000. Face value says the first is better. Statistics says almost the opposite:
- Twelve reviews can be the seller’s friends, family, and a launch-week promotion. Twenty thousand cannot.
- A true 4.6-quality product can easily show 5.0 across its first dozen reviews by luck alone. A true 5.0 across twenty thousand strangers essentially doesn’t happen — at scale, someone always receives a dud, a late package, or the wrong color.
- Perfect low-volume scores are also the cheapest to manufacture, which is why they’re a favorite of review farms. A perfect score on a young listing is a flag to check, not a reason to buy.
Adjusted ratings, in plain English
Rating systems that take this seriously — movie rankings are the famous example — use a Bayesian adjusted rating. The idea in one sentence: start every product at the category’s average rating, and let its own reviews pull the score away from that average only as the review count grows.
A product with 12 reviews barely moves from the average, no matter how perfect those 12 are. A product with 20,000 reviews is trusted at close to its face value. In practice, that 5.0-from-12 might adjust to roughly the page average — say around 4.3 — while the 4.6-from-20,000 stays a 4.6, and the comparison flips to match what your instincts should have said all along.
The point isn’t the formula; it’s the principle: a rating has to earn its distance from the average with volume.
A 60-second manual method
You can approximate an adjusted rating in your head with three rules:
- Under ~100 reviews: treat the rating as unknown. Not bad — unknown. The product may be great; the number just can’t tell you yet.
- A few hundred reviews or more: the rating is meaningful; compare within this group normally.
- Perfect 5.0 with low volume: actively suspicious. Combine with the basics — reading 3-star reviews and checking review dates — before trusting it.
One caveat: review counts on Amazon can be split across variant listings of the same product, so a low count occasionally understates a product’s real review base. If a listing looks established but shows few reviews, check whether its siblings carry the rest.
Automating it
The manual method works, but applying it across forty results while also ignoring sponsored placements is exactly the kind of repetitive judgment a tool should carry. Our extension, Shortlist This, ranks the products visible on an Amazon search page using a review-count-adjusted rating rather than the raw stars — alongside review confidence, price position, and relevance — and shows the full score breakdown for every product, so you can see exactly how much the adjustment moved each rating.
There’s no AI guesswork in that ranking: it’s a deterministic calculation from the numbers on the page, the same page you’re looking at, computed locally in your browser. Same page, same settings, same result — and every score can be opened up and checked.
So — are Amazon ratings reliable?
As a raw number, no: a star average without its review count is close to meaningless, and sorting a results page by rating alone actively rewards thin, gameable scores. As a volume-weighted signal, yes: adjusted for review count and read alongside the actual reviews, ratings remain the most useful single piece of information on the page. The difference between those two answers is one habit — never read the stars without reading the number next to them.