/ Technology

The CMA: can you trust what you read in online reviews?

magnifying glass on laptop

The Competition and Markets Authority today published a report on online reviews and endorsements. Here’s the CMA’s Senior Director, Nisha Arora, reflecting on the debate their report has already started to generate.

The most important message from our report is that reviews and endorsements are really valuable to consumers.

That said, there’s work to be done if we want to preserve consumers’ trust. We’ve received information about fake reviews, genuine negative reviews not being published and a lack of clear disclosure of paid endorsements in blogs, vlogs and other online publications.

That’s why we’ve opened an investigation into a number of companies in connection with potential non-disclosure of paid endorsements. And we’re looking into other concerns that have been raised to determine whether we should be taking further action.

Reaction to our report

When I did the rounds on TV and radio this morning the same important questions came up repeatedly:

  • Can we believe what we’re reading in reviews and endorsements? Given the concerns we’ve identified, I can’t vouch for every review, blog and tweet. Nevertheless, the outcomes for users speak for themselves. Eight out of 10 people who use online reviews and blogs find that the thing they buy matches their expectations.
  • What can be done to address your concerns? While the CMA will play an active role, the main responsibility lies with businesses in the sector. It’s critical for them to maintain your trust and ensure that you continue to get good outcomes when they use these tools. We’ve tried to help by producing information for businesses explaining what they need to do to comply with the law.
  • What do you suggest people should do? There’s no sure fire way for you to tell if a review is real or fake – or indeed to be certain that the site that they are looking at is publishing all reviews. However, our research on the way that people use reviews and endorsements suggests that some of you are already pretty savvy about using online reviews and endorsements. For instance, testing what they read with other information sources including reviews, blogs and friends and family.

Online reviews can provide valuable information and it’s important that your trust in them is maintained. ​That’s why we’re continuing our work in these sectors.

What’s your experience of online reviews? Have you been suspicious of a review that’s been published? Have you ever had problems posting your own user reviews?

Which? Conversation provides guest spots to external contributors. This is from Nisha Arora, Senior Director Consumer at the Competition and Markets Authority. All opinions expressed here are her own, not necessarily those of Which?.

Comments
Profile photo of malcolm r
Member

I listened to Nisha on Radio 4 this morning and was distinctly unimpressed with the CMA. When asked what hard evidence the CMA had of fake reviews, she had no information to give. I am not suggesting there are no fake reviews, and I am strongly against faking “independent” endorsements, but I would have liked a more prepared response to support the CMA’s assertions.

Looking quickly through the report there seems to be “reports of” and maybes, but no hard evidence. Just the suspicions of some (unquantified – 1-2%, maybe higher) malpractice that perpetrators should address while CMA continues to investigate.

Perhaps it was all a bit early to go for publicity? Perhaps I am ill-informed and being unfair.

Oh, and please, can we try to ban people on the airwaves from beginning sentences in reply to questions with “So….”. Perhaps that put me in an unsympathetic frame of mind..

Profile photo of banjo
Member

I always ignore reviews if there only a few (say less than 10) reviewing a product I am interested in. If there are hundreds then I am more likely to believe them. I’m sure fake reviews are everywhere.

I’ve posted negative reviews on Argos only to have them removed.

Profile photo of John Ward
Member

I was wondering where this issue figured in the CMA’s work schedule of the top ten important consumer concerns.

My opinion on reviews has always been that they are worth what you pay for them.

Profile photo of wavechange
Member

I look for reviews that contain detailed information and explain some of the disadvantages of a product as well as making positive comments.

Some people take a great deal of effort to provide us with useful information that retailers omit.

Profile photo of John Ward
Member

Yes, I agree with that but it has been widely alleged that in the hotel sector some of the adverse reviews have been posted by competitors.

Profile photo of wavechange
Member

I was thinking mainly about household goods, John. You are right about hotels.

Profile photo of rarrar
Member

Just looking at the restaurant and cafe reviews on TripAdviser for my local touristy town, they all get 4.5-5 stars with the most recently opened or revamped ones getting the highest scores due to a burst of recent reviews from locals.
Considering the wide variety of establishments covered absolutely of no use at all for making a decision,

Profile photo of Patrick Taylor
Member

Unfortunately Which? is not guiltless in the reviews stakes.

Before I explain that I should mention that I have been following this area for several years collecting examples of these deceptions from computer and consumer sites. I have highlighted a potential case on Which? reader reviews some months back. I have also suggested to Which? how to make their reader review system more robust and reliable.

Which? carries out product testing and subscribers can view the results and if subscribers are owners they can add their own review. This is very important as Which? does not employ test labs for longevity/durability trials on appliances.

Anyone browsing the Web can read these Readers Reviews. Unfortunately , and particularly where it concerns Best Buys, Which? is not responding to Readers Reviews. In what might be described as the star exhibit we have a product where over 20 people out of 22 with valid scoring give it the lowest rating of one star. The comment now is this should be a Do Not Buy not a Best Buy.

Given the offending item is around £20 you would think an investigation would not be too difficult to arrange and given the two common faults expessed seem design faults and the average time to fault around 6-12 months in normal usage not too difficult to arrange.

Which?’s unique selling point has been independent testing but it must back this up with taking the input from readers seriously so its reputation is enhanced not weakened.

Profile photo of wavechange
Member

I would expect that the reviews of products written by Which? members are more trustworthy and useful than those published on retailers’ websites but in many cases there are few comments on products, whereas Amazon may have numerous reviews of a product.

I agree that Which? should look at information posted on its own websites. I was very disappointed that the recent article about LED bulbs in Which? magazine made no mention of the problems of radio interference and premature failure mentioned by so many contributors to Which? Conversation.

Profile photo of Patrick Taylor
Member

I think there is a dysfunction with Which? taking on hundreds of thousands of subscribers yet making very poor use of them and the money.

I see now that Which? are using Revoo system for the Wshing machine reviews presumably at a cost. I think there are drawbacks with ths commerical company which as with Which? currently has absolutely no follow-up system.

Revoo claims
Independent
We collect reviews, recommendations and answers to consumers’ questions for 100+ companies – but we’re totally independent. We can’t be swayed by these companies.
Impartial
We never, ever remove bad reviews. We never edit reviews. We don’t publish reviews with swearing or spam, but we do let these reviewers amend and re-submit.
Trustworthy
We only collect reviews from verified purchasers – people who have real experience of what they’re reviewing. And we never pay for reviews.

Profile photo of Patrick Taylor
Member
Profile photo of malcolm r
Member

“Chris Gardner, MD, Which? Publishing comments: “We are delighted to be working with Reevoo. We know that our members want to see the perspectives of other consumers alongside our expert reviews and advice. Reevoo’s large database of consumer reviews and the way they are collected made Reevoo an obvious choice.”

I had never heard of Reevoo until you posted here dieseltaylor. As a cynic, I worry that a large commercial organisation, seemingly devoted to helping its clients (including manufacturers), is being used by an independent consumer organisation. Are there clients (who presumably pay them) happy for Reevoo to pass negative reports on their products to Which? Is that not a bit of a conflict of interest? Hope I’ve got it all wrong and its all squeaky clean.

I would have though Which? together with its companion organisations throughout Europe would have been able to gather sufficient of its own customer reviews to make product assessment that were known to be independent. Why, as dt says, do they need to pay for this through a commercial organisation over which presumably they have no control?

Has anyone submitted a review to Reevoo? Where do they get there information from and how?

Profile photo of wavechange
Member

I’m familiar with Reevoo though have never contributed a review.

In another Conversation we learned about AMDEA coordinating recalls on kitchen appliances, but that there was the danger of our contact details being used for marketing and market research. That’s totally unacceptable in my view.

I have no idea whether or not Reevoo is impartial but because of the commercial involvement there is every reason to be concerned.

The problem is that it costs money to provide a properly run review service and the easy way out is to involve the companies that can benefit most from the information.

Profile photo of wavechange
Member

Dieseltaylor – the Reevoo review you provide a link to has 30 pages of comments. Most of them are brief but there are some useful points that could help prospective purchasers.

Some of us would like very detailed technical information but we might be the exception. Inviting comments on positive and negative features is, I believe, a useful approach to help reviewers focus on key information.

Profile photo of Patrick Taylor
Member

I have used several Review sites and added feedback on products. However I am very conscious that several of them seemed very open to abuse.

I have downloaded all that Revoo offers and its claims for verified second generation reviews. I have read all four pdf’s which confusingly they suggest are e-pubs and they download with no file extension.

My original suggestions – getting on from two years now combined review and survey AND given only open subscribers would provide better security and significantly better data for Which?. Conversely it would reduce the need for Verve and Revoo. I know Verve when last reported was costing over £0.5m per year.

Out of 800,000 subscribers it would not seem difficult to launch a review system that actually followed purchasers through the intial satisafaction or otherwise with a product, but then followed up with a continuing survey of usage and repair costs and when and why it was chucked. Or even passed on.

A discount on subscriptions, and an on-line detailed record of devices/appliances could be the carrot. Not to be ignored is the desire of many subscribers to be helpful to fellow consumers which I think could be quirte strong.

Not to forget that in the event of a Best Buy like the Logiks steamer ending up with the worst possible score Which? would easily be able to see what percentage of overall ownership that it respresents.

All in all a win for the charity, a chance for subscribers to get involved in a practical way, and hugely more accurate results on equipment owned.

Profile photo of Patrick Taylor
Member

Wavechange – your last comment I find surprising in that page after page is simply a score – from someone who has just bought the washing machine. They generally chase you up a week or two after the purchase.

One of the scores was a four – again without any comment. I just do not see how this advances the process. Roughly 12 of 30 pages there were no comments.

An interesting one would be 24 months later from this lady. A maximium user possibly.

Margaret Village in North Wales GB Frequent user
Good Points +
Ease of removal to clean the powder container.
The 15 minute cycle – brilliant for our B&B! We do use other cycles but that one we use the most. It saves time and money – and the washing is still clean!

Bad Points −
Would have liked a ‘spin only’ cycle as well as the ‘rinse and spin’.

Profile photo of wavechange
Member

I did not look through all the pages but did see some useful comments including the one you have posted. What is significant is that I did not see multiple comments about the same problem.

In contrast, if you have a look at the reviews of Apple laptop chargers you will see repeated comments about how the cable has fractured near the MagSafe connector. Mine has broken today. 🙁

It might be useful to prompt users of review sites about what to consider when writing a review but as I said, asking for good and bad points is useful.

Profile photo of malcolm r
Member

What I want from a review are facts. So initially, how well does a product do its job? This needs to be provided by someone experienced in the type of product and its likely plus and minus points. So I would hope an independent testing organisation like Which? should professionally assess the product.

Then, just as important, I want to know information about reliability and durability – will it continue to do its job in regular use and will it last a reasonable length of time without a breakdown – will it be durable?

To my mind these are issues that a knowledgable organisation needs to deal with (that is knowledgeable about testing and assessment). So just how do Which? find Reevoo fills this role?

I often read reviews before buying something I know nothing about (having looked at Which? if they cover the product) but start with the low scorers. These tend to point to any product weaknesses – more important than the strengths in my view)

Profile photo of wavechange
Member

Here are helpful reviews about a power adaptor on the Apple website. Many refer to the problem I have had with the cable: http://store.apple.com/uk/product/MC556B/C/apple-85w-magsafe-power-adapter-for-15-and-17-inch-macbook-pro?fnode=5a

I knew these things were fragile before I bought my laptop in 2011 and handled the cable very carefully. I have carried out a repair when I saw the plastic was starting to break and hopefully it will last a bit longer.

I bought a new Apple laptop in February. It uses a different power adaptor with the same problem: http://store.apple.com/uk/product/MD506B/B/apple-85w-magsafe-2-power-adapter-for-macbook-pro-with-retina-display?fnode=5a

Yes reviews can be very useful. I cannot understand why Apple, which pays a huge amount of attention to detail in their products can carry on selling substandard power adaptors. Their older adaptors were much better.

Profile photo of Patrick Steen
Member

Hi Diesel, just on your original comment. As you know, we carry out the most thorough and independent product tests in the UK but, since we want to get results up as quickly as possible, even our tests can’t uncover long term faults. That’s why we carry out annual product reliability surveys – and we won’t award a best buy if the reliability figures are too low.

We also, as you say, invite member reviews. We love getting these and we do take them seriously. We also use these as a sense check on the best buy status of a product. Our researchers read the reviews and will add comments when they think it will be necessary or helpful. I’ll share your feedback that you’d like them to respond more often. One thing to watch out for is that someone who has a bad experience with a product is more likely to post a review than someone who has had no problems at all, so researchers have to consider the whole picture.

To everyone else who’s been talking about Revoo – your comments make really interesting reading. We’re actually just piloting including reviews from Revoo (initially in washing machines and TVs) so you can see more views from other customers. We carefully selected Revoo due to their independent and transparent approach, as well the fact that they have appropriate checks in place (only verified owners can leave reviews). Still, as I say it’s a new thing, so it’s good to hear what you think. Oh and we’d love to have even more Which? members making reviews, so to that end we’re promoting product reviews in two areas (vacuums and kettles) to get more of you leaving them.

Profile photo of malcolm r
Member

Patrick, not a new suggestion, but perhaps worth repeating. Can you not make more use of Which Connect to elicit reviews – both good as well as the bad? And could you not campaign to enroll more Which? subscribers onto Connect? Which? has a special community that should be prepared and willing to help the consumer cause. Perhaps it needs a bit of a nudge.

Profile photo of Patrick Taylor
Member

patrick – As I have notched up over 100 surveys for WHich? over the last few decades I am very familiar with the quality of the surveys that I fill in.

AND until such time as the surveys ask about usage they are irredeemably flawed. A relative hoovers daily and my carpets are lucky to see the vaccum fortnightly. Her machine by rights will dies ten times earlier than mine all other things being equal. Is this a fair comparison on reliability?

My shredder forms part of a survey this year and I can truthfully say I have not had any problems with it – but then it has not been used for four years and you do not ask me that question. Ditto my hadge trimmer.

I have been writing this on surveys since it was mail based. To be fair detailed questions were asked in the sewing-machine survey in November last year. Which rather begs the question why were those questions on usage not asked about garden appliances. For the life of me I cannot think of a good answer.

Profile photo of Patrick Taylor
Member

As to Revoo. I have given it some thought.

Firstly is Which? edging towards abdicating from running readers reviews?

Scondly, looking at how Revoo runs I think I can see two ways it can be spoofed. For say a caravan site it simply generates a sale to a friendly person which generates a Revoo response. Job done. Particualrly easy for on-line bookings. For manufacturers a little more devious and dangerous but doable.

However in the final analysis if it is not doing a follow up in a years time we are really aswell off as reading a Which? test. Afterall its not as though you can post direct to Revoo a year later saying the Logiks steamer only lasted 11 months and you want to revise your rating..

And of course Revoo has set up its own on-line selling shop off the back of the reviews. Given WHich? derives money from pricerunner.com for purchases carried out from the Which site is there not a major danger that Which? may reduce its income? Though the amount earned has never been acknowledged as far as I am aware.

Profile photo of John Ward
Member

My highest review on a product is to describe it as adequate and satisfactory [i.e. it does what I bought it for]. This has occasionally led to some sarcastic comments from companies who feel that all reviews should be composed in superlative terms – which, of course, many are because the writers want to win a prize.

I judge products solely on their own merits, not by comparison or contrast with alternative products, a sort of empirical indifference. The average review format does not usually allow for that degree of absolutism.

I always abstain [if possible] from the question “would you recommend this product to a friend” since I think it is pointless.

Profile photo of malcolm r
Member

If I look at Which?’s mobile phones reviews both the Sony Z2 and Z3 appear as best buys, with no hint of problems people report of screens cracking and retailers / Sony’s apparent unwillingness to help. There are 5 customer reviews for the Z3; only one mentions a cracked screen.
It seems odd that after 571 comments on the parallel conversation nothing has been done to flag up a warning at Which?

So what value reviews (isn’t it another name customer feedback?). And given these Conversation’s rich veins of information why does Which? not make use of it?

Am I alone in thinking I’m grumpy 🙁

Profile photo of Patrick Steen
Member

Hi Malcolm – thanks for the comment. We acted on those comments by testing the phones in our labs and have been in touch with other consumer orgs across Europe on the issue. We haven’t found anything in these specialist lab tests that suggests an inherent problem yet, which is frustrating. If we had found something we certainly would cranked things up a gear. I’ll ask the Tech team about whether the comments should be mentioned in the reviews.

Profile photo of malcolm r
Member

Thanks for the response Patrick. 🙂

Profile photo of Davy Nook
Member

I often find that using a search engine to discover a user forum, preferably independent from the manufacturer/service provider, is a useful place to find what sorts of problems are being discussed. Most forums have their share of trolls but they are usually evident from their poor grammar and style.

Taking those caveats into consideration one can form an opinion after looking through reviews on trusted comparison sites such as CA which is better at giving a short-list before looking elsewhere.

As a long-term member of CA my experience is that their reviews do tend heavily on the set-up of items (done only once) and omit to cover information for the more technically literate members. Other sites have their own strengths and weaknesses which comparison between them illustrates.

Caveat emptor.

Member
Jane Frost says:
26 June 2015

I bought a recommended ‘whisper quiet’ vacuum cleaner on the say so of Which.

Sounded like a low flying aircraft.

I e never trusted a review since …but if I read them I read the average scores only.

Profile photo of Patrick Taylor
Member

wavechange –
” Margaret Village in North Wales GB Frequent user
Good Points +
Ease of removal to clean the powder container.
The 15 minute cycle – brilliant for our B&B! We do use other cycles but that one we use the most. It saves time and money – and the washing is still clean!

Bad Points −
Would have liked a ‘spin only’ cycle as well as the ‘rinse and spin’. ”

I must admit I was thinking of this from the point of view of the durability factor given the machine would be in a semi-commercial environment. I have only now belatdly realised the implications of the 15 minute cycle which I assume is for sheets etc.

There cannot be any possibility of a hygenic clean in such a short cycle. Rather a worrying idea.

Profile photo of John Ward
Member

I agree with you Diesel. B&B’s are under intense competitive pressure so they are drawn to skimping on essentials. I don’t think a standard 1-1½ hr wash programme is excessive; if you’re in the hospitality trade clean and hygienic bed-linen is a fundamental and keeping up with the changeovers goes with the territory. It makes you wonder how widespread this approach is. Unfortunately the average guest will be unaware of this practice and therefore unable to refer to it in a review.

Profile photo of wavechange
Member

Dieseltaylor – I had not thought about this either. I had just registered that there were both positive and negative comments.

Whenever I am going to be staying away from home I take my own pillowcases. If travelling by car I take a pillow too, irrespective of whether I’m staying in a B&B or expensive hotel. I started doing this as a teenager because feather pillows could trigger my asthma but now it’s more of a hygiene consideration.

At least user reviews help provide us with all sorts of interesting information that we might not otherwise think about.

Profile photo of wavechange
Member

There is a recent Conversation about expandable hoses: https://conversation.which.co.uk/energy-home/xhose-expandable-garden-hose-problems/#more-89175

Whether or not the online reviews are good quality, it is fairly obvious that many are not happy with their purchases.