/ Technology

Robot vacuum cleaners – our new overlords?

Our Which expert holds up a robot vacuum cleaner.

Is it time to hand over our cleaning to robots, or are the old ways still best?


For the first time ever we tested robot vacuum cleaners looking at how well they clean compared to a conventional plug-in model…

Testing robot vacuum cleaners

The best robot vacuum cleaners in our test can suck up 50% of the fine dust that we embed into our carpets, but that’s not much compared to the best corded vacuum (86%), or the best cordless (83%).

But you can schedule a robot vacuum cleaner to come on every day if you want to, and clean much more often than you might do with a normal vacuum cleaner.

Not only did we test how well robot vacuum cleaners suck up debris and pet hair from carpets and hard floors. We also built a specially designed test room to see how well they navigate.

We filled the room with tables and chairs, floor standing lamps, rugs, fold out chairs and low lying curtains. This showed up the stark difference between one model of robot cleaner and another. Some navigated around obstacles effortlessly and cleaned nearly all of the available floor space. Others got stuck and left large uncleansed patches.

We thought we had considered every type of obstacle in our testing – until we read about the poopocalypse. Perhaps it’s something to consider for our next batch of robot vacuum testing.

Would you but a robot vacuum?

Head straight to our robot vacuum cleaner reviews to get the lowdown on the best and worst, and see them in action in our testing room.

So what do you think? Is it time to welcome our mechanical overlords and get yourself a robot vacuum cleaner? Or is the high price – as much as £800 – too much for a product that might not quite meet your high expectations?


They don’t clean as well as manual vacuum cleaning – getting into awkward places for example. As two thirds of us need more exercise (see Convo on obesity currently in progress) perhaps we should think a bit wider than a device that lets us sit on the sofa while it does (some of) the work?

Adding to what Malcolm has said, I don’t see how any of these machines can get into the corners, especially where there is furniture close to the corner. Using a vacuum cleaner with a hosepipe and tubes one can go along the tops of skirting boards, picture rails, and mouldings on furniture, between the bannister rails, behind radiators, into the spaces and crevices around media units, and in all the other gaps and cavities that a robot cleaner cannot get to. This should be the test for comparing these products with the usual alternatives, not what percentage of dust they lift from a floor [where they seem to be unimpressive anyway].

Washing machines can do a better job of cleaning than we can using our hands, dishwashers are genuinely labour-saving, vacuums also work better than humans with brushes and beaters. But, it makes me sad to see so much money and effort wasted in making a robot vacuum which neither covers the ground or picks up dirt efficiently. There seems to be little chance of inventing anything that picks up all the dust and dirt without a human directing it. There may well be useful outlets for the camera technology that directs machines round obstacles, to help a blind person or aid someone with a handicap but who decided that a robot cleaner could use this technology efficiently? What a waste of brain power!

The Which? review system for the robots is flawed. As a simple example they compare the robots in the Conversation with the percentage of dirt picked up from carpets by normal vacuum cleaners – that is 50% compared to the 8O%+ on carpets.

You might be interested, and I am interested in the efficiency on tile or wooden floors. Do we have a comparison figure for that. No. I am willing to bet that the dust extraction is substantially higher but why should I bet surely Which? have that detail. And it is relevant.

Secondly we do not have a maximum swept area before returning to base for a recharge which surely is of interest. Is it 20 square metres, 50 or 100 ? How soon will the dust container need emptying, every two days, every three hours? Can the machine do multiple rooms in a morning for light dust removal?

If this seems picky it is been known for several years that hard surfaces like timber laminates and tile have been taking market share from the carpet sector. I know of a house tiled completely downstairs with a total floor area in excess of 100 square metres.

“Hard floor cleaning
Hard floor
How effective the robot is when cleaning large debris from hard floors. This should be a robot vacuum cleaners bread and butter, but some vastly outperform others.”

Whats with the large debris test? How large is large? How does this compare with normal vacuums? What is the dust uptake like?

What we do know is the Dyson is better than the other robots but apart from one statistic we have no hard figures comparing other aspects of the robots to the standard vacuum cleaner.

Regarding VH’s point. My mother-in-law in her prime vacuumed daily but I suspect that not many people do that now. Now I am curious whether sending your robot cleaning daily results in an overall cleanliness level higher than a human hoovering weekly of bi-weekly. If I limit my activity to light dusting of skirting-boards and higher once a week plus using a robot for daily cleans, will I have a cleaner room than provided by a human?

And what is the time saving over one month …..

The testing does not really seem to answer the quite reasonable expectation that robots are competing with cordless and corded vacuum cleaners and this testing by restricting itself primarily to robot vs robot ignores the bigger picture.

Incidentally I saw nothing about air filtration which in an era of allergies would be something of interest you might think.

I think it all proves that the robot vacuum cleaner is not worth Which? spending much more time and resources on as it’s a solution looking for a problem. Now that Which? have built this test room I hope they have plenty of other uses for it. It could be used to test air conditioners, humidifiers, room deodorisers, useful vacuum cleaners, and so on. In fact, why don’t they just buy a typical house on a new estate and use it as a test lab for all sorts of home products under realistic conditions?

I actually don’t think that the testing of robot cleaners is wasted as I am sure the poor reviews help to prevent consumers buying them.

I can also see that for some people there might be a valid case where genuine infirmities these would be a boon.

My main concern is that Which?’s blind devotion to reducing everything to five little blobs – because it fits into the magazine, [our readers are superficial] – seems to under estimate how much room there is on the Net and that it should be used.

I continue to believe that this charity is doomed if it does not get its act together soon on its reason for being. My belief is that the serious millions spent on commercial ventures and focus that way has meant the research side has been ham-strung.

The idea of a normal house as a test-bed you would think remarkably valid given that they could have been doing serious long term testing on long-term products like doubleglazing, air heat pumps, etc etc. Or perhaps a pair of houses so the before and after effects can be checked.

A supplementary question that entered my head while thinking about the test room is – has anybody assessed how much influence Which? has actually had on product design and performance, and more to the point home design, layout and convenience, through all the decades of testing? Product testing for consumers is essentially a reactive process and no doubt contributes, in a sub-Darwinian way, to a process of natural selection and fitness for purpose through the elimination of weak performers from the market place, but could it go further than this and become a creative force for improvement? In the natural world there is an instinctive determinism that drives evolution; in the commercial world it is the profit motive; is that good enough?

Profit of course means least cost design and parts and maximum price. Which?’s role in design might be that once it stopped testing for durability to advise to subscribers it left a yawning hole for cheap goods to achieve Best Buy status.

Demonstrably true with the Logik’s steamer plus no doubt other products.

I truly believe that the Stiftung Warentest in Germany, despite having a much smaller income, has by its rigorous testing improved the quality of many German products.

So, you’ll need 2 robots for a normal house, one downstairs and one up stairs, plus a best buy £200 Miele proper cleaner for the stairs, skirting, carpet edges, behind the tv, and other nooks and crannies. £1800.

A robot takes 32 minutes to clean one room (incompletely). Which say with a normal vacuum it takes 38 mins to clean a whole house (properly).

A best robot picks up half the dust in a carpet, doesn’t deal with hair well. Even a best cordless picks up 83% of dust. So how can you possibly give 5 stars to a robot?

How much more electrical energy does the Dyson use when it takes maybe 6 times as long to do its partial job?

I think the concept of “Best Buy” needs a rethink. Products doing a particular job should be compared with all other products doing that job – in this case robots vs. corded and cordless. I also think price should be taken into account, not on its own but in relation to the likely life. For example if two dishwashers – one at £1200 and one at £600 – perform equally well, have similar build quality and are likely to last just as long then I would say the £600 one is a better buy. Perhaps another criteria should be “value for money”; an assessment that would be useful would be cost per year of likely fault free life.

This would avoid seemingly silly comparisons like the Dyson hair drier at £300, given a best buy when it only scored 1 star for low speed drying, whilst a Remington best buy at £35 scored 5 stars. Which is better?

What I am looking for is one robot that would use the existing vacuum cleaners properly throughout our house – which has two staircases – so that I can put my feet up once in a while. I don’t care how long it takes and I would be prepared to share its services with the neighbours on either side [whose staircases have multiple winders] in order to spread the cost. Alternatively, over the next two years at least, we could employ one of the many young people from elsewhere in the EU who offer such services in our area and perhaps that is a better solution overall [overalls provided as well].

The trouble is with our dealing with incomplete data from Which?

Basic problem has outlined above is we have a problem – hoovering houses. As far as I am concerned all solutions are in the frame so that is three types of electrical should be discussed against each other. And possibly why not the cost of a weekly cleaner to provide a context for the costs. There must be some time-stricken yuppies reading this!!

Batteries have a finite life of say 1000 cycles whereas corded can go on for decades so this aspect should and could be fully costed, and convenience factored in also.

Efficiency in cleaning is unrevealed for dust on hard floors where I am sure they can better the carpet test. I was curious to note one reputable robot had a 4% efficiency rating on carpet and was very good on hard floors which raises two unanswered questions:
1] is the machine meant for carpet?
2] is it faulty
and finally is it operator error.

I am please to note that from the current magazine that Which? may be doing some testing itself, and we have some named scientists, I thing we need some interaction. Probably the Community Forum would be ideal..

dt, I might have missed what you mention in your last paragraph unless you mean staff taking products home 🙂 but I’m not sure that counts as scientific testing.

I believe Which? should start doing some of its own testing again. Not only would they get a better insight into how products perform, but if they took them apart they might help us with quality of build and components, and repairability.

Discussing testing on the members community forum, as well as Which?’s policy, would be good. Judging by the number of contributors to the forum it would be a very compact debate.

Gavin Robertson says:
21 August 2016

£800 for a vacuum cleaner – not for me. I bought a iRobot Roomba 620 in 2015 after research of the features of the older models and paid £249 online. This is a very diligent cleaner as it goes over areas several times more than I would with a manual cleaner. OK it does have limitations and no fancy features but does a very good job. The dust box is usually full of fluff from carpeted areas and pick-up is good. I would say that the filter may not be the best. I suspect the exhaust probably does emit some fine dust.

I have spent around two hours viewing videos on robot cleaners and found other sites. I feel infinitely more comfortable that I understand the Dyson Eye and the other brands – many of which all curiously seem to share a similar body.

The Eye has recently had a software update and has a quiet mode for a longer working time before returning to base for a recharge plus an improvement to the app so you can see battery charge status.

I am perplexed with all the space on the Internet that Which?’s reviews are boiled down to little dots and are so sparse. If, and this is serious, Which? has not the resources why not simply direct subscribers to the reviews you think reliable on YouTube. A passionate amateur reviewer can do a very good job but picking them out of the dross is tricky. You could be the director to the best reviews.!

PS the guy doing the Eye spent 92 minutes starting from opening the package and it was a very useful video. As was the 12 minute follow-up.

Chris Moss says:
21 August 2016

I bought an iRobot Roomba 651 recently for £347 as it was one of the cheaper fully automatic versions and am impressed. There are a lot of misapprehensions in the comments above. e.g. power consumption. These things use around 40 watts compared with 1,200-1,500 for the average vacuum cleaner. So even if they take longer they’ll use much less (even the Dyson).

And who cares if they take longer? It’s not using my time. As I get older the cleaning doesn’t get done so often, so if this is more regular the house will almost certainly be cleaner. For many people the cost has to be compared with the cost of paying someone to do it, and it’s much lighter and easier to carry around.

Mine isn’t annoyingly noisy either. Our cat shoots out the house when I start the normal vacuum cleaner but stares in fascination at this thing. Ok, she jumped when it hit the chair she was sitting on, but came back to have another look!

Nigel Brealey says:
25 August 2016

We own a IRobot Roomba 650 – and my wife loves it!

We don’t think it really replaces a regular vacuum cleaner but supplements it to give us permanently pristine floors since it is used frequently because it is trivial to do so. I think my wife does still use the normal vacuum cleaner for a more thorough clean, but not often.

I’d comment also that we’ve had little problem with it getting stuck, and its “magic wall” stops it going where we don’t want it to go.

I see Amazon sell this model for £379 which is less than the £800 models in the Which? review.

We recommended this to my brother and sister-in-law who have a dog. They also use it in a similar way – and they also love it.

It is far from essential – but certainly nice to have and considerably reduces the normal vacuuming chore.

I brought a Iroomba 530 in 2008 for £172 and it still works fine.

I brought a new battery 3 years ago because it was charging slower for £50 direct but other then that it’s all still original.

I have 2 dogs and 1 cat and use it everyday on carpets and hard floors and it works great.

Lynda Goldenberg says:
4 September 2016

I have had various roomba models over the last 7-8 years as we have a house rabbit so lots of hairs and bits of hay etc near his food dishes. (Replacements have just been to upgrade not because they’ve conked out). The Roombas do a fantastic job keeping the area looking great on a daily basis , and I do a ‘proper’ vacuum once a week to do bits it can’t get into etc although the side brushes do a pretty good job anyway. I had noticed that you have the Best Buy as a dyson, but the capacity is lousy and wouldn’t last me 5 mins never mind a full clean. I think this should be accounted for in your next scoring system. I appreciate that just for household dust this will be less of an issue but add in pet hairs and it will make a huge difference to user friendliness. And mine covers the area completely by using the control towers to restrict the area to just my (large) L-shaped hall. I guess if I extended to the whole downstairs given the size and layout this may be more of an issue, but I’d just do each end of my house on alternate days if I needed to. I have a roomba 780 which was designed for pet hair and picks up the rabbit hair from my carpet tiled floor brilliantly…. And other carpeted areas if it escapes into other rooms and tiled floors if it gets in the kitchen, which it does occasionally if the batteries in the control towers have run down and I’ve not realised. And rabbit hair is very soft and fine so a real challenge even for my Dyson v6 ( I’ve had to buy a special head for the Dyson which is very small so hell to do a large area clean). Roomba also is Time programmable so it sets itself sets off hoovering on its own every day. – are the others? It means if I’ve gone away leaving the rabbit with pet carers popping in, the hoovering has still been done. The other issue not covered in your review is ease of cleaning the brushes etc, and the cost and availability of replacements. I can clean the brushes and dust bin in the dishwasher if I remove the hairs first) and get replacement brushes and filters easily and have even replaced the battery once. No idea with your other models if this is possible. So I will be sticking with Roomba unless there is a really good reason to change, as I know it well so can maintain it easily myself, and to switch to a small Dyson is a very expensive experiment and given my Dyson v6 doesn’t like the rabbit fluff I doubt the robot will be better than my Roomba.

A good description, Lynda, of the trouble with having a rabbit running around the house, but it sounds as though you are on top of it.

Michael S says:
18 September 2016

I was sorry not to see the AirCraft Vacuums Pilot Max included in the comparative test. At under £250 online it is significantly cheaper than the other brands tested and I have been delighted with the performance.

C Wakefield says:
18 September 2016

I bought an iRobot Roomba 780 for my daughter in March after reading many reviews for weeks. We were so impressed with it that I bought one for my new home too. The 780 on line reviews were far better than the 980 (which you tested).
Yes, it was expensive but with childminding and a moulting dog, it has been invaluable. It does corners, is thorough, doesn’t knock things over, cleans for quite a time without the need for recharging and I’d be lost without it now as its made my life easier.

You may not wish to add this comment but 3 of us actually ready reviews individually then compared notes before purchasing the 780!