/ Uncategorised

Help us shape the future of Which?

This is your space to discuss all things related to Which?, including our governance. Together with other Which? Members and our Ordinary Members you can discuss our past, present and help shape our future.

For those of you new to Which? Conversation, welcome! Our community website is your space to discover and debate the burning consumer issues of the day.

While our community enjoys getting down to the nitty-gritty details of big issues like stopping nuisance callers or exposing the wider risks of product safety, we also want our Ordinary Members* to be able to connect with one another and have their say on the governance of Which?. If you want to have a say in how we’re run and help to shape the future of Which?, you can become an ‘Ordinary’ (or voting) Member.

We’ve just celebrated the 60th anniversary of Which?. The views and support of our members have been key to helping us get this far, and so your views will also help shape our next 60 years.

Which? discussion

So we’ve created this space for you. Here you can connect with other members and discuss all things Which?, including governance, feedback about our organisation and issues you want our Council to consider.

To keep things running smoothly, we have a few simple rules specific to this area for you to follow:

  • Keep comments relevant to this area – sharing your thoughts about Which? and its governance
  • If you have something off-topic to share, please share it in ‘The Lobby’ or find a relevant conversation to join. If you’d like to talk about the closure of Which.net, we’ve created a dedicated discussion area for you here.
  • Before you share a resolution, please check to see if another member has shared something similar and add your support to it by replying
  • Please be polite and speak to others as you’d like to be spoken to, whether that’s with other members or Which? staff.
  • Keep personal contact details private (eg email, phone number or address) – whether they’re yours or someone else’s.
  • We’re here to listen to your feedback about Which?, and we’ll try to answer any questions as soon as we can, but please be patient with us and mindful that we don’t work weekends

These guidelines are here to keep things running smoothly, so if you spot a comment that breaks these rules, please bring them to our attention by using the ‘Report’ button.

It’s easy to register on Which? Conversation, just click this Sign in/Register link and click the ‘Register’ tab. Our ‘Help getting started’ guide should explain any questions you may have about getting involved in Which? Conversation. Now, it’s over to you.

*If you aren’t already an Ordinary Member and are interested in finding out more, you can read about how to become an Ordinary Member right here.

Becoming an Ordinary Member means you can:

  • Vote in the annual Council election – the Council is the ultimate governing body of our charity, the Consumers’ Association, and oversees our whole organisation. We’ll send you the ballot booklets every November.
  • Come along to our AGM – meet our chairman, chief executive and team. You can find out more about what we’re doing and why, hear from our teams and ask your questions, and see how Which? works from the inside.
  • Stand for election to the Council – you can nominate yourself and ask for support from other ordinary members.
  • Nominate other ordinary members who want to stand for Council – to help make sure we have the right people governing.
  • Get our annual and interim reports and accounts – we’ll keep you up to date with all that’s going on, so you’re always in the loop.

Read the 2017 AGM Q&A here.

Read the 2018 AGM Q&A here.

 

Comments

Chris Matthews says: Today 13:44
Lab testing does take time as you’d expect, and is very expensive. Secondly, (as) we don’t have budget to lab test everything…

I’m going to go out on a limb, here and propose that the volunteer idea I’ve been attempting to get Which? to explore – seriously – might be perfectly suited to this context.

Let me explain: among the Regulars, we have Engineers, Statisticians, Mathematicians and Scientists holding degrees up to and including PhD level and a colossal wealth of experience. Is it not beyond the realms of possibility that these people could be trained by W? to become testers for products? Now, before the staff at Which? fall off their chairs cackling at the apparently naive stupidity of such a suggestion, those of us with Higher degrees are already more than capable of carrying out and carefully documenting investigations to a very high standard, and it’s a pretty safe bet that those with associated training and experience could easily do the same.

Which? would need to specify the precise tests required and the formatting of subsequent reports and check the submissions for accuracy but would this not save the organisation a lot of money? If successful, would this not also extend the range and number of models able to be tested?

Which? would also have to hold training sessions to ensure parity of presentation between different reporting styles, but would this not still result in significant savings for the organisation?

This was the founder’s original vision: Young saw using the membership in a very practical way, ;and certainly not simply form filling, such as W? connect surveys.

I do appreciate that tests requiring oscilloscopes and other equipment wouldn’t be feasible at home, but I’m willing to bet that most user tests of products are essentially comparative in nature and could be done by qualified and well trained volunteers.

I’ve disposed of my Tektronix oscilloscope, so maybe you are right, Ian.

I see testing by volunteers as complementary to that done in labs. One of the most annoying purchases I ever made was a cordless phone with multiple handsets. The Which? review did not warn me that I would have to put each phone number into each handset. Although that could be useful in some households, it is a feature that should have been mentioned in the review.

In some cases there will be differences in opinion about what is good or bad, but analysis of feedback from volunteers could help identify tissues that are worth reporting.

I am available [together with some others in our neighbourhood to ensure a spread of reactions] to test fine wines, chauffeur services, leather shoes, boutique hotels, and other niche products and services that Which? struggles to cover satisfactorily.

Ever the opportunist. 🙂

Quite so, Wavechange. Well, I can’t do scientific testing so I would have to confine myself to things requiring a subjective opinion where prolonged testing would produce a more reliable verdict.

SWMBO made a similar observation about gold rings, diamond jewellery and a good white wine. But on a more serious note, it did start me thinking. Whilst rigour is a pre-requisite for any testing, technical equipment almost certainly isn’t. The evidence for that is the very existence of Which?’s own Car Guide. Each year I’m invited to complete a fairly lengthy questionnaire on our cars. The results of all the completed surveys appear to form the basis for the guide.

The car guide works to a simple formula: precisely worded questions with copious options requiring only a tick or a cross and space for extra details if felt to be needed. And I’d venture to suggest a car is possibly one of the most technical items W? could test.

Anyway, as the rain and wind seem set to perform a passable recreation of a biblical deluge today, I’ll dig out last month’s W? mag and analyse the testing done therein, just to see how much has to be done by means of added equipment. Could be interesting.

Quite naughtily, I enjoyed that typo “corss” – I like the idea of test questions requiring either a tick or a curse 😉

Using volunteers isn’t always fast, but can help to get things done.

Even if you just get a bit more done, that can still help.

Over time, I am getting to meet a few more of my fellow volunteer “computer buddies” at Gloucester Library. We seem to come from a variety of backgrounds, but we do all seem to be motivated to help where we can.

You were very quick with that ‘corss’ comment; I changed it as soon as I’d spotted it – or so I thought.

Memo to self: must proof read better

The Which? annual car survey and the Car Guide that follows seem to me to be rather better than Connect surveys and those designing the latter could usefully study how the car surveys are constructed.

I’ve started to look at the August tests and have found at least two products that probably couldn’t be tested easily by volunteers, if only on space grounds. Artificial grass and Mattresses are probably out, but the other items are most certainly capable of being tested by trained volunteers.

Phil says:
10 August 2019

Product testing has to be carried in a manner which is consistent, repeatable and under controlled conditions. I don’t think this would be achievable with amateurs working in their own homes even if they did have the right kit.

Of course once upon a time Which? had it’s own laboratory stuffed to the rafters with oscilloscopes, AVOs and every other device you could possibly need, all properly calibrated and traceable back to the NPL. That was before technophobe Mckechnie was allowed to stick her oar in and destroy something which had taken decades to create.

There is no doubt that comparative testing needs to be done under standard laboratory conditions, but is there not plenty of scope for doing additional testing in the homes of volunteers, Phil? As Ian says, some training would be required.

Some of us are very keen to make a worthwhile contribution to the future of Which? and surely engagement with members should go further than completing surveys.

Phil says: Today 12:21
Product testing has to be carried in a manner which is consistent, repeatable and under controlled conditions. I don’t think this would be achievable with amateurs working in their own homes even if they did have the right kit.

That’s only valid where such precise measurements are required. But in the majority of the product test examples in the W? Magazine, that’s simply not the case.

Let’s look at a single example at random from the August 2019 Magazine:

Inkjet vs Laser printers
Aspects tested and reported:

Text printing
Duplex printing or not
Speed of printing
WiFi / airplay compatible
Colour printing
Ink cost
There is nothing in each of those categories that could not be fulfilled by trained volunteers. Once the calculations related to the overall percentage score are revealed then that score, also, could be calculated by volunteers.

Volunteers would probably be less likely to pen conflicted summaries. For example, the snappily named Brother DCP-L253ODW was rated as a ‘Best Buy’ with 76% while also described as ‘Mediocre to poor at printing office graphics and pages of images with text’.

The language in the summaries is also lacking in precision: how, for instance, is the reader supposed to differentiate between ‘Fantastic black-text print quality’ and ’Superb at printing black text’? Descriptions of this type are as useful as a Wine Connoisseur’s report.

So yes; perhaps for some products such as paint, for example, lab testing is needed. But this is only one of two product test reports that could easily be done and done well by trained volunteers. Which? is letting a lot of staff go because they’re short of money and this thread started because a W? staff member reported product testing was very expensive. But as I believe I’ve shown it doesn’t have to be.

I agree Phil. Testing needs to be done under controlled conditions to ensure consistent and repeatable results that lead to sensible conclusions. Looking at 3rd party inks (August Which?) would, in my view, need to use a variety of printers, paper, standard procedures and print samples, means to assess objectively the quality of the results, the accurate representation of colour, and in my view the durability of the result; I don’t want my photos to fade or discolour.

Checking the life and capacity of rechargeable batteries requires standard drain sources, chargers, and long term cycling to determine retained capacity and likely life, and a lot of samples to ensure representative results.

These are investigations not within the capability of individuals and need to be done in properly-equipped laboratories. As. of course, do any safety tests.

Surveys are useful, as for cars, to elicit owner experiences but even these must be treated with caution as cars are driven with varying degrees of care and owners have differing perspectives on problems. Nevertheless I believe properly constructed surveys, used in conjunction with testing, provide very valuable information in assessing products.

Where I believe members could help product testing is in usability – handling a new product and reporting in a structured way on how it performs; hopefully able to do this over a long term to establish some assessment of durabilty. I believe many Which? staff do this. Members could also help by advising on the types of products that should be tested, and the range. The danger is Which? trying to do too many with their limited resources. They could also review proposed test reports before publication.

However, members who are interested might take an example and spell out here exactly how they would test a specific product so we can better discuss the pros and cons of what can be done at home.

Reviewing Which? material is something where I believe Which? members could make a real contribution, both in the formative stages and in draft form. I have been critical of a number of published articles where, in my view, they have been inaccurate, selective, biased and defective to different degrees. My principle would be that what Which? says in print and in public should be objective, accurate, balanced and fair. Maybe I am alone in that view, but I hope not.

malcolm r says: Today 13:54
I agree Phil. Testing needs to be done under controlled conditions to ensure consistent and repeatable results that lead to sensible conclusions.

As I’ve said that’s only in cases where precise measurements are required; printer inks is, in fact, one such area. But battery testing could be done by volunteers. Cheap and easy to post both batches of batteries to be tested, plus simple and standardised devices to test for retention and capacity.

We are all aware that some products cannot be tested without specialised and expensive equipment; I’ve already stated that at the outset but – and it’s a major but – there are numerous cases of testing that could be done by volunteers, that don’t require such equipment and could easily and cheaply form the basis for a testing article to be checked and written up by Which? staff.

On the subject of batteries, Which? tests (non-rechargeable) batteries in medium and high current applications. I would use rechargeable batteries here and ordinary batteries only in applications where they are likely to last a year or more – clocks and remote controls, for example. Capacity does not matter much here and I often choose ones that cheap and don’t tend to leak. Cheap Kodak batteries from Toys R Us have served me well, though I will have to change my supplier. 🙁

As I have said, some – possibly many – products are not within the scope of individuals if they are to be tested properly to achieve meaningful results. I’m not sure about batteries, as a lot of types and samples would be involved plus standard equipment to charge and drain at different rates representative of the various products within which they will be used. Probably best left to a properly equipped lab and experienced testers. Some information could be gleaned from user experience.

There are so many products with widely differing requirements that it would be difficult to train and equip willing volunteers. However, if some format could be agreed as worthwhile and if Which? would cooperate I’d be happy to take part.

I use a Panasonic breadmaker and it crossed my mind – is that product suitable for home testing? on the face of it, I suppose in principle it would seem to be. However, to do that would require the available breadmakers to be tested. They should be used to make a range of typical bread products products. They should use a range of different brands of flour presumably, as these may differ in properties. Several samples would need to be made – mine produces, for a standard white loaf, large, medium and small, dark medium and light crust. The resulting loaves would need to be properly assessed for bake, texture, consistency for example. That is an awful lot of testing (and bread) to achieve even basic results, plus a knowledge of final assessment. This is I think typical of many product types.

The example of ink jet vs laser printers would need a range of printers testing several standard inputs with a variety of inks on a variety of papers, and a means of objectively assessing the quality of the output. Is this something an individual could do? I’m not sure. A lab set up specifically to do such rests with knowledgeable testers would seem the best bet.

It does cost money, but then we do give Which? £100m a year to work for us.

malcolm r says: Today 14:34 As I have said, some – possibly many – products are not within the scope of individuals if they are to be tested properly to achieve meaningful results.

Well, W? only tests a comparatively small range of many products and – as I believe I’ve shown – the restricted range they do test plus the narrow range of tests carried out suggest that a trained and willing volunteer could, indeed, do a very satisfactory job.

I’m not sure about batteries, as a lot of types and samples would be involved plus standard equipment to charge and drain at different rates representative of the various products within which they will be used. Probably best left to a properly equipped lab and experienced testers.

But that’s the thing: as I’ve said they don’t test a wide range of battery types. To be precise, they tested only six types in the August tests – hardly qualifying as ‘a lot’ and well within the scope of a trained volunteer. Furthermore, a simple all-in-one device to test charge, capacity, drain and endurance could be easily and cheaply made and sent out, along with the packs of batteries for testing to the trained volunteers.

I think we have to be very careful about being excessively negative here; I started this thread because the cost and time involved in testing was presented as a defence by Which? for not testing a greater range of products. . I know there is a strong force within the organisation that wants nothing whatsoever to do with volunteers – willing, trained or otherwise – and to see negative perceptions about using volunteer testers in here will comfort them greatly. It does, after all, save them the trouble of having to mount an argument for not doing it.

But Which? is supposed to be a cooperative endeavour; it’s a charity and is supposed to use the resources it has in the best possible way. We are a resource; possibly the greatest resource they have after the income stream – which they wouldn’t have without the subscribers. We’ve offered numerous times to help but the offers have never, ever been accepted. Now the organisation is having to make a lot of staff redundant because of falling income. I’m not sure they can afford to ignore the offers of help for much longer.

I am far from being negative as my previous posts about Which? engaging with Members have demonstrated. What I am trying to do is to be realistic about what can be achieved by “volunteers”, having come from a laboratory and testing background. We simply need to understand what is involved in doing meaningful testing and choose those things that amateur home testers with limited equipment and resources can accomplish.

I want to see really useful and authoritative assessments of products that will help consumers buy wisely. I believe in many cases that requires knowledgeable testing by experts using appropriate facilities. I hope Which? use these in “their” test laboratories. I am all for Members providing help where they can. That is not disagreeing with your proposal but simply injecting a note of realism into what is possible. I’d suggest you pick a product group and explain exactly how you would test it, so I (we) do not misunderstand or misconstrue your proposal. At the moment it seems very general.

I would like to know from Which? just how much of their testing is done in collaboration with their European equivalents. I would hope it would be the rule rather than the exception since it seems to me that would benefit us all. However, I have not, for example, seen any reports on washing machine durability based on test.de’s accelerated life testing. Maybe I’ve missed it but this would be an example where we might be missing out.

Which?’s reports on tested products must, above all, be trustworthy which does limit the scope for volunteer testing, but I think there are some products that could be tested satisfactorily under domestic conditions. It could be made clear to readers that such tests included assessments by trained volunteers.

What is remarkable is that despite the effort and expertise put into scientific testing, many people choose to buy, or not to buy, something on the basis of its appearance and/or price rather than its functional performance.

Why should anyone have to test batteries? The manufacturers should be required to label them with life and output statistics derived from a certified and audited quality assurance system. There is quite a bit of useful performance information on the packaging of LED lamps – until you get to the lifetime forecast which is not much more than a guess rounded to the nearest 5,000 hours.

I firmly believe that there are many tests that could be done by trained volunteers. I won’t use the word ‘amateurs’ since that embodies a pejorative stigma and I note that those who’ve posted negatively about trained volunteer testing have used the term pejoratively.

This thread is a response to a Which? staff member who argued that W? is unable to test everything or even many things as testing is too expensive. It’s not our job to attempt to dismiss positive suggestions made to Which?; if the organisation feels something is not suitable then they’re more than capable of mounting their own opposition to it and far better placed to give precise details about why something is felt to be unfeasible. At that point we can debate further with them but unless someone is actively employed by Which? in their testing teams then I believe creating arguments for why something won’t work is extremely counter productive, not to mention doing the work of those who oppose volunteer involvement for them.

Interestingly, I note one of the earliest posts on the preceding page is about Alfa’s suggestion for a sustainability topic, a topic that W? invited her to create and which was submitted over a month ago. Nothing has been heard since. Five of us cooperated on a simple FAQ about EVs; that was almost a year ago and nothing has been heard since.

The organisation is extremely prejudiced against volunteers and without any good reason. This happens in all sorts of organisations; those who work in them become overly sensitive to the ‘non-professional’ becoming involved, in many cases for no good reason other than protectionism.

Which? should be very different; it was founded by volunteers, for goodness sake, copies being run off in a garage in Bethnal Green. Has it become too concerned about its own importance, perhaps?

Whatever the reasons, making any changes to the organisation is an incredibly difficult and and almost sisyphean task. They certainly don’t need us to do their job for them by arguing against those changes.

If a Which? subscriber is looking to benefit from access to Which? product tests, then, for such folk, it may be argued that the financial and other benefits of those tests should outweigh the costs of their Which? subscription.

From that, I think it makes most sense for Which? to test relatively expensive items. For example, before settling on a reasonably inexpensive and fairly durable blender, some friends of mine tried a couple of very cheap ones that plainly did not work well or weren’t durable. But, thanks to modern legislation, those cheaper models were readily returned to the shop where they were bought and full refunds were obtained. But they could not have done that as easily with large white goods such as washing machines, dryers or cookers.

Secondhand shops in general (including many charity shops) also now provide an inexpensive route for home product trials, if you think you might enjoy a product but don’t want to risk the full new price.

As it does with cars, Which? could supplement the lab testing of products with member surveys. For example: “If you’ve bought a new washing machine recently, tell us what you think of it and why?”. There would not be any need for homes to try complicated laboratory style tests, but, nonetheless, the aggregation of data from many consumers could help find the products that have least flaws and the ones best liked. Although the likes of Argos and Amazon already provide some facilities for that kind of feedback, they don’t manage its collation and analysis much within their efforts.

John: your point about battery testing is well taken 🙂 But if they continue to do it, it is something which I believe can be done very effectively by trained volunteers. If we look at the most recent test it focussed on four key aspects:

Aspects tested and reported:
Stated capacity
Single charge life expectancy
Overall long-term use
Charge retention

I believe this is a product tailor-made for volunteer testing. Ignoring the fact that two columns in the W?Mag report effectively duplicate each other there is nothing in the test that would militate against volunteer testing. A simple and cheap device can be procured to measure the final three categories (the first is given on the battery label) in a standardised way.
Some things can’t be tested at home: mattresses and Artificial grass, for example. But as I’ve already shown many of the test comments about printing were very imprecise, to put it mildly. I just think we could save the charity a fair bit of money.

Derek: that’s a good observation. Maintaining a testers panel could be a very good way to measure longevity and real-world performance against the lab tests. The panel members would have to keep log books, possibly compiled by the same people who design the car surveys, but it could make a substantial contribution to the testing regimes.

Ian – I would expect assessments by trained volunteers to be only one part of a test report with some, most or all of the technical testing being done in a laboratory under standardised conditions. If volunteers covered only half the test [on average] on each product that would enable more product testing to be done.

My point about battery testing was just one example. There are hundreds of products where their fundamental characteristics and performance data should be provided by the manufacturers. That is one of the benefits and economies that the investment in quality systems was developed to achieve. Neither test labs not volunteers should have to spend time measuring the dimensions of products, their capacity, speed, weight, watts, whatever. Unfortunately we have had a history of imprecise or deliberately wrong data which has required independent checking. I see volunteer testing as covering the useability factors, and the sensory aspects that no measuring stick or gauge can probe. The Ward Fist of Quality would be a useful tool for assessing large appliances and furniture, and the Mark One Eyeball can also detect things no machine can spot.

You don’t suppose there is some sort of age or experience prejudice against the use of competent volunteers, do you? Some of Which?’s committed supporters have been performing domestic, motoring, tech and leisure functions for decades and might know too much perhaps. They might also have wisdom and insights not found in younger generations. Lots of public activities are run almost entirely by trained volunteers – museums, urban farms, heritage railways, nurseries, adult education – and very competent they are too; I don’t think there is anything to be frightened of in letting outsiders contribute their time, interest and expertise; Which? shouldn’t feel threatened by this.

patrick taylor says:
11 August 2019

https://batteryuniversity.com/index.php/learn/article/charging_lithium_ion_batteries
The web has empowered individuals to seek information. There is top quality resource out there as the site above. I have repeatedly said that the charity in its educational remit should be a director to good resource for those who need more than a simplistic answer.
It would seem basic logic that if Which covers batteries then it should also provide useful links to expand the subject.
In its base support there must be thousands that are using decent sites or videaos that could be whittled to one or two excellent ones by dedicated sub-groups.
All of this presupposes that the staff of Which, and the Trustees, realise that it is an increasingly iisolated single source of rather simplistic information and testing. They need to embrace a more open way of working and a clearer view of what the charity is about. And this needs to include obviously more rigorous weeding of user reviews that are important for the functional usefulness by subscribers.

As I have also said there needs to be a centralisation of user complaints rather linke that offered by the Dutch Consumentenbond where there is an open complaint site that automatically forwards on the details of the complaint and if approptiate machine type, production, purchased etc. Encouraging the nation to funnel complaints through this conduit allows companies to also respond but importantly does show common faults on brands and models in a meaningful way which would provide early alerts on things like dryer fires, rogue burning Vauxhalls, enforced buying of PC set-ups etc. Surely using the Web and this funnel is hugely empowering to consumers – but very likely viewed by businessmen as a totally bad idea which should not be implemented.

A charity leader has pointed out that charities can ossify and the careers of staff assume more importance than the mission. Certainly under PVS the pay of the executives, and the motives of those that agreed the series of bonuses, would seem to indicate a serious loss of mission focus and entrenching executives.

Lastly as to the power of the volunteer – Citizens Advice is vastly run by volunteers in fairly technical matters – a mere 21,000 plus. Slightly less technical work perhaps are the 70,000 volunteers of the National Trust of England and Wales. So please Trustees grasp the nettles of a charity that refuses to utilise its devoted members other than in “consultations” which lead nowhere.

patrick taylor says:
11 August 2019

batteryuniversity.com/index.php/learn/article/charging_lithium_ion_batteries
The web has empowered individuals to seek information. There is top quality resource out there as the site above. I have repeatedly said that the charity in its educational remit should be a director to good resource for those who need more than a simplistic answer.
It would seem basic logic that if Which covers batteries then it should also provide useful links to expand the subject.
In its base support there must be thousands that are using decent sites or videaos that could be whittled to one or two excellent ones by dedicated sub-groups.
All of this presupposes that the staff of Which, and the Trustees, realise that it is an increasingly iisolated single source of rather simplistic information and testing. They need to embrace a more open way of working and a clearer view of what the charity is about. And this needs to include obviously more rigorous weeding of user reviews that are important for the functional usefulness by subscribers.

As I have also said there needs to be a centralisation of user complaints rather linke that offered by the Dutch Consumentenbond where there is an open complaint site that automatically forwards on the details of the complaint and if approptiate machine type, production, purchased etc. Encouraging the nation to funnel complaints through this conduit allows companies to also respond but importantly does show common faults on brands and models in a meaningful way which would provide early alerts on things like dryer fires, rogue burning Vauxhalls, enforced buying of PC set-ups etc. Surely using the Web and this funnel is hugely empowering to consumers – but very likely viewed by businessmen as a totally bad idea which should not be implemented.

A charity leader has pointed out that charities can ossify and the careers of staff assume more importance than the mission. Certainly under PVS the pay of the executives, and the motives of those that agreed the series of bonuses, would seem to indicate a serious loss of mission focus and entrenching executives.

Lastly as to the power of the volunteer – Citizens Advice is vastly run by volunteers in fairly technical matters – a mere 21,000 plus. Slightly less technical work perhaps are the 70,000 volunteers of the National Trust of England and Wales. So please Trustees grasp the nettles of a charity that refuses to utilise its devoted members other than in “consultations” which lead nowhere.

Ian said:
“how, for instance, is the reader supposed to differentiate between ‘Fantastic black-text print quality’ and ’Superb at printing black text’?

A big failure of product reviews is the lack of images other than the standard manufacturer ones. Firstly the product out of the box showing what exactly is in the box then not showing any results or proof of testing in the form of images. Camera review sites put cameras through a standard set of tests and show results allowing the reader to judge for themselves. Showing printer results would allow readers to make up their own minds.

Malcolm said:
Where I believe members could help product testing is in usability – handling a new product and reporting in a structured way on how it performs…
However, members who are interested might take an example and spell out here exactly how they would test a specific product so we can better discuss the pros and cons of what can be done at home.

I agree with that malcolm, having brought up the shortcomings of a quite a few product reviews.

DerekP said:
From that, I think it makes most sense for Which? to test relatively expensive items.

I agree. If people are always looking for the cheapest items, are they likely to subscribe to Which?

patrick taylor said:
All of this presupposes that the staff of Which, and the Trustees, realise that it is an increasingly isolated single source of rather simplistic information and testing. They need to embrace a more open way of working and a clearer view of what the charity is about. And this needs to include obviously more rigorous weeding of user reviews that are important for the functional usefulness by subscribers.

Absolutely agree with that patrick. Stars out of 5 with no explanation of how the scores were achieved are quite meaningless. User reviews need to be reinstated with follow-ups after 6 months or so.

There are definitely rolls for volunteers within Which? Will they listen to us though?

DerekP says:
11 August 2019

It might also be a good idea for Which? product testing to begin by testing Which? itself. This would be a task best tackled with the aid of subscriber reviews and ought to lead for useful feedback, to generate improvements to Which? testing coverage and methods.

John, using Which? for technical testing with subjective assessments by “volunteers” is a good combination for many products.

And, as DerekP suggests, better Member feedback on products will be invaluable, including long term experiences.

Let’s hope Which? understand the positive help being offered by interested Members and will join in finding a way to best make it work.

patrick taylor says:
11 August 2019

Regarding expensive items as being preferable testing subjects. I agree but with a caveat that what is frequently used is also very important if there is limited life with some products. Think toasters with no replaceable parts and a life span of two years compared to my Dualit ofnearing 30 years of heavy use.
This angle ties in well with environmental concerns.
There are also induction hobs which cost under £100 which work very well however there is a problem in that many pots and pans claim to work with them but the efficiency varies greatly. This has two effects more power used by some who soldier on and others who throw them away. I am sure all consumer groups would see the benefit of having an efficiency rating [ and minimum] for all pots and pans claiming induction functionality. My own testing was very revealing.

As I am on testing I can confirm previous comments that over two years of testing the Roomba is far superior to the Neato. The Neato is very confused by speckled tile floors constantly stopping and asking for its optical sensors be cleaned, and of course approaching obliquely a drop it topples over.
None of the four consumer sites I frequent mentioned these flaws. My feedback you might hope would be incorporated in future tests.
I would add that Choice was dismissive of robot vacuums true effectiveness on carpets. And that at the time Which?’s top choice the Dyson was mid-range according to two other consumer reports. It may reflect different testing but an interesting area to discuss.

DerekP said:
It might also be a good idea for Which? product testing to begin by testing Which? itself.

We have had a couple of surveys Derek, but I don’t know how much value they were.

We had a survey on Which? Connect August 2018 ago that started with questions to agree or not with like:
– W?C is a great tool for keeping members involved in the work Which? does.
– W?C makes me feel W? is listening to its members.
– W? proactively ensures that I am getting the most I can out of my subscription.
– The layout of surveys on screen makes them visually appealing.

I finished with these comments restricted to 100 characters:
– Suggest trial run of surveys with Convo regulars with private Convo page to discuss and suggest improvements.
– Truthful answers often missing. Suggestions on the Convos to do a trial run before sending out has so far been ignored.

There was also a ‘Future of Which?’ survey in May 2018 with questions that were all over the place.

Some more relevant statements asking how interested you were or not:
– Having a say on what products and services W? should be testing and investigating.
– Getting involved in testing products and services on behalf of W?

W? is a charitable social enterprise. We need volunteers to help us make decisions about the future of the organisation:
– Sharing your ideas and thoughts on the important decisions W? makes as an organisation.
– Having a say on the direction of the W? organisation.

There was then a section on a number of things that you could potentially do with W? and how interested or not you were:
– Join the W? consumer research panel – Help W? shape ideas for product and services research and testing.
– Become a super tester for W? Train to help W? evaluate local services such as care homes, airports and dentists.
– Become a W? regional representative.
– Become a regional W? campaigns ambassador.

I finished with this comment:
The questions do not allow for people already involved. You can’t be made to feel more involved if you are involved already, so can only answer honestly that it wouldn’t change how I feel.

Which? gives these Dualit toasters (the British made ones with replaceable parts) a poor score because by modern standards they do not perform well, though they do enjoy cult status. The company does have better modern designs of toaster that perform well and it would be good to see replaceable parts, especially if sold at a more sensible price than their existing spares.

One of the problems with assessing the performance of certain brands (e.g. Apple, Dualit, Dyson, Miele) is that views can be coloured. Hopefully Which? testing guards against this but I do detect some brand loyalty is users’ reviews.

I agree. For Which? tests to be trustworthy the testing must be disinterested and indifferent, but training should secure that.

Ian said yesterday 10 Aug https://conversation.which.co.uk/technology/which-discussion/#comment-1572328
“But battery testing could be done by volunteers. Cheap and easy to post both batches of batteries to be tested, plus simple and standardised devices to test for retention and capacity.

The reason I sounded a note of caution on testing, say, batteries as suggested apart from the equipment involved is the time to test.

You’d need, I’d suggest, sample batteries from a few batches, and maybe around say 8 batteries in total, to ensure consistency of product. Rechargeable batteries would seem to have two principle characteristics to measure – time to discharge to their cut-off voltage and number of recharge cycles until they reach a minimum acceptable capacity. For a high current device the discharge time can be 7-8h, and a good battery will recharge 200 times. That amounts to 1400h minimum of test time per sample (around 60 days). Multiply that by the number of samples and the number of different manufacturers/types to be tested and it is an awful long time unless you have extensive facilities.

That is my amateur interpretation. Which? may have a different test regime and it would be useful if they explained it here to see if it is something that could be done at home. Whatever else, a strictly controlled test regime is necessary to achieve meaningful results.

Although the August Which? test article only looks at 6 types of AA and 7 AAA the online review looks at 16 AA and 15 AAA.
https://www.which.co.uk/reviews/rechargeable-batteries/article/best-aa-rechargeable-batteries
https://www.which.co.uk/reviews/rechargeable-batteries/article/best-aaa-rechargeable-batteries

A little about how Which? test is given here.
https://www.which.co.uk/reviews/rechargeable-batteries/article/how-we-test-rechargeable-batteries

However, you may have other ways you’d suggest testing or assessing, perhaps in conjunction with Which?’s more formal tests?

My reference to “amateur” testing was in no way disparaging, but simply to distinguish it from “professional” testing.

patrick taylor says:
11 August 2019

I think brand loyalty is a problem and needs to be considered carefully. I can understand you may think I am biased towards a toaster that has served 26 years and which has had two sets of elements and a new timer in that time. In terms of damage to the planet perhaps there should be awards for avoiding buying multiple toasters ……
For another view there is this which does mention technophobia for simple items:
bestreviews.com/best-4-slice-toasters
In any event I am not wedded to Dualit other than a love of frugality and toast. And note how parts are not a subject covered by bestreviews.

I certainly agree we need to consider sustainability and look forward to discussion about ‘right to repair’. I would like to see a requirement for parts like heaters to be replaceable and affordable spares to remain available for a set period as a prerequisite for a toaster to be offered for sale. I remember discussing this with friends when I was a student, bemoaning lack of repairability.

Until we have the right to repair, I would like Which? to inform us of the availability of affordable spares when it tests products. If a manufacturer does make parts available to the public, the chances are that DIY repairs are fairly easy and that having the job done would not be excessively expensive. I am certainly not anti-Dualit and was glad to see the announcement of a kettle with a replaceable heater, but I have not seen a price and I believe that it has to be done by the company. I would have more respect for the company if they produced toasters that combined repairability with good performance rather than focusing on one or the other.

The website you quote does mention the importance of a guarantee and I believe that a decent guarantee or extended warranty at a sensible price is probably the best indication that a product may be durable.

(Incidentally, I do hope that you managed to get your old washing machine working again.)

Repairability has been a requirement for Which? to examine and promote by a number of Convo commenters as a contributor to durability. I don’t expect all products to be repairable – some may be prepared to pay less, perhaps for a product that is going to see less use; however I’d like to see a good proportion of products offered that have sensibly-priced spares, easy to replace, so the life of the product can be extended. I’d expect to see more pattern parts produced to feed this market just as happens in the car spares business, as well as parts from OEMs.

More durable products will not mean an economic loss to industry, because of loss of new replacement product sales. We will have more money to spend on other stuff. Up to the enterprising manufacturers to meet that market.

If we look at our much discussed example of white goods, cheaper products represent the bulk of sales, so making products that are uneconomic to repair means that they will be scrapped. There are many people who could buy better products but opt for cheaper ones and replace rather than repair when they break. It simply is not sustainable to produce large volumes of goods that will have a short working life and the most effective strategy to reduce waste would be to focus on making cheaper products last longer. That’s analogous to focusing on single-use plastics to tackle the plastic waste problem. White goods have dropped greatly in price and you can get a washing machine for as little as £200, which is the price I paid for a machine in 1982. A minimum price of about £400 might provide a basic but reliable machine and the secondhand option is always available if you cannot afford a new one.

More durable products are available but spares and repairs can be exceedingly expensive as some have found to their cost. It’s one reason some people switch to cheaper white goods.

Malcolm, I think you would be the perfect person to write a convo on product sustainability. Durability, repairability and the premature end of the life of products because they are not longer supported like computers and phones.

Would you be interested in this Malcolm? If so, happy to pick this up in an email.

We’ve got a similar convo in the planned obsolescence / right to repair / sustainability vein coming next week from the Restart Project (bit later due to August holidays), so this could fit nicely around that as well.

Jon, I’ll give this some thought.
We all have views and general comments on durability, repairability, and such. However, I think this is such an important topic that a good deal of work needs to be done to make it fact-based, rather than a collection of mainly generalised views and opinions. For example, we’d want to know just what products are sensibly repairable (as examples of what we’d like to see), what impedes that approach more generally, what the consumer wants (or should be legislated for), how products can be designed to be economically repairable, what would be needed to encourage manufacturers to do that, what infrastructure is needed to provide the repairs, how an affordable spares market might develop and the consequences of using common components in products (say motors and pumps in cleaning appliances).
I am sure a number of organisations are looking at this, hopefully with manufacturers and others, and that background would need pulling together. I would have thought a properly-researched investigation by Which? with other organisations would be of great value to consumers and may be better than a chat about it in a Convo.
So, I’m not sure what the usual sort of Convo would do. Perhaps you would let me know what the objective would be of what would probably turn out to be a chat about the topic.
One way it could work is if we, and Which?, could get people from the various parts of the industry, Which?, WRAP(?), and other organisations involved in different aspects of the problem to commit to following the Convo and making regular contributions, providing information and facts, responding properly to commenters questions and views. That way we might end up with a Convo that leads somewhere? It might also be useful to have a (very) small team putting the Convo together, and perhaps regularly summarising it, so the lead it gives embraces a number of viewpoints and experiences and shares the work.

Sorry, a bit of a ramble.

I agree. The problem is would it ever be published?

I do like Derek’s suggestion: it might also be a good idea for Which? product testing to begin by testing Which? itself.

Currently, all three of the £200 washing machines that I helped various family members to buy about 7 years ago are all going strong. So also is the £200 machine I bought about 30 years ago. Hence, I doubt that £200 machines are inherently incapable of leading long and reliable lives.

That said, I’ve also seen one case of a very heavily used and abused £200 machine failing to last more than about five years.

It would be really useful to collate more statistics here and see if there is a correlation between purchase price and reliability and longevity. We might then also learn if some designs last better than others.

DerekP. the collection of appropriate statistics seems key to reporting on product performance in the longer term. Whilst some brands change, many may perpetuate their own design and manufacturing philosophies, good and bad, that can produce generally better or worse products. As well as tracking product durability we could track fault incidence – such as, do any brands of washing machines stand out for glass breakage, ovens for glass doors shattering, tumble dryers for fires (or not); I think this needs a better approach than Connect surveys, and the patience to collect significant data for long enough to help properly assess experience and durability.

I agree that a £200 washing machine may well perform adequately. However, a few years ago Ken Watt responded to a similar comment with the costs involved in putting cheap machines on the market; after manufacturing costs there were the manufacturer’s margin, shipping, wholesaler margin, retailer margin, contingency for guarantee work, packaging, vat which, rather like a cheap bottle of wine after you deduct bottling, transport, duty and vat leaves very little for the contents. Nonetheless a decent manufacturer can make the best of this.

Even cheap machines can be made repairable, I would suggest. There will be extra cost in making a drum with a replaceable bearing but I do not see why that should be excessive. Avoiding chasing costs down to the extreme could still leave an “affordable”, if basic, machine.

I think with inexpensive toasters, washing machines, laptops and mobiles (to give but four examples), the time and trouble to effect repairs is hard to justify against the cost of replacement items.
For example:
When you can buy a secondhand laptop for £50, why spend £45 to buy a replacement screen for a similar machine?
But if even a five year old secondhand iPhone costs about £100, it can be worth spending about £15 to buy a replacement screen and then using that for a DIY repair on an iPhone 6.
Similarly for cars, I usually find that when MoT time comes around and my annual repair bills start to regularly hit the £1000 mark, it is time to get a newer car.

As far as I know, almost all washing machines now have a ‘sealed tank’, making it impossible to replace bearings, seals and ‘spiders’, so a failure means replacing the tank, drum and associated components. In the event of failure the machine will be unlikely to be economically repairable. This is a direct consequence of a design change that makes assembly easier and cheaper but significantly reduces repairability.

To reduce the risk of bearing/seal failure, don’t overload the machine, never use excessive detergent and use a lower spin speed if possible.

Malcolm – I agree with your final paragraph but manufacturers do chase costs down and what really annoys me is to see examples of this in more expensive products.

I have an old Espace – 25 years. As you say, MoT is the crunch time (although expenses do occur mid-term). This year it needed the back brakes replacing, and a new drive shaft. They probably equated to the market value of the car. I use it regularly for local journeys but its big attraction is its space – take out the back seats and you have a van. Yesterday it transported ladders, steps, tools and wood to help my son renew his balcony decking. Add the insurance, VED and MoT costs and it is an uneconomic proposition in money terms, but convenient and useful. and that counts for a lot, if you can afford it.

I take the view that an old car is still a car – it has the functional value of a similar but much more expensive version – so keeping it going has sense. At what point do you give in? I’ll report back when the time comes.

I agree with that Derek, but many products could be made with DIY, easy-to-replace slot-in parts and circuit boards that don’t require an expert.

Toasters are fairly simple items, so to make them with slot-in parts shouldn’t be too difficult to produce.

To replace I think it was the main circuit board on a Bosch washing machine cost us around £300 with a service engineer visit, an absolute rip-off on a machine that cost less than £400. A DIY slot-in circuit board could have made the repair economically viable.

We have an old Lenovo laptop that has broken hinges attaching the screen, so it cannot be closed. Not worth paying for a repair and a DIY repair would entail taking the whole thing apart. The last laptop I dismantled was not easy to reassemble.

One problem is in diagnosing the problem, usually entailing an “engineer’s” visit. My fridge freezer went wrong – stopped working totally – and that was down to a failed circuit board which was replaced under warranty. I don’t know what that would have cost otherwise (although my Worcester boiler did have a circuit board replaced which, with a transformer and ignition lead cost £125. I was pleasantly surprised). The problem arises, paying say £70 for someone to come and diagnose a problem that may not be economically fixable may, probably does, lead people to simply replace an older appliance with a brand new one when it stops working.

Maybe we should make such repairs vat free?

The spares market is a minefield, and can be a rip-off. It happens in commercial circles as well as in the domestic market.

There are repair workshops around the country where “volunteers” (I believe) get together and will look at products you are able to bring in to diagnose faults and see if they are fixable, and then fix them. I don’t know how widespread they are but this would seem a good way forward for portable appliances. The problem would be getting larger appliances to them for most people; maybe extending the network of refurbishing companies would be a good way of dealing with large domestic appliances – giving an exchange price for a repaired product.

Many products could be made in this way, Alfa, but I can see no way other than legislation to make products such as toasters repairable.

Circuit boards can be expensive to design but the cost of spares can be exceedingly expensive, far more than the cost of the components used in their construction. It’s a way of encouraging people to replace rather than repair their products.

Field service engineers tend to carry spare circuit boards and other parts where they have experienced regular problems. Several years ago I learned about what tends to go wrong with Worcester Bosch oil-fired boilers and the likely cost of repairs.

Some people dismantle scrap washing machines and sell circuit boards and motors on eBay.

Back in the 1980’s, I used to repair the toaster than my grandfather had given me.
These days, I have a simple cheap toaster that has worked well for many years.

With new toasters available from as little as £3.99, I doubt that it would ever be economic to repair my present one, if or when it eventually fails.

I’ve just spent some time grappling with a small desk fan that my friends kids dropped and broke last week. As far as I can see, it uses an a.c. shaded pole induction motor and the motor coil is now open circuit. So it looks as though that’ll be heading for the scrap bin soon…

My guess is that one of the wires from the coil has fractured where it is soldered to the terminal supplying power.

Many cheap appliances are not worth repairing, particularly if you pay for the labour. Providing they are properly scrapped to recover the materials then there is some sustainability involved. Maybe we should ban the use of non-recyclable materials in products where recyclable ones can be substituted?

I used to have an old Swan toaster – elements in the centre and a flap each side that held the bread – which needed turning over to brown each side. Replacement elements were available and I made new heat-proof knobs when the original ones broke. Maybe modern domestic appliances have just become to complicated? Bring back the twin tub? Perhaps not.

DerekP says:
12 August 2019

That’s certainly possible.
If I absolutely had to fix it, I’m sure I could but I don’t think I will.
On top of any other considerations, if I repair it and then return it, I become liable to some extent for the subsequent electrical and fire safety of the product.

I agree. Nowadays I’m very wary of doing any repairs for anyone other than family or close friends, especially with anything that plugs into the mains.

malcolm r says: Today 11:25
The spares market is a minefield,

Apropos which it was discovered some years ago, now, that even Boeing Passenger aircraft were discovering faked replacement parts being sent to those performing routine maintenance. Superficially impossible to tell, but longevity and stress tolerance often wildly below what was specified.

This is another nonsense of spares. My newer Espace needs a Xenon headlight bulb. £156.94 just for the bulb from a main dealer (which is no doubt what an innocent victim or company car owner would pay). Amazon Osram bulb £28.94. Or Osram from Amazon Warehouse – slight packaging damage – £18.10. I’ve ordered two of the latter.

Does the main dealer’s price include fitting? The ease or difficulty varies greatly depending or which car you have.

No – just for the bulb as I said. Access to the drivers side is quite easy. The passenger side requires a temporary rearrangement of the steering fluid reservoir, and a bit of wiring. The garage that looks after my cars says just bring it down and they’ll fit them there and then at no cost (I think it’s worthwhile changing both at the same time). Assuming it all goes smoothly, and doesn’t take long,a donation to their Christmas fund will result.

I thought I had found a main dealer that was competent and honest but they claimed that both my rear shock absorbers were leaking. According to my trustworthy garage they must have sprayed some WD40 in to simulate leakage and the car has passed two MOTs since.

Oh for the days when you could replace bulbs at the side of the road.

As bulbs are so important to the safety of a vehicle perhaps we could seek legislation to ensure they were easily replaceable by the owner. As it is a requirement in some countries to carry spare bulbs this seems rather pointless if you have to dismantle the car to fit them.

Like providing spare wheels, having bulbs that cannot be replaced easily is not going to happen without legislation. I suspect that the move to LED bulbs will remove the need.

France dropped the well known requirement to carry spare bulbs years ago. I carry a full set apart from daytime running lights.

We seem to think of Which? in isolation when considering product testing, its scope and, particularly, its cost. Most of the products Which? test are used by consumers throughout the whole of the EU. These consumers presumably want the same products tested as we do, and each of these EU countries has a consumers’ association. If all the appropriate product testing was funded and organised by the whole group of these associations I’d suggest we’d get a lot more done, more often, and better. Does Which? just want to go it alone or would it arrange an initiative, maybe through BEUC, to organise routine product testing Europe wide for the benefit of all the consumer groups and their subscribers?

patrick taylor says:
10 August 2019

Just to say that Which does do some joint testing with EU partners. How much is of course subject to Which’s transparency policy! Ha. This “lie” perpetuated distorts a lot of what the charity does. Admit that it pays outside bodies, announce proudly when it works with other consumer bodies, and when it works with other sources such as the Optical Express sufferers.

patrick taylor says:
11 August 2019

An amateur tests rogue fuses:
youtube.com/watch?v=IHA-WQEmsRE
Regrettably the supply cahin is not mentioned.

The ‘amateur’ is a qualified electrician. He has another YouTube video about supposed BS1362 15 and 20 amp fuses – ratings that were never covered in the British Standard: https://www.youtube.com/watch?v=InpT_QMXaZ8&t=126s His videos are generally very good and I have learned a lot, but some of the points he makes about British Standards are not correct. If he had access to the standards I’m sure he would be better informed.

patrick taylor says:
10 August 2019

It is good to see serious consideration of “testing” and how Which? accomplishes this. As I belong to several consumer bodies and actual can read the magazines or on-line offerings of further organisations I can say categorically that Which? punches below its weight.

We all realise that Which is very reliant on outside expertise for the testing, of a single item , and then the results I suspect are translated and edited for publication by staff who have very likely no familiarity with the item.

The fact that Which? sells the Best Buy label and that in service these Best Buys reveal frailities that the public rightly feels should have been mentioned or found out in testing.

The Amazon reviews mentioning WhichBB’s in unflattering terms must do a great deal of damage to the reputation to our organisation. The “canaries in the coal mine” were of course the members who could make comments themselves and highlight inaccuracies and faux pas that Which could of and should have evaluated to see if a change was necessary in the BB status. This facility having been removed both subscribers, members , and then the general public perforce post on an open site.

It seems peculiarly inept.

Which?’ has finally taken a stance on fake reviews and it almost seems to save itself from the same problem it has thrown the baby out with the bathwater. Several years ago seeing the problem I suggested that subscribers use their correct names and also the length of time they had been a paying subscriber to avoid fly-by-night damaging smears or puffs of products.

To reduce even further rogue postings and to make even more certain of veracity one could also be active in registration of machines, have a roving duo of engineers to ascertain veracity of complaints, and of course more transparency.

Other things

1. My scales are from Aldi or Lidl and work fine. Those two companies are aware that test.de [ Stiftung Warentest] in Germany do rigorous tests and I sincerely believe that because of this scrutiny that these companies are rigorous in quality control – to the benefit of the customer.

It is said that Stiftung Warentest is the third most trusted organisation in Germany which is interesting vis-a-vis where Which might be placed by the UK public.

2. Which publishes daily for the Press and yet stating the obvious about trains and planes to be recycled in the daily Press is an interesting dichotomy versus its non-comments about leaseholds and rogue companies.

Essentially where it should be crusading on a serious wrong it is silent. It does not lead, it does not consult its members if they think it is worthy subject, or spending say £50k of staff time on , it does not expose the scale of the villainy perpetrated against consumers by the new builders and the pet solicitors.

I do think that Which is escaping from a pro-business slant and the magazines are improving. It does need to think smarter and be very obviously pro-consumer on worthwhile matters. It also desperately needs to mobilise the power of the members and particularly improve the testing side/comments.

Patrick wrote: “I do think that Which is escaping from a pro-business slant and the magazines are improving. It does need to think smarter and be very obviously pro-consumer on worthwhile matters. It also desperately needs to mobilise the power of the members and particularly improve the testing side/comments.” I agree with that.