Consumer Reports Inconsistency

[CapitalTruck]

Throughout my childhood my parents have steeped me on the virtues of Consumer Reports. They always make their vehicle purchases based upon the magazine's recommendations. However, I have begun to really question their reliability/repair history ideas. The most recent indicator of something being amiss is the fact that the Pontiac Torrent is on their recommended list with a better than average reliability/repair history. On the other hand, the Chevrolet Equinox, which comes off the same assembly line (and has the exact same guts), gets a worse than average rating and is not recommended. This is the sort of thing that gets my dander up.

[Rolandd]

This is standard fare with CR. I have read reports for years where sister vehicles got different ratings on the same components such as engine and driveline reliability. I could understand if there were differences, but these reports have been on vehicles that were pre-computer tailoring for brand identity (e.g.: smoother shifts for GMC).

[asilverblazer]

The Pontiac isn't as "GM" as the Chevrolet, they cater to the ignorant and Chevrolet is stereotypical cut rate, which can't be as good as a top notch imports. Pontiac on the other hnd isn't a Chevrolet and therefore they can say it's better with out giving props to a GM, at least to the ignorant... Goofballs.

[tnord]

consumer reports is useful if you're looking to buy a dishwasher, otherwise, it's trash. especially for automotive stuff.

[SilveradoLAX7]

CNN reported that CR is in some deep S*** regarding their reviews of car seats. Evidently they trashed most of the items tested stating that survivability was unlikely in a 38 MPH crash. The problem was that the seats were tested at about twice that rate of speed. Another case of irresponsible reporting from CR.

 

here is the link: http://www.cnn.com/2007/AUTOS/01/18/car_seat/index.html

 

-Jay

[ratchet]

I believe the pontiac torrent/chevy equinox are made overseas these magazines seem to love anything made over seas.

[Wingnut]

While CR is great idea in theory...You know...Unbiased testing FOR the consumer...Once you throw the human equation in there, it gets all mucked up.

 

Because people are wishy-washy, opinionated, inconsistant weirdos.

 

They're a nice guideline, but I'd hardly base my purchase of anything solely on what they say.

[Norseman]

All the examples above just underscore why I take anything comming from CR as worth little more than a sack of horse hockey.

[67ChevyRedneck]

If I recall correctly, it's been a while since I read it, but in their most recent car study they recommended the Toyota Matrix and rated the Pontiac Vibe lower than the Matrix....

 

THEY'RE THE SAME d**n CAR!

[asilverblazer]

All the examples above just underscore why I take anything comming from CR as worth little more than a sack of horse hockey.

 

<{POST_SNAPBACK}>

 

 

 

I'd say worth a little less.

[95Sierra2500]

I don't read and can safely say I've never picked up a copy of Consumer Report. I feel that it would be more valuable shredded and hanging on a toilet paper rack...a little scratchy, but MUCH more useful.

[rjirvine]

As I believe, the ratings for reliabilty come from subscribers...a very "iffy" way get objective data.

[az'03Silverado]

I just buy a car I like, take good care of it, and it lasts just as long as any other.

[mschuyler]

I think the issue of twin brands getting different reports reflects the way CR gets their data. There are several issues:

 

1.) "Members" fill out a suvey and send in their data. One getting a different rating simply reflects the data they compiled. It also betrays a rather simplistic use of statistics. You would think they could take a little more time and match up the twins. They've 'split' their data and increased the chances it is invalid.

 

2) That the two are that far off also reflects the fact that the sample size is insufficient to draw any conclusions. As the sample size increases, those differences should (and would) lessen.

 

3) Another issue is the people furnishing the sample. They are CR subscribers. That is NOT a random sample of the population that buys cars. What kind of people subscribe to CR? Do you? Maybe they are all nice guys, but my guess is they don't quite have the same world view as the rest of us. This is a sample taken from subscribers who are willing to take the time to fill out the form and stick a stamp on it and send it in. This is a very 'specialized' group, not a random sample of the population.

 

I've taken a few statistics classes. The FIRST thng you learn is how to validate the sample size. You have to use a big honking equation to do it, and you'd better use the right one for the circumstances you have, but that's why statistics can be a very powerful tool in science. I don't see any evidence that CR even uses the equations. Maybe they do, but it looks like they just compile the data. That doesn't really even qualify as a valid use of the statistical method. It is also the type of behavior that gives statistics (which really is powerful) a bad name.

 

Having said all that, I still think more general trends are apparent in their reports and are probably valid. I say 'probably' because statistics NEVER is certain. The usual method is to say, "This report with this data has a 95% chance of being valid, but there is still a 5% chance that the differences this report shows are attributable to chance." If you tighten up the equations and put more pressure on the data, you can get to 99% and 1%. But the data has to be more 'obvious' to support a more stringent conclusion. The easiest way to be more stringent is to get a larger sample. That way a few odd data points won't throw you off, but at some point more data does not get you greater validity. The curve flattens out. And that is why, incidentally, you need only a little over 1,000 voters to call a national election. If they truly are a random sample--it's valid.

 

If you see a brand that is largely solid red dots across all their models and compare it to a brand that is largely black dots over all their models, then a fair conclusion is that people are having a whole lot less trouble with the red dot brands. The reason is that when you put all the data together for all the models of a brand, you very likely do have a statistically valid sample, whether you know it or not. The conclusions you can make are equally general and broad. Red dots, on the whole, are better than black dots. That doesn't mean you can draw conclusions about the odds of having a problem with your tranny in your model.

 

What is clearly not valid is reasoning from the specific to the general. Because you have discovered that twin data doesn't look right because the reports are different for the same twin cars does not invalidate the overall trend CR is reporting. Don't throw out the baby with the bathwater. You guys who are trashing CR totally are missing the point entirely. Given the data you have before you, your reaction isn't logical. Red dots are still better than black dots. You may not like it, but the fact is that people report more problems with GM per vehicle than they do with Toyota. That is not anti-American or anything close. It simply reflects reported incidents of problems. You can still love your Duramax over a tinny Tacoma and feel good about yourself. It's just that, statistically speaking, you are more likely to 'experience' some 'black dots.' If you're willing to live with that (I am.) Okay.

 

Of course, CR is only interested in problem reporting. It's not like they care about HP or foot pounds of torque or even more obvious things like towing capacity. They are in a different world. I saw a cartoon in Motor Trend or Car & Driver a few years ago. In the distance are a couple of guys with clipboards and white coats and eye glasses carefully watching cars being shoved off a cliff. The punch line is:

 

"There goes Consumer Reports testing cars again."

[BowTieDude]

I wouldn't use Consumer Reports to buy dog biscuits!