There are numerous reasons why products may fail only some of the time. Sometimes, safety standards call for a product to be tested in a variety of ways; the product may fail when used one way but pass when used another way. Test-to-test repeatability and lab-to-lab reproducibility can also contribute to variations in test results. Manufacturing variations may play a role in intermittent failures, as can changes in a product’s materials or design. So even under optimal conditions, it is not uncommon to see some variation in test results—up to and including situations in which some samples of a given model pass a test, while others fail.
But our point of view is that, when it comes to safety, no test failure is acceptable—and that’s especially so where child safety is concerned. Products should be designed with enough margin of safety that they pass each and every time.
One recent example: When we first tested the Baby Jogger City Mini stroller, we were troubled to find sporadic problems with the harness restraints. The voluntary industry standard specifies that the harness system must hold a test dummy in place even when the stroller is turned upside down. But in our tests, the results were inconsistent. In further testing, when we pulled directly outward on the buckle it held fast. But if we pulled sideways on the buckle, it could release with as little as two pounds of force.
When we shared those results with the manufacturer, it turned out that the company had already changed the design to put stronger buckles on newer strollers. Nevertheless, the Baby Jogger Company, working with the Consumer Product Safety Commission, voluntarily recalled some 41,000 older strollers that had the earlier version of the buckles.
Meanwhile, Consumer Reports testers, who continually update our test procedures, have revised our stroller protocols to include the sideways pull test, even though it is not part of the industry standard test protocol. Reason: We want to anticipate problems that might arise under ordinary, real-world conditions, and not just in an industry or government test.
Another recent instance of intermittent failures: One of our testers was exercising on the Stamina model 1725 elliptical trainer when the left pedal of the machine came loose and fell to the floor. We later found that on both the Stamina 1725, and on the similar Stamina 1772, the bolts holding the pedals in place would sometimes work their way loose as the user pedaled the crank. The problem didn’t happen every time—but it happened often enough that we rated both machines a “Don’t Buy – Safety Risk.” And in July, the CPSC recalled both Stamina trainers.Perhaps the most dramatic recent example of intermittent failure involved an outdoor gas grill called the Broil King Signet 90. As part of our standard testing, we always put several very fatty ribeye steaks on each grill at high heat in order to see if the dripping fat catches fire. Most top-rated grills flare up momentarily and then stop. But the Signet 90 flared up quite a bit, made a popping sound, and then black smoke came out from under the lid. The testers turned off the propane tank and pulled it out of the cabinet under the grill. But as they did, molten metal dripped down onto the area beneath the grill, just missing one tester's hand. The grill's firebox had melted.
We tested two more Signet 90 grills. In one of them, the firebox also melted. In the other it didn’t, but there was cracking and deformation. Yet we cautioned readers about the grill last year, and its maker, Onward Manufacturing of Canada, voluntarily recalled the model and sent buyers a kit intended to fix the problem. (As it happens, we also later criticized the kit).
The point of these three examples is that even when failures are intermittent, we pay serious attention to them if they affect your safety. We believe it is our responsibility to tell consumers that they face a potential risk. Our tests are not designed to explain *why* a product performed as it did; that’s a matter for the manufacturer and/or regulators to determine. Nor are we in a position to compel a manufacturer to make changes in design or even to recall a product, as, say, a government agency might. Moreover, our reporting on intermittent failures may mean government agencies and manufacturers don’t always agree with us. But it means we are fulfilling our responsibility to our only constituents: Consumers. The fact that a product fails only intermittently is not necessarily a good sign, in our view. We think buyers have a right to expect that the products they buy will be safe to use under real-world conditions each and every time.
The issue with Orbit's test doesn't appear to be an intermittent failure, but rather a failure of the testing facility to use the device as directed/designed.
Either Consumer Reports is in fear of another round of bad press after 2007's car seat report debacle or they REALLY hate Orbit. Regardless, I've lost all trust in Consumer Reports.
KC nailed it. Instead of addressing the issue of why they failed to use the SrongArm, Consumer Reports lectures the unwashed masses on how an "intermittent product failure" can occur. Gee thanks. If Consumer Reports read ANY of the many responses to their report, they would know that their readers who have installed this seat understand how an "intermittent product failure" could occur: by blatantly disregarding the manufacturer's instructions. Heck, even without looking at the directions the average 4th grader installing this seat would say "Hey, what's this big knob on the front for? Oh, look Daddy, turning it makes it tighter."
The directions say it takes an average of 11 rotations of the big knob to achieve proper tension for an average install. How many spins did CR give it: 0. And then they want to suggest that it's somehow a manufacturer defect. What's the basis for this? Seriously, between undertaking the test in this manner and then spinning the criticism this way, Consumer Reports has slipped to a new low.
I am not an Orbit owner, but have been following Consumer Report's recent recommendation regarding the Orbit infant carseat/base with fascination. I've never installed a carseat before, but when looking at the Orbit manual, it's pretty obvious that Orbit wants its users to use the StrongArm mechanism to tighten the carseat base to the vehicle's seat. It is printed so clearly in the manual that you would need to almost go out of your way to think that manufacturer thinks it is OK for the user not to use it. Consumer Report seem to claim that according to their interpretation of the instruction, it is OK not to turn the StrongArm mechanism at all despite manufacturer's explicit instruction to do so. It seems really strange to me that Consumer Report went out of their way to look for a way to Orbit infant carseat/base was not safe. Apparently Consumer Report found a way... which was to not follow Orbit's instruction in installing the seat.
What's even more fascinating is this recent article stating the importance and significance of intermittent test failures. Was anyone arguing that intermittent test failures are not significant? I thought it's pretty clear that Orbit wasn't arguing against the significance of intermittent test failure. Rather, Orbit was pointing out that Consumer Report did not follow the installation instruction as provided by the manufacturer (Orbit should know since they wrote the manual) and hence test result negatively misrepresents the "safeness" or the product. As someone else mentioned in their blog, it's like complaining about a cookie recipe because all the cookies came out burned... and the person baked the cookies at 400 degrees for 30 minutes when the recipe said to bake it for 15 minutes at 300 degrees.
I find CR declarations totally unprofessional. I don't understand why they can not accept they where wrong. You can not use your power to undermine anyones credibility just because you CAN (because not following basic instructions in a test and making a Don't buy - Safety Risk declaration is so irresponsible).
I bought the orbit system after researching very well about safety features, and that is why we decided to buy it. The BASIC security info is the STRONG ARM!! .
Intermittent test failure? Please! in real life and common life conditions (in any area) if you don't follow BASIC instructions you FAIL! Example: if you don't respect a red light you are probably gonna crash not every time but most of the time. If you don't study for a test you'll FAIL.
If you are not responsible for your actions you will loose credibility CR.
"Our tests are not designed to explain *why* a product performed as it did"
If you can't explain why an observed behavior occurred, you risk having been the cause of it.
Wow - I am shocked and dismayed by the lack of objectivity and professionalism that CR has displayed with regards to the Orbit. The manual clearly states you must twist the know on the infant base until the tension is appropriate (the green wheel will show in the display). How in the word would testers believe that the Strong Arm was not necessary? Why would you just sit the base in the car? And, it's interesting to note that CR refuses to address this negligence on their part to follow manufacturer directions. Instead they discuss the need to pay attention to intermittent failure of products. But, what about your failure to install the seat properly? If CR found it so difficult to use (which it's not), then why not simply rank it low on "Ease of Use" instead of indicating it is a safety hazard, which it is not. I am rethinking my use of CR based on their review of this very safe product.
"Readers of such a report may wonder how intermittent failures arise..."
Actually no, you told us in the followup. You didn't follow directions. I see this failure as lying outside the acceptable margin of error.
Why not try to prove you're right CR? Why not repeat the tests with a third-party and an Orbit rep to keep you honest?
"...when it comes to safety, no test failure is acceptable..."
Good lord. This is really getting absurd. Who wrote this?
I am not a Orbit owner but I have to agree with the comments posted. If Orbit (as seems to be the case) makes it very clear that the StrongArm knob needs to be turned, then Consumer Reports did NOT install the car seat properly by failing to do so.
We all agree that intermittent test failures are unacceptable. HOWEVER, they are unacceptable only if misuse was easily forseen or if failures occur DESPITE doing everything properly. If intermittent failures are caused by improper use or installation that is clearly avoidable, then the fault lies with the user and not the product.
In this case:
1) Everyone who owns the Orbit feels that the instructions are quite clear on the need to turn the StrongArm knob. CR is the only one so far who claims otherwise. Therefore I don't see an issue of the car seat being routinely installed improperly. The instructions spell it out clearly.
2) If in fact the intermittent failures are solely caused by not turning the StrongArm knob properly, then CR has simply shown that failure to install the car seat properly can cause intermittent failures. But this is NOT a safety risk. At most it is an improper usage risk which goes with pretty much every product. It is no more a safety risk than if CR chose to install tires with low tire pressure in defiance of instructions and then tires intermittently failed and then claiming said tires were a "safety risk"!
Previous
