This is my last Standards Watch column. I have looked at sleeping pad insulation, sleeping bag temperature ratings, and active insulation breathability and insulation. I also interviewed independent gear tester Steve Seeber, plus equipment designers Mike Cecot-Scherer, Michael Glavin, and Dan Durston. You’ll find a complete list of columns at the end of this story.
Along the way, I found critical faults with many industry standards, problems with testing backpacking gear using those standards, and issues with product marketing around standards and testing. I had planned to cover much more, but I grew frustrated and disillusioned. A few standards may be great – but I haven’t found them yet.
Even the best-planned backpacking trips can go haywire. Rather than plowing ahead, it’s a good idea to review what happened, learn from the experience, and change plans.
I spent most of my career purchasing and using products that adhered to important standards. Buyers could more-or-less easily check for standards compliance, and stop doing business with companies that didn’t conform.
When I started writing these columns, I assumed the backpacking industry worked roughly the same way. Over and over, I was surprised that the standards, testing, and marketing were flawed in important ways, and it was almost impossible for consumers to compensate. I tried to reflect those messes in the columns, and I discovered other issues that never made it into stories.
Here are the problems that surprised me the most.
Sleeping Bag Temperature Ratings
The advertised temperature for most sleeping bags and quilts is the Lower Limit based on ISO 23537, often called the EN rating. But that standard is based on a healthy male in his mid-20s, laying on his back, not moving, wearing warm clothes from head to toe, cocooned in a fully-closed mummy bag, with his face buried in the hood, on an R-4.8 sleeping pad, inside a windless shelter, at 59 °F (15 C) and 50 percent relative humidity. If you, your gear, and your campsite don’t fit that description, you might not sleep warm enough at the advertised temperature.
And multiple tests on the same bag can turn out significantly different at a widely-used U.S. test center. As a result, several bag or quilt makers downplay or ignore the standard.
For example, one well-known sleeping bag maker uses decades of experience to rate their bags. Their published temperatures are about 5 F (2.8 C) higher than their well-hidden Lower Limit ratings. A quilt maker gave up after spending thousands of dollars on testing. They measure loft and convert to degrees instead.
Sleeping Pad R-values
The standard sleeping bag temperature test uses an R-4.8 sleeping pad. Yet many gear makers, retailers, and reviewers tell us that R-4.8 is a winter pad, and R-3 or R-4 is fine for three-season (spring-summer-fall) use. Even though the climate in coastal Hawaii is very different from central Alaska!
And good luck figuring out how much colder you might sleep on an R-3 sleeping pad. Some standards units are very hard to use in real life.
Laws in several U.S. states require all tents to meet the CPAI-84 flammability standard. Yet the cotton canvas tent industry developed those tests, reportedly designed in part to put nylon and polyester at a disadvantage. As a result, most American shelter makers used fabrics coated with toxic flame retardants. Tents made in Australia, Britain, the EU, and Japan don’t use those chemicals.
The good news is this year’s CPAI-84 updates allow U.S. retailers to sell backpacking tents made without flame retardants. We should see more American-made shelters with less toxic fabrics, and maybe our stores will carry interesting tents from the rest of the world.
The ASTM F 1934 Minimum Tent Weight standard did not include stakes for good reasons. But if you don’t know why, it seems like a rip-off. Plus a large retailer rebranded it Trail Weight – yet most backpackers carry more, including stakes and a stuff sack.
Yes, there is a standard for measuring backpack volumes, ASTM F2153, last revised back in 2007. Yet one company’s 50-liter (3,000 cubic inch) pack might hold much more than another’s. In part because the standard doesn’t include open-topped pockets, like the rear mesh pocket or so-called cup holders on each side of many lightweight backpacks.
But the standard also requires very expensive 20 mm (0.79 inch) spheres for testing. 60 liters (3,600 cubic inches) of those small balls cost more than $1,000! So some small backpack makers measure volume using non-standard methods including rulers, ping pong balls, and soft foam.
Moisture Vapor Transfer Rate should tell you how fast water vapor moves through fabric, typically in grams per square meter per 24 hours, often shortened to gsm or g/m2. Testers can measure fabric MVTR using six different standards, to help you figure out how much sweat you might evaporate through clothing while hiking.
All the standards have problems, and you can’t compare the results across different standards and testers. Many fabric makers choose the standard that gives them the highest number, but won’t say which one they used. And no clothing maker publishes the total MVTR of a garment constructed with multiple layers of fabric and insulation, even though each layer and the dead air between lower the number.
Hydraulic head measures waterproofness, usually expressed as the millimeter height of a vertical tube of water pressing on the fabric. There are at least five standards for testing and computing HH. Gear makers rarely state which test they used.
But the standards are flawed anyway. Plus none of them tell you how the fabric will perform after a few years of use – or even a few weeks.
Air permeability tests, often expressed in CFM (cubic feet per minute) or LPM (liters per minute), are a big mess too, with four different well-recognized standards. Technically what gets measured is CFM per square foot, or LPM per square meter, of air moving through one layer of fabric. Lower values block the wind better, and higher values help you cool off without opening or removing the garment.
But the standards aren’t rigorous enough to reliably compare test results between standards, or even different testers using the same standard.
Headlamp and Flashlight Performance
The ANSI FL 1 standard for measuring headlamp and flashlight performance is more than a dozen years old. And in 2019, a portable lighting industry trade group took over maintenance of that document from another association. Yet too many manufacturers exaggerate those numbers, retailers pass the values along knowing they are wrong, and none of them seem to care.
Even a straightforward standard like 7075-T6 for the aluminum used in a few potty trowels is incomplete. Which might not seem that important. But using a trowel made with low-quality metal, while you are urgently digging a deep enough hole in rocky ground, could lead to an unpleasant mess.
Committees of people who work in the industry write most of the formal standards applied to backpacking gear. Seems like a good idea – they should have the expertise. But in addition to all the jokes about products designed by committee, the members often focus on making their company’s products look better and their competitor’s look worse. In other words, they produce standards written to help companies sell, not to help consumers buy. This results in compromises that often misinform and mislead the consumer.
Standards Usefulness and Enforcement
Standards and tests can be quite useful – if they produce results that are sufficiently accurate and reliable. As we’ve seen, that is rarely the case. Many backpacking industry standards are a half-step above useless.
For better or worse, U.S. laws do not require testing backpacking gear to most industry standards, or clearly communicating test results to consumers. But REI suppliers must furnish standards-based sleeping bag temperature ratings and sleeping pad R-values. Yet the company continues selling some gear without those numbers. And many makers don’t sell through REI, so the requirement doesn’t affect them.
Smaller gear makers, and independent gear testers who aren’t rich, cannot afford to test in-house using most standards. Not only are the documents expensive, the testing setups can cost over $100,000. Some get creative, like Steve Seeber. But they still need lots of expertise, plenty of time for construction and testing, and too much money.
Designers and Designs
Many backpacking gear designers either don’t know about or don’t pay that much attention to most formal standards and testing. In their experience, the materials they use and their designs are good enough, or they can use simple, practical tests to choose better. And some of them have learned the same hard lessons I have – important standards are almost useless.
Good design always makes a series of tradeoffs that might not work for you in a particular setting. Take an extreme example: the perfect Pacific Crest Trail peak-season thru-hiker shelter wouldn’t last long on a Denali winter expedition. But hauling a 10-pound gale-proof snow tent on the PCT in summer doesn’t make sense either.
And a few patches of high-quality repair tape can quickly fix many problems for much less weight or cost than making gear from heavier or more expensive fabrics.
Specification Wars and Diets
A few equipment makers and gear geeks focus on achieving the best number for a few measurements. But once you get beyond good enough for many values, you don’t gain much from pursuing more, more, more.
Yet the market gets into spec wars at the cost of other useful features. We’ve seen a few makers of backpacks, tents, and sleeping bags pursue lower weights while compromising longevity, usability, performance, cost – and even the truth.
And too many mass retailers show you little more than weight, price, and a few glamor shots before you click the buy button. Sometimes you can’t even get weight without asking. Even gear makers seem to be following spec diets for their product pages. Why publish all that confusing information when consumers are buying anyway?
Product Ratings and Reviews
Unfortunately, the star ratings and short reviews below the buy button are no substitute for standards and tests. Major online retailers are waging losing battles with companies that pay others to post positive and negative comments.
Even unbiased reviews are often from inexperienced buyers who barely used a product under ideal conditions and give it five stars. A few independent websites claim to help you judge the quality of retailer ratings and reviews, since sorting the wheat from the chaff is almost impossible. Hard to know who to trust.
Today’s substandard standards, temperamental testing, and misleading marketing are not the answer. Too bad a publication like Consumer Reports doesn’t cover backpacking gear, combining rigorous independent testing with well-designed surveys completed by thousands of members. Until then, maybe we can test products ourselves. Watch this space!
Don’t get me wrong – I learned a lot by studying standards and interviewing gear designers and testers. I hope you did, too, and that knowledge helped you become a better consumer of backpacking gear.
But when it comes to standards, testing, and marketing, we are largely on our own. Thank goodness for online communities like Backpacking Light plus a handful of careful and reliable reviewers.
Standards Watch columns and related stories:
- Standards Watch: Introduction, September 2020
- Improving R-values for Consumers, September 2020
- Sleeping Bag Temperature Ratings, September 2020,
- Sleeping Bag Temperature Ratings (video podcast), October 2020
- Steve Seeber Interview, November 2020
- Tent Designer Mike Cecot-Scherer Talks Standards, December 2020
- Mike Cecot-Scherer on Tent Design, January 2021
- Interview With Gear Creator Michael Glavin, February 2021
- Standards Are Important (But Not Enough), March 2021
- Dan Durston Chats About Standards and Testing, April 2021
- Dan Durston Talks Design and Business, May 2021
- Mike Cecot-Scherer Talks Potty Trowels and More, June 2021
DISCLOSURE (Updated November 7, 2019)
- Some (but not all) of the links on this page may be “affiliate” links. If you click on one of these links and visit one of our affiliate partners (usually a retailer site), and subsequently place an order with that retailer, we receive a small commission. These commissions help us provide authors with honoraria, fund our editorial projects, podcasts, instructional webinars, and more, and we appreciate it a lot! Thank you for supporting Backpacking Light!
Home › Forums › Standards Watch: Hikes into the Sunset