Black Packing Light
  • Sections
  • Today’s Features
  • Subscribe
  • Sign In
  • Subscribe
  • Home
  • Email Newsletter
  • Membership Info
  • Articles
    • Recent Features
    • Gear Reviews
    • State of the Market Reports
    • Gear Guides
    • Gear Lists
    • Skills & Techniques
    • MYOG
    • Science, Technology & Testing
    • Stories
    • Calendar
  • Education
    • Podcast
    • Webinars
    • Masterclasses
    • Online Courses
    • Guided Treks
    • Education Portal
  • Forums
    • Forum Index
    • Recent Forum Posts
    • Gear Forums
    • Gear Swap (Buy/Sell)
  • Gear Recommendations
  • Gear Deals

Backpacking Light

Pack less. Be more.

You are here: Home / Science & Technology / Testing & Research / StoveBench: A Stove Testing Protocol for Comparing the Performance of Backpacking Stoves

StoveBench: A Stove Testing Protocol for Comparing the Performance of Backpacking Stoves

by Ryan Jordan on January 3, 2019 New Features, Testing & Research

STOVEBENCH black 1

Introduction

StoveBench defines a protocol that is used to measure two important backpacking stove performance features (power and efficiency) in order to determine a single, quantifiable performance factor called the StoveBench Score.

Power is important because it characterizes a stove’s ability to perform well in adverse conditions (e.g., wind, cold temperatures, cold water). Efficiency is important because it defines a stove’s ability to conserve fuel and save weight for the hiker. Most consumers are enamored by reported boil times (a measure of power), while most ultralight backpacking gram-counters are enamored only by fuel usage amounts (a measure of efficiency). In actual use outdoors, both are important to consider.

This article introduces the StoveBench protocol and presents the definition of the StoveBench Score and how to determine it using the protocol.

As a case study, the protocol is applied specifically to the testing of upright canister stoves. Future applications of StoveBench will be applied to other types of stoves. In addition, special considerations, limitations, potential sources of error, and other applications of StoveBench are discussed.

Listen to our podcast about StoveBench to learn more (click here for the podcast show notes):

Context

Backpacking stove manufacturers often report the performance of their stoves in terms of:

  • Boil time (i.e., the time required to boil a given volume of water) – a measure of a stove’s power; and
  • Burn rate (e.g., the amount of fuel burned in a given amount of time) – another measure of a stove’s power.

In addition, ultralight backpackers, in particular, are interested in a metric that defines the mass of fuel required to boil a certain amount of water (a measure of the stove’s efficiency). Calculating this metric is popular amongst the more serious kitchen-counter stove testing enthusiasts.

However, the conditions during which these metrics are determined are not necessarily standardized from manufacturer to manufacturer or stove tester to stove tester and can’t necessarily be used to compare stove models from different brands.

Therefore, we have developed a protocol at Backpackinglight.com that we’ll be using in our stove reviews, called StoveBench (a portmanteau derived from the term benchmark, a type of test that defines a standard point of comparison).

In particular, we don’t see metrics that define “power” (e.g., boil time, burn rate) or “efficiency” (mass of fuel required to boil a certain volume of water) as mutually exclusive performance indicators. After all, any manufacturer can optimize stove performance for one vs. the other. However, balancing both power and efficiency with system weight is a design and engineering challenge of developing and using any cooking system (which may include multiple components, such as the burner, fuel container, windscreen, heat exchanger, pot, and lid). After all, a stove that is 100% efficient but takes 20 minutes to boil a pint of water, or doesn’t have the power to boil water in cold temperatures or breezy weather is as disadvantageous as a stove that boils a pint in three minutes but requires 50 grams of fuel.

Thus, the real purpose of StoveBench is to provide a meaningful way to compare different stoves so that efficiency, power, and fuel economy are implicitly defined in a single quantifiable performance factor.

The StoveBench Score

The goal of the StoveBench Protocol is to produce a numerical “performance factor” that considers both a stove’s power output (proportional to its boil time) and its fuel efficiency (proportional to the amount of fuel used during the test) during a standardized stove operation test, during which a stove will be used to heat some predetermined amount of water.

This StoveBench Score (“F”) can be described as follows:

F = output ÷ input

In any cooking scenario, output is defined primarily by water volume and temperature change (heating). More energy is required to heat larger volumes of water, and to change the temperature by more degrees.

Input is defined primarily by time (more energy is used the longer a stove boils) and fuel mass (more energy is used by a greater amount of fuel).

It follows that higher values of F are better, as they represent higher levels of output (water volume boiled) for a given amount of input (energy expenditure).

Therefore, F can be written in these terms:

F = [ V ⨉ ΔT ] ÷ [ t ⨉ M ]

where

  • V = volume of water used in the test
  • ΔT = difference in starting and ending water temperatures
  • t = time of test duration
  • M = mass of fuel used during the test

For example, a test performed under the following conditions at sea level might look something like this:

  • V = 0.5 L
  • ΔT = 100°C (i.e., start with 0°C water and end when the water boils at 100°C)
  • t = 4 minutes (boil time)
  • M = 15 g (mass of fuel used to boil the water)

The calculated StoveBench Score would thus be:

F = [ 0.5 L ⨉ 100°C ] ÷ [ 4 min ⨉ 15 g ] = 0.83 L•°C/min•g

Which can be read in layman’s terms like this: in this test, this stove was capable of heating up 0.83 liters of water by 1 degree Celsius (or, 1 liter of water by 0.83 degrees Celsius) in one minute using one gram of fuel.

The StoveBench Score will vary across different test conditions, being influenced by factors such as:

  • Pot geometry
  • Ambient air temperature
  • Fuel type
  • Presence of air current (e.g., wind).

However, because the StoveBench Score is normalized for water volume and temperature differential, it has the potential to be less sensitive to factors such as the elevation of the test location or starting water temperature, which means that across a narrow range of test environments and materials used, results should compare relatively well. Unfortunately, manufacturers seldom publish the details of their test protocols, emphasizing the importance of testing stoves using a standard methodology when comparing stoves from different brands, or in different environments.

For example, let’s consider two tests using a well-known canister stove from a major US brand.

Test #1, performed by the manufacturer, is conducted with water having a starting temperature of 70 °F (21.1°C), an ending temperature of 212 °F (100°C), and water volume of 1.0 L. In this test, the manufacturer reports a boil time of 3.5 minutes and 14 g of fuel consumption. The StoveBench Score is thus calculated as follows:

F =  [ 1.0 L ⨉ 79°C ] ÷ [ 3.5 min ⨉ 14 g ] = 1.61 L•°C/min•g

Test #2, performed by the author, is conducted with water having a starting temperature of 32.9 °F (0.5°C) and an ending temperature of 199 °F (92.7°C), noting that these tests are conducted at an elevation of 7,205 feet, where water boils at a lower temperature than at sea level. A 0.85 L titanium pot was filled with 0.5 L of water for the test, and the stove operated at full power. The boil time was 3 min 5 sec and 11.3 g of fuel was consumed. (Full details of the test protocol are described below in the “Materials and Methods” section of this article.) The StoveBench Score for this test is thus calculated as follows:

F =  [ 0.5 L ⨉ 92.2°C ] ÷ [ 3.08 min ⨉ 11.3 g ] = 1.30 L-°C/min-g

The differences between the two results are not dramatic (F is only 20% lower in Test #2), given significant differences in starting water temperature, water volume boiled, and elevation of the test location. However, the differences highlight the need to ensure standardized testing when comparing the performance of different stoves.

This is important because the vast majority of “gear guides”, “best stoves” comparisons, and stove reviews simply re-publish manufacturer-reported performance data, and make judgments accordingly, without regard for the factors that influence stove test results.

Considering Efficiency

A stove’s efficiency can be loosely defined as its ability to minimize waste heat. Heat is wasted via a number of processes, including incomplete combustion of fuel, a flame pattern that causes heat to spill up the sides of a pot vs. being targeted to the bottom of the pot, and heat losses from the pot itself (e.g., always use a lid!).

Efficiency is a function that balances fuel economy with flame power. A high-power stove aids efficiency by delivering more heat in a shorter period of time which can combat system heat losses. However, a high-powered flame can also waste more heat because the heat cannot be absorbed into the water fast enough.

Consideration of efficiency is built into the StoveBench Score, which is calculated from both boil time and fuel usage. Short boil times (high power) and low fuel requirements (high fuel economy) both contribute to higher StoveBench Scores.

Specifically, the fractional contribution of stove performance by fuel economy is defined by the Feconomy = [ V ⨉ ΔT ] ÷ M part of the StoveBench Score equation, and the fractional contribution of stove performance by flame power is defined by the Fpower = [ V ⨉ ΔT ] ÷ t part of the StoveBench Score equation.

Since efficiency can be defined as the ratio between the actual amount of fuel used in the test and the theoretical amount of fuel that should have been used in the test if the stove was operating at 100% efficiency (i.e., no heat loss), the StoveBench Score is particularly useful for comparing stoves of different types (e.g., alcohol vs. compressed gas vs. liquid gas), where inherent inefficiencies in both the heat capacity of fuels and heat losses in stove systems will be reflected by both Feconomy and Fpower.

Often, there is a direct correlation between heavier stoves and stoves that are more efficient. Thus, the StoveBench Score is a versatile metric that can provide a backpacker with valuable information about which stove type might be best for a given set of water heating requirements based on trip duration and required water volumes. The StoveBench Score can provide the foundation for a variety of subsequent analyses to aid the hiker in evaluating different stove systems. For example, the StoveBench Score divided by the total cook kit weight (which might include the stove, fuel container, pot, windscreen, etc.) would provide valuable insight into the performance:weight ratio of a particular stove and cooking system.

The StoveBench Test Protocol: Overview

What follows are the materials and methods used in a standardized protocol for determining the StoveBench Scores for any stove type.

Environment and Instrumentation

In general, the standard (“control”) boil test of the StoveBench protocol measures stove performance under the following conditions:

  • Indoor room temperature (ambient)
  • No ambient air flow (wind)
  • Measure the time and fuel required to boil water originally near its freezing temperature
  • No stove windscreens or other accessories unless they are integrated into the burner design.

The following table describes my own test environment and instrumentation in more detail.

Ambient EnvironmentIndoors; Air Temperature = 19.5 °C +/- 0.5 °C, thermostat-controlled fanless convection heater; RH = 20-40% (monitored); elevation = 7205 feet above sea level.
Water500 g of water (0.5 L) +/- 2 g; measured starting temperature = 0.5 to 5.0 °C (water temperature stabilized by ice); stopping ("boiling") temperature = 91.0 °C (measured temperature at the beginning of a rolling boil at this elevation)
Water Temperature MeasurementHTI HT-9815 Digital Thermometer w/K-type Thermocouple Sensors
(±1 °C accuracy, 0.1 °C resolution)
Weight MeasurementAdam Equipment CBK 8a scale, 4kg capacity, 0.1g accuracy (calibrated)
Stove System Thermal ImagingHTI HT-18 Thermal Imaging Camera, -20 °C to +300 °C range, image resolution 220x160 pixels.
Infrared Temperature MeasurementEtekcity Lasergrip 1080 Infrared Thermometer, 0.1 °C resolution.
Water Heating Containertitanium pot with lid (0.85 L capacity, 5.0 in wide x 3.75 in height, 4.0 oz dry weight)

General Test Procedure:

  1. Prepare ice water using a 50/50 mixture of ice cubes and tap water in a gallon container that can be easily stirred so as to ensure uniform temperature distribution of the water. Store the water in a cold environment when not in use.
  2. Record the ambient air temperature of the test environment to within 0.1°C regularly throughout the test.
  3. Using infrared temperature measurement, ensure that the temperature of the fuel is the same as the ambient temperature of the test environment by comparing the temperature of the test fuel to the temperature of fuel stored nearby that isn’t being used in the test. This is important because different material types can emit different levels of infrared radiation, and the surface temperature of the material itself may differ from the ambient room temperature. Ensure that the surface temperature of each batch of fuel is within 0.1°C of each other.
  4. Measure the starting weight of the fuel (this generally requires measurement of the stove plus fuel) to within 0.1 g.
  5. Place the water heating container on the bench scale, tare the scale, and add the specified amount of ice water (filtered so as to remove the ice) to within +/- 0.5% of the target weight. The target weight should be converted to water volume, and water volume used as V in the StoveBench Score (F) formula above.
  6. Place the water heating container on top of the stove burner (centering it on the stove’s pot supports), insert the thermocouple into the water so that it measures the temperature of the water halfway down the water column and does not touch the sidewall of the container. Place the lid on the container. The thermocouple wire should be suspended vertically over the pot, so waste heat from the stove system can’t materially cause damage or temperature measurement interference with the thermocouple wire.
  7. Measure the starting temperature of the ice water to within 0.1°C resolution. Ensure that the starting temperature is less than 5.0°C.
  8. Within the span of 1 second, light the stove burner/fuel, open the stove valve to a fully open position (not applicable for stove types with no fuel flow regulation), and start a stopwatch.
  9. When the water temperature reaches a predetermined temperature defined as the boiling point (to within 0.1°C resolution), record the elapsed time to the nearest second and extinguish the stove (i.e., turn off the fuel flow) immediately. This elapsed time shall be recorded as the “boil time”. Boil time should be converted to decimal minutes, which will be used for t in the StoveBench Score (F) formula above.
  10. The difference between the starting and ending water temperatures will be used for ΔT in the StoveBench Score (F) formula.
  11. Measure the ending weight of the remaining fuel to the nearest 0.1 g (or as applicable, the stove plus fuel assembly). The difference between this weight and the starting weight shall be recorded as the “fuel consumed”, and used for M in the StoveBench Score (F) formula above.
  12. Calculate the StoveBench Score (F) for the test.
  13. Repeat the test as needed to ensure statistical confidence in the result (1-2 additional times in a highly-controlled test environment with accurate instrumentation; up to 6 times in less controlled environments).

StoveBench Protocol Application: Upright Canister Stoves

What follows are the specific materials used in the application of the StoveBench Protocol for upright canister stoves:

Fuel 80/20 mixture of isobutane/propane; 227 g net weight canisters between 30% and 80% of their fuel capacity*

* Canisters are retired when fuel capacity reaches 30% or less when changes in internal canister pressures can skew results materially. At high canister capacities, high canister pressure causes excess heat losses that can skew results materially when stoves are operated at full throttle. See below for more details.

Specific Procedural Notes When Using Upright Canister Stoves:

  1. Fuel Weight Measurements. During the fixation to and removal from a canister, a stove may cause small amounts of compressed gas to leak out of the canister, which could skew results. Thus, fuel weights are determined by calculating the difference in weights of the entire stove-canister assembly at the start and end of the test.
  2. Fuel Temperature Measurements. The surface temperature of the canister is measured with an infrared thermometer to ensure that it’s the same temperature of a nearby control canister that isn’t being used in the test (and is thus, at ambient temperature).
  3. Starting the Test.  A lighter is held to the stove burner while the stove gas valve is turned on very slowly until the burner is lit. Within one second, the stove valve is turned to its maximum at the same time a stopwatch timer is started.
  4. Ending the Test. When the thermometer readout reaches the stopping temperature, the stopwatch timer is recorded and the stove gas valve is immediately turned off.

Proof of Concept: Preliminary Stove Test Results

Comparing Upright Canister Stoves

As part of a series of new Backpacking Stove Gear Guides, we are planning on releasing our upright canister stove gear guide later this month.

What follows is a small selection of control boil test results (using the protocol above) from one test batch that included 18 models of upright canister stoves (the final gear guide will include approximately 25 models). For the purpose of this article, brand and model names have been hidden. They will be revealed with the rest of the results in the upcoming gear guide.

Results have been sorted by order of highest to lowest StoveBench Scores.

Model ID #Boil Time (mm:ss)Fuel Usage (g)StoveBench Score - F (L•°C/min•g)
13:308.31.56
22:4513.21.25
Average3:3012.31.16
34:0510.91.02
43:1019.70.73

The StoveBench Score rewards not only stoves that have good fuel economy (low fuel usage), or stoves that have high power output (low boil times), but stoves that are efficient – that can deliver a high amount of power for the least amount of fuel.

Sources of Error

  1. Scale accuracy. Calibration standards were used to monitor and verify scale accuracy using the USBR 1012 protocol. The accuracy of measured standards was determined to be less than 0.04 g, resulting in an error % of reported fuel consumption amounts of less than 0.3%. The scale manufacturer reports linearity of +/- 0.2 g, which could contribute an error of up to about 2% in reported fuel consumption amounts.
  2. Fuel usage at start and end of test. Up to one full second of time was required to start and stop the stove during which the valve was turned (two seconds total). Thus, up to two additional seconds of unnecessary fuel consumption may have occurred. The average amount of fuel used in the tests was 12.3 g over the course of a burn time of about 210 seconds (i.e., 0.059 g/sec). Therefore, over the course of 2 seconds, up to 0.12 g of additional fuel may have burned, amounting to an overstatement of approximately 1% of reported fuel usage. This error was consistent over all tests.
  3. Temperature measurement accuracy. Thermocouples were calibrated in both ice water and boiling water. Measurement accuracy proved to be within 0.1°C. Since the test was stopped when the water temperature reached 92.7°C, and if heating is assumed to follow a fairly linear progression from 0°C to 92.7°C, then over the course of the average boil test (210 seconds), the approximate rate of heating was 2.3 seconds/°C. A temperature inaccuracy of 0.1°C would thus introduce a boil time error of only 0.23 seconds, which corresponds to a fuel consumption error (see #2 above) of only 0.014 g (about 0.1% error).
  4. Starting and Ending Canister Temperatures. Starting canister temperatures were measured with a laser infrared thermometer (Etekcity Lasergrip 1080) and were always within 0.1°C of another canister (not in use) that was used as a room temperature control. Ensuring consistent starting canister temperature in all tests was critical to minimizing error. Canister temperatures often dropped 4 to 6°C during the course of a test due to the heat of vaporization principle, the process by which heat is converted to energy required for liquid gas to become vapor. Starting the next test immediately after resulted in boil times that were as much as 20% slower than the previous test, since lower-temperature canisters had lower vapor pressures (i.e., lower fuel flows, lower burn rates, and thus, longer boil times). While some method of controlling canister temperatures during the test (e.g., placing it in a water bath with temperature controlled by a thermostat) might be valuable in order to prevent the canister temperature from decreasing, doing so would add significant complexity to the protocol and further decrease the protocol’s relevance to how stove systems are used in the field by most users.

Error percentages will be higher when using consumer-grade instrumentation (e.g., digital kitchen scales and meat thermometers). For example, a typical made-in-China digital kitchen scale has an accuracy of 0.1 oz (3 g) with linearity of up to 0.2 oz (6 g). With this much potential error introduced in the measurement of fuel consumption during a single boil test, replicate testing becomes even more important. Verification of scale accuracy with calibration standards is important, even for at-home stove testing enthusiasts.

Other Considerations

What about using tap. vs. distilled vs. water from a lake or stream?

This concern is propagated by Betty Crocker et al., based on the common kitchen practice of putting a teaspoon of salt into a pot of water to make it boil faster. In other words, do differences in the concentration of solutes have an effect on boiling time?

For those of you that think physicochemistry is fun (you have a copy of the CRC-HCP by your bedside), consider the basic principles that:

  • Solutes will lower the boiling point (water will boil faster) because the specific heat capacity of those solids (e.g., salts) is lower than the specific heat capacity of water.
  • However, solutes will also increase the boiling point of water (water will boil slower) because they raise the vapor pressure of the solution, requiring more heat energy to boil.

So, at what sort of solute concentrations does this make a difference?

A 1% solution of salt water (e.g., 10 g of salt dissolved in 1 kg of water) will increase the boil time by about 1% (physics fans: use the heat equation Q=MCΔT) to determine this). Adding more salt starts to tip the scales, however, and by the time we reach a 5% solution of salt, the boil time is decreased by about 1.5%. Of course, these calculations are based on a perfectly efficient system (where 100% of the heat produced by the fuel is transferred to the water). But even with highly inefficient (10% to 40% efficiency) stove systems characteristic of solid fuel, alcohol, and gas stoves without pressure regulation, the effects of solutes on boil times are likely negligible relative to the other sources of variability inherent in performing testing like this.

In other words, we are talking about minute effects on the boil time and concentrations of salt that would make your water unpalatable.

For my own testing, I use tap water that has a total dissolved solids (a measure of solutes) concentration of less than 100 mg/L (0.01%).

Do changes in canister pressure skew results?

For tests involving canister stoves, the amount of fuel remaining in the fuel canister could impact results.

As a fuel canister is used, the volume of fuel inside a canister decreases (which also decreases the pressure and resulting flow rate of fuel that is delivered to the burner).

In addition, fuel mixture composition could change, but that effect is probably not as dramatic as it’s assumed to be.

To evaluate this effect, the control boil test described above was repeated using a stove that does not include a pressure regulator (since the effects will be less dramatic on a stove that has a built-in pressure regulator). A 227 g (net weight) canister containing a mixture of 80/20 isobutane/propane was used for this test.

Boil times, fuel consumption, and StoveBench Scores are presented in the following chart as a canister is used from full capacity to nearly-empty capacity.

imageLikeEmbed

The following table provides a basic statistical analysis of all tests performed for this canister:

 Boil Time (mm.ss)Fuel Consumed (g)StoveBench Score - F ( L•°C / min•g )
Average3.410.081.32
Standard Deviation0.490.830.12
StDev% (100 x Standard Deviation / Average)14.5%8.2%8.9%

Based on examining the graph above, I attribute the high variability to the high pressures in the canister when full (or nearly full), and the low pressures in the canister when nearly empty. Here are my conclusions when examining the graph, noting, in particular, those values that are near to or exceed +/- one standard deviation from the average values. StDev% is a useful statistic for estimating the range of error of experiments that should be repeatable. For the purposes of this protocol, I’m hoping for a range of StDev% of +/- 5%. From the table above, all three ranges are outside this criterion.

  1. Boil times are abnormally high at low canister fuel capacities (<30%).
  2. Fuel consumption is abnormally high with a full canister (>80% capacity) and abnormally low with a nearly empty canister (<5%).
  3. StoveBench Scores (F) are abnormally low at low canister fuel capacities (<20%), and (possibly) abnormally high at high canister fuel capacities (>80%).

Based on these observations, the most reliable repeatability for the two measured performance metrics (boil time and fuel consumption), and the calculated StoveBench Score, occurred when canister fuel was in the range of 30% to 80% of its capacity.

With this restriction in place, if all tests performed at capacities exceeding 80% and lower than 30% are discarded, we can examine the resulting statistics:

 Boil Time (mm.ss)Fuel Consumed (g)StoveBench Score - F ( L•°C / min•g )
Average3.1610.11.39
Standard Deviation0.130.410.07
StDev% (100 x Standard Deviation / Average)4.1%4.1%5.0%

This constraint results in StDev% values for boil times, fuel consumption, and StoveBench Scores that are 5% or less.

Consequently, all tests measuring StoveBench Scores will be performed when the canister is within the 30% to 80% range of fuel capacity.

These results indicate that (at least for upright canister stove tests adhering to the control protocol defined above), a StoveBench Score can be interpreted to have an experimental standard error (defined by +/- 1 standard deviation) in the range of +/- 5.0%.

What are the problems associated with running stoves at full throttle?

Running a stove at full throttle probably wastes unnecessary heat when using stoves that don’t have built-in pressure regulation. Most users will “feel” that their stove needs to be turned down a little in order to run it more efficiently, but thermal imaging can reveal the effect more objectively.

heatmapsstoves 1

On the left, the thermal image shows a stove system with the burner off. In the center image, the same system is operating with the burner turned down about 25% towards its off position, using a nearly-full canister. On the right, the system is in use with the stove throttle turned to its maximum using a nearly-full canister. Note the high amount of wasted heat on the right (full-throttle) image, as indicated by the thermal pattern surrounding the pot. Interestingly, thermal imaging of stoves operated at full throttle on canisters containing less than 80% of their fuel capacity revealed images that looked more like the one in the center rather than the one on the right.

Unfortunately, without (extremely difficult to make) direct measurements that correlate the fuel valve orifice opening, fuel valve handle positioning, and a stove’s heat output, running a stove at any output level less than 100% (fuel valve handle turned all the way counterclockwise so the fuel delivery orifice is at its maximum) will introduce too much subjectivity that may interfere with test repeatability and/or performance comparisons between stove models.

That said, having a control test at full throttle provides a good foundation for further testing where stove output is controlled by the fuel valve handle, and results of this type of comparison testing will be presented in our forthcoming Upright Canister Stove Gear Guide.

Interestingly, the effect of wasted heat was much more significant for canisters having a fuel capacity that was greater than 80% (see “Do changes in canister pressure skew results?” above), suggesting that operating a stove at full throttle has more detrimental impacts when using fuller canisters. I used thermal imaging to verify this.

What about mismatched burners and pots?

Large pots don’t necessarily match so well with small burners, or low power fuels (e.g., solid fuel tablets). On the other end of the spectrum, small pots get overwhelmed by large burners, resulting in significant heat loss and fuel waste.

I recognize that burner-pot mismatches will result in a stove system that perhaps scores lower than it should in the control test protocol defined here.

This control test protocol is optimized for solo cooking in three-season conditions (small pot volume, low water volume). Users should consider StoveBench results for large pots and large water volumes as part of any burner’s overall performance, and not rely solely on the control test described above before choosing a stove for boiling larger volumes of water.

“No wind, full throttle, not my pot, this test isn’t representative for me.”

This is a controlled testing environment where we’ve arbitrarily defined a set of test conditions and a test protocol that gives us repeatable results that can be used to identify one performance metric benchmark that we can:

  1. Use to compare different stoves; and
  2. Use as a point of reference for different test environments and conditions (see next section).

Is the StoveBench test useful for other types of stoves, and how do F factors compare?

Yes, of course. The rationale for determining a StoveBench Score is applicable for any type of stove and fuel, including solid fuel (e.g., hexamine tablets), alcohol, upright/inverted/integrated canister stoves, liquid fuel (e.g., white gas, kerosene) stoves, or wood stoves.

We’ll be developing protocols for each of these stove types as part of the StoveBench program.

StoveBench Scores for stoves using different fuel types will differ primarily as a result of the specific energy capacity of a fuel (i.e., energy per weight), and the efficiency of the benchmarked stove system (e.g., stove/pot/burner combination).

When we publish stove reviews and gear guides, we’ll also include a metric equal to the StoveBench Score divided by the stove system weight. This will (perhaps) represent at least some of the benefit of carrying so-called ultralight stove systems (e.g., solid fuel and alcohol), even though these stoves use fuels that don’t contain as much specific energy (energy per gram of fuel) in them as gas stoves.

Variations of the Control Protocol

In addition to Control Boil Tests (described above), we are devising a number of other performance scenarios that will be featured in upcoming reviews and gear guides, including:

  • Wind Test: add a low-speed fan to the test environment to create a wind speed characteristic of a light breeze. We performed a test similar to this in a recent comparison of integrated canister stoves and the differences in stove performance results was dramatic.
  • Large Water Volume Test: 1500 g of water in a titanium 2-liter pot, a common scenario for groups of 2-3 hikers sharing a cook kit.
  • Cold Temperature Test: a test performed at an ambient temperature of 32 °F (0°C) or less, a common scenario for winter backpacking environments and/or snow melting.
  • Stress Test: 1500 g of water, ambient temperature of 32 °F (0°C) or less, and wind induced by a fan.

Our first round of StoveBench Scores will be released later this month when we publish the results of our new Upright Canister Stove Gear Guide, which will include StoveBench Scores for all of the above scenarios from a subset of the approximately 25 canister stove models being reviewed in the Gear Guide.

How to Perform Your Own StoveBench Tests and Contribute to the StoveBench Database

You don’t need fancy instrumentation to run your own StoveBench tests. In fact, your own tests can be extremely valuable for our community!

If you can reasonably measure the temperature of your water and measure fuel consumption weights, then you are well on your way to running your own StoveBench tests.

Here’s a short video that illustrates how I run a StoveBench Control Boil Test for an upright canister stove:

I use the following spreadsheet template for each test I run:

  • Download StoveBench Test templates

Here’s a video that shows how I use the spreadsheet test template:

We are building a user database of StoveBench Scores calculated from your own test environments and protocol parameters.

  • Submit your own test results here

We’ll be publishing this information in a subsequent article to be published in early February 2019. In addition, if this information proves to be valuable to our community, we’ll be releasing a live version of the database (with real-time updates as new user tests are submitted) as well.

StoveBench Testing Services

  • If you are a stove manufacturer and would like to have your stove tested using the StoveBench protocol, here is some more info.
  • If you’re interested in starting your own StoveBench Lab, click here.

Final Commentary and Disclaimer

When interpreting StoveBench Scores, do so with an understanding of this protocol’s limitations and error sources (outlined above). When interpreting StoveBench Scores calculated from user-submitted test data to our database, consider that different test environments, different types of measurement instrumentation, and differences in protocols may result in StoveBench Score variability that may not be present when we perform our own StoveBench tests in our own laboratory environment. The StoveBench database will distinguish between user-submitted stove testing data and stove testing data we perform on behalf of Backpacking Light that adheres more strictly to the protocol described herein.

Finally, the StoveBench Score is not the holy grail of stove performance. While it can factor into your decision-making process to purchase a stove, or select a type of stove for a particular trip, there are other issues of importance as well: the dry weight of the system, the starting weight of the system including fuel, the environment you are cooking in, the cost of the fuel, the allure and satisfaction that comes with making a homemade stove, brand loyalty, and so much more. So, let StoveBench be one of your guides in making decisions, but not your only one.

Acknowledgments

The author wishes to acknowledge Roger Caffin, Hikin’ Jim Barbour, Gary Dunckel, and Jerry Adams for their helpful and critical reviews of the StoveBench protocol.

alcohol stove, canister stoves, cooking systems, solid fuel stoves, stovebench, wood stoves

Get ultralight backpacking skills, gear info, philosophy, news, and more.


Comments

Home › Forums › StoveBench: A Stove Testing Protocol for Comparing the Performance of Backpacking Stoves

Viewing 25 posts - 1 through 25 (of 73 total)
1 2 3 →
Forums are supported by our merchant partners (disclosure)
REI (Coupons) • ZPacks • Hyperlite • Patagonia • Arc'teryx • RBTR • Drop • Backcountry • Feathered Friends • CampSaver • Gaia • Mountain Hardwear
Gear Deals • Gear Search
Login to post (Basic Membership required)
  • Author
    Posts
  • Jan 3, 2019 at 6:19 pm #3571231
    Ryan Jordan
    Admin

    @ryan

    Locale: Central Rockies

    Companion forum thread to: StoveBench: A Stove Testing Protocol for Comparing the Performance of Backpacking Stoves

    StoveBench defines a protocol that is used to measure two important backpacking stove performance features (power and efficiency) in order to determine a single, quantifiable performance factor called the StoveBench Score.

    Jan 3, 2019 at 7:51 pm #3571268
    Jerry Adams
    BPL Member

    @retiredjerry

    Locale: Oregon and Washington

    I like the thermal images of the stove run at different speeds

    Jan 3, 2019 at 8:18 pm #3571281
    Eric Blumensaadt
    BPL Member

    @danepacker

    Locale: Mojave Desert

    I see these tests as being highly objective as laid out by Ryan.

    However (you knew there was a “however”) the use of a titanium pot v.s. an aluminum pot introduces the factor of the much higher heat resistance of titanium and the much lower even heating on the pot bottom. (More of a “hot spot” with Ti pots.)

    This ti hot spot problem is very evident when comparing a ti skillet to an aluminum skillet when frying, for example.

    Then there is the matter of the pot width-to-height ratio. There is a “golden mean” for pot efficiency which is somewhere in the neighborhood of bottom width : side height of 2:1. What ratio of pot height-to-width will be used?

    Run a thermal imaging device from an overhead view and this problem of the ti hot spot becomes evident.

     

     

    Jan 3, 2019 at 8:38 pm #3571286
    Ryan Jordan
    Admin

    @ryan

    Locale: Central Rockies

    Eric, all good points.

    The choice of titanium was made for the standard protocol because Ti is considered the gold standard of lightweight cooking pot materials, esp. in our community. The choice of the Ti pot used in this control test was motivated by its geometry – it’s W:H ratio is about 1.3. With 500 ml of water in it (the amount of water used in the control test) the water column W:H ratio is about 2.3. I didn’t want to use the common “tall and narrow” solo cookware because it makes the system even less efficient. As a comparison, a Toaks 850 with 500 ml of water in it has a water column W:H ratio of about 1.3. It’s gonna be fun to compare StoveBench Scores between these two pots :)

    What can be done from here, now that we have control test data, is run tests to compare different pot geometries, different materials, etc., and compare those StoveBench Scores to the control score, and get a much better handle on how these types of changes improve or reduce the performance of the cook system.

    Thermal measurements on the bottom of the pot in these tests (where the stoves were turned up to full throttle) revealed no major differences in the size of the hot spot on pots of this size (5 in dia), with one exception – the little tiny-burner stoves like the BRS3000t, when they are running at a low throttle.

    So we’ll probably see less difference for Ti-vs-Al with smaller pots, higher throttles, and larger burners. It will be cool to actually run those tests and look at the differences quantitatively, though.

    Jan 3, 2019 at 8:58 pm #3571292
    Barry B
    BPL Member

    @ve7vie

    What about alcohol stoves?  Surely ultalighters like these! I have used Trangia and now Vargo for a few years. Recently I tested ethyl versus methyl alcohol and found that ethyl boiled faster and also burned longer, which surprised me.

    Jan 3, 2019 at 9:16 pm #3571302
    Ryan Jordan
    Admin

    @ryan

    Locale: Central Rockies

    Hey Barry – I have protocols for solid fuel, alcohol, wood, integrated canister stoves, inverted canister stoves, and pump-pressurized liquid fuel stoves, and have performed tests with all of these types of stoves. The protocol is totally adaptable to any stove type (see the “General Test Procedure” section above).

    I need to do some more rigorous statistical testing and tweaking to the alcohol and wood stove protocols before publishing them, though. The other protocols are pretty dialed and we’ll be releasing those as each of the respective stove gear guides is getting ready to be published.

    Jan 3, 2019 at 11:01 pm #3571336
    Ben H.
    BPL Member

    @bzhayes

    Locale: No. Alabama

    so this is efficiency divided by time to boil…. and then multiplied by the lower heating value of the fuel and divided by the specific heat and density of water to get a really confusing dimensional parameter.

    You could just take time to boil and divide it by efficiency and call it efficiency corrected time to boil which would only have the units of time.  The problem with all of these is you have to exactly boil the water to get a good reading which means elevation will skew your results.  Do you want a better stovebench number? just hike up the mountain!  OK… on a backpacking forum maybe that does make sense.

    I think it makes more sense to compare efficiency corrected power (= stove power/efficiency) since you are trying to optimize power and efficiency.  If you use power then you can calculate a reliable number in real time throughout the test and your test protocols are not dependent on bringing the water to a boil.

    In the end it will be interesting to see how your proposed linear combination of power and efficiency (as opposed to a power of either efficiency or power) identifies as “optimal” stove systems.

    Jan 3, 2019 at 11:30 pm #3571347
    Scott Chandler
    BPL Member

    @blueklister

    Locale: Northern California

    Did you find variation in weights of full canisters, or were they pretty consistent? If you used different brand canisters, would you share “full” and empty weights of each?

    Jan 3, 2019 at 11:38 pm #3571349
    Ryan Jordan
    Admin

    @ryan

    Locale: Central Rockies

    @bzhayes: the formula actually doesn’t depend on boil time, elevation, etc. – just a heating differential between two temps over the course of some time interval. It’s normalized on a per-degree T basis.

    Ice water and boiling were chosen as the two reference points because they are pretty easy to validate with consumer grade thermometers, or for estimation purposes, even without thermometers at all – just prepare ice water, start with that. Wait for the water to boil, and end with that.

    I looked at calcs involving efficiency (based on the technical definition, see below) divided by time to boil but the physical description of that parameter no longer has a strong basis in evaluating the reality of stove performance, which (defined here) equals output divided by input. At some level, of course, it becomes a semantics debate.

    Also, fun fact about efficiency: from a technical standpoint, efficiency is defined by the theoretical amount of fuel used in a test (if all of the heat in the fuel was transferred to the water) divided by the actual amount of fuel used in a test. For the 25 canister stoves we are reviewing right now for the gear guide, we’ve measured efficiencies from 20% all the way up to about 50%. There’s a lot of variability here, and this is where stove design and system design really come into play. We’ll present this data in the upcoming gear guide as well.

    Jan 3, 2019 at 11:43 pm #3571350
    Ryan Jordan
    Admin

    @ryan

    Locale: Central Rockies

    @blueklister: We’re using MSR IsoPro 227g net weight canisters in all tests. Mainly because they’re the only ones I could afford to send to a gas chromatography lab for analyzing impurities (they have less than 4% n-butane which is a really good purity for consumer grade gas).

    I’m hoping to do that analysis at some point on all of the gas brands, but we gotta save up a little cash for that one, the tests aren’t cheap.

    The gross weight of these canisters (canisters + fuel) doesn’t vary by more than about +/- 2 g. Some of that may be in the retail pricing labels :D I’ll have a better handle on the empty weight variability as soon as I start emptying all these things and weigh them! But the average of six empty MSR 227 net canisters that I have here now is 150.1 g, with a StDev% of 0.6% – and that could be fuel residual. Very difficult to empty these all the way.

    Jan 4, 2019 at 1:44 am #3571372
    James Marco
    BPL Member

    @jamesdmarco

    Locale: Finger Lakes

    Barry, Alky stoves generally conform to the same limitations as all other stoves. By using Ryans F values within the tolerances he specifies, it should be easy to compute the various values. However, most of us kitchen testers it is more of a matter of testing what we have in the configuration we will be using. (I personally noted differences in the shape of pots, heat exchanger systems, color of pots, metallic make up as Eric alluded to above, with/without a heat shield, a large difference between high and low cannister settings, etc. Mostly these were small 4-5% values in each case except cannister settings, but present. Anyway, getting back to the testing values… You will find that Ryan’s calculations will not vary much among the same type of stoves, but, you cannot use them to compare stoves using different fuels. For example compare his best F values with his worst F values. Assuming the times remain consistent, we see that just the amount of fuel used changes the equation by about half: F-1.56 vs F-0.73 Likely, it will be more of a difference since around 10gm/L is a rough guess for Canister fuel and around 40g/L is needed for alky and it would take a LOT more time…perhaps an F-0.50 would be a really optimistic goal to shoot for…not even in the same class.

    As far as heat value of methanol vs ethanol, you will find that methanol has a lower heat density than ethanol. Here is a public reference: https://en.wikipedia.org/wiki/Heat_of_combustion
    Methanol has about 9800BTU/lb, Ethanol has about 12,800BTU/lb or roughly 25% more heat per volume. It is easy to see that Ethanol will be more efficient since both burn roughly equivalently. The overall mass density is about .78 or so. “Ounces” is based on water density. 1 fluid ounce of water weighs 1oz. Alcohols/WG/Kero weigh around .78oz per fluid ounce. This means that a 16floz ‘coke’ bottle of ethanol will actually weigh slightly more than 12oz, discounting the weight of the bottle.

    Or, in other words, 1floz of methanol is roughly the same as 3/4floz ethanol in a stove. Anyway, drop me a line offline and we can discuss it if this is a poor explanation.

    Jan 4, 2019 at 2:22 am #3571377
    Ryan Jordan
    Admin

    @ryan

    Locale: Central Rockies

    you cannot use them to compare stoves using different fuels

    Well you can but the solid fuel/alcohol guys may not feel good about it 😂

    But we can cater to them by noting the F divided by system weight values. A canister stove plus the empty weight of a canister is much heavier than many alcohol stove, windscreen, pot support, and fuel storage bottle combos.

    Jan 4, 2019 at 2:39 am #3571378
    James Marco
    BPL Member

    @jamesdmarco

    Locale: Finger Lakes

    Ryan, unfortunately I do not agree with the full throttle protocol. It is probably the easiest and perhaps the only way to objectively test various stoves, though. But, as you know, turning the heat down can save as much as 30% of the fuel you would use on high. Some canister stoves cannot be turned down to a slow burn. Example: the MSR Reactor. The actual heat output also depends on the jet size. Some stoves simply do not produce a lot of heat (example: JetBoil Sol) and would be penalized by the time to boil yet produce hot water quickly with a minimum of fuel in the field as designed.

    I really like the idea of starting with icewater. This is fairly consistent. More than this would require distilled water to be used for ice and for the water to boil. The DeltaT  normalizes off standard boils due to altitude and is a good number, also.

    I do not worry so much about type of fuel. nButane/Butane/Propane/WG/Kero all have very similar specific heats. I just lump these together as petroleum fuels. Pressure, however, WILL effect the high settings you are using. As you discussed, a water bath to stabilize the cannister fuel temp would help. Typically in a lab setting, this would be a constant flow of some Temp of water, but difficult to set up in the kitchen labs. Anyway, this is an excellent try at standardizing the testing. Thanks!

     

    Jan 4, 2019 at 2:46 am #3571381
    Vance P
    BPL Member

    @vance

    Ryan –

    This is an excellent step in standardization of testing. Thanks for the work! I’m glad to see that you had Roger, Hikin’ Jim, Gary and Jerry help with input. Guru’s are they all!

    Just as you note in your comment to Eric above, once there are benchmarks, then expanded comparisons can be made. The F number is essentially focused only one tiny (but significant) portion of the entire set of system components and actions – the flame. I think the next important move is incorporating some kind of total system weight into a further efficiency rating.

    Of all the zillion other variables that can cause us to lose sight of standardization for meaningful comparison, the main factor for comparison is adding in a factor regarding weight – not just basic total system weight but also how that weight changes across the duration of an outing – that would be part of practical efficiency measurement we need to help us make good decisions about what cooking system will work for which outings and back country environments. A total system weight needs to include, at least the pot, the fuel as we will start our outing (often full), fuel container, any windscreen and a fire-starter.

    [For myself, I’ve gone through a few stoves through the years: A Svea 123 (jet rocket!), a Camping Gaz Trident (the ‘low’ setting is NOT simmer), MSR Windpro, a Primus Gravity, a Optimus Vega, a Kovea Spider (using Jon Fong’s wonderful kit) – but the one stove that always kept me thinking “there must be a better way (weigh?)” was the Coleman PowerMax stove – and fuel. It was so easy (even our scouts) to squeeze efficiency out of those things! But, of course, it was the fuel container that set it apart – and led to it’s ultimate demise. I solved my yearning for a similar approach to the lightweight fuel container when I read on Jan Rezac’s isobutane solution to Roger’s winter stove – https://backpackinglight.com/forums/topic/83643/. That combo kills on weight-related efficiency when compared to anything needing a STEEL can for containment*.]

    The total system weight comparisons I’ve done, both at trailhead weight and end-of-outing weight, show that fuel & fuel container make a huge difference in practical efficiency.

    >>>>Keep up the great work!!   -V

    * – MSR Iso-Pro (20/80) 8-oz steel canister = 72.72 btu’s per gram of canister weight; ; Ronson 5.82-oz aluminum canister (100% isobutane) = 158.12 btu’s per gram of canister weight.

    Jan 4, 2019 at 3:34 am #3571389
    Ryan Jordan
    Admin

    @ryan

    Locale: Central Rockies

    @jamesdmarco:

    I do not agree with the full throttle protocol. It is probably the easiest and perhaps the only way to objectively test various stoves, though. But, as you know, turning the heat down can save as much as 30% of the fuel you would use on high.

    I totally agree. Once we have a full throttle benchmark, we can evaluate from there how to optimize the systems accordingly.

    Some canister stoves cannot be turned down to a slow burn. Example: the MSR Reactor.

    Right, as is true with most pressure-regulated stoves like the Reactor. More accurately, the range of throttle is limited at the high end.

    Some stoves simply do not produce a lot of heat (example: JetBoil Sol) and would be penalized by the time to boil yet produce hot water quickly with a minimum of fuel in the field as designed.

    And therein lies the beauty of the StoveBench Score – it rewards efficiency. Although some “stoves simply don’t produce a lot of heat” (which isn’t so great in adverse conditions), “stoves that don’t lose a lot of heat” is even better.

    Pressure, however, WILL effect the high settings you are using.

    Yes, for sure. That’s why it’s really important to start with a canister temp at a consistent temperature, it levels the playing field a bit and rewards stoves with pressure regulation, which is a very good design feature to improve cold weather performance and extend canister life (# boils) without *as much* need for manual control and monitoring of the valve.

    As you discussed, a water bath to stabilize the canister fuel temp would help. Typically in a lab setting, this would be a constant flow of some Temp of water, but difficult to set up in the kitchen labs.

    Yeah, I thought about adding this to the protocol, but eventually canned it for exactly that reason. I want people to be able to come up with StoveBench Scores on their own and have it be as reasonably comparable to this protocol as much as possible.

    Anyway, this is an excellent try at standardizing the testing. Thanks!

    Thank you, great feedback!

    Jan 4, 2019 at 3:37 am #3571392
    Ryan Jordan
    Admin

    @ryan

    Locale: Central Rockies

    @vance:

    The total system weight comparisons I’ve done, both at trailhead weight and end-of-outing weight, show that fuel & fuel container make a huge difference in practical efficiency.

    Yes, for sure – this is the kind of analysis that needs to be done in comparison reviews, and information about “how to choose a stove” etc.

    Jan 4, 2019 at 7:55 pm #3571492
    Jon Fong
    BPL Member

    @jonfong

    Locale: FLAT CAT GEAR

    First off, kudos to the team, this kind of analysis is rarely done.  The first results will be pretty interesting.

    I understand the concept of testing at full throttle as “tuning” each stove to a metered output would be extremely difficult.  I do have concerns about using a small 0.8 l pot as a benchmark as the diameter is pretty small for larger burner head (MSR WindPro).  Given the squirrel full throttle test, it may be more relevant to use a larger pot (8″ in diameter) to reduce the noise level in the test.  Yes, it doesn’t emulate a typical backpacker but, very few backpackers would run at full throttle.  The test results could be leveraged over to snow melters who are trying to produce volumes of water quickly.

    From my own experience with the Kovea Spider, I can boil 2 cups of 70 F using about 6 grams of fuel on a 1.3 liter titanium pot.  Wide open, I have seen the numbers increase to +12 grams.

     

    Best regards, keep up the great work.

    Jan 5, 2019 at 6:48 am #3571573
    Eric Blumensaadt
    BPL Member

    @danepacker

    Locale: Mojave Desert

    As these tests go forward over the next few years we will see that Ryan’s BASE DATA will be truly a benchmark for comparison tests of:

    ->various fuels

    ->various shapes of pots

    ->different pot metals

    ->different burner shapes

    ->flame distances from pot bottoms (esp. with different fuels)

    And here I submit for your approval new tests to be included:

    ->the efficacy of religious incantations on boil speed

    ->the efficacy of cursing on boil speed

    ->the bliss of complete ignorance of stove efficiency (In which we test the blissful cook’s blood pressure and compare with the BP of those doing the testing.)

    GOOD LUCK Ryan!

    Jan 5, 2019 at 3:55 pm #3571600
    James Marco
    BPL Member

    @jamesdmarco

    Locale: Finger Lakes

    Eric, not so bad as all that.   Come, come, now… we know that prayers, magical incantations or curses have little effect on the stove’s F value, though they may indeed help warm up the person and surrounds a bit depending on how energetically or vehemently they are performed. StoveBench numbers do not represent much beyond the confirmation of the manufacturers specs in most cases. The question is, what does the F value actually represent, and the corollary, why should we be testing for it here at BPL?  Unfortunately,  I was hoping for a more simple and complete testing protocol rather than the bits and pieces being proposed here as simply a part of a larger whole. But, science is always plodding, ponderous testing, testing and more testing…

    Yes, I agree with Ryan totally in the sense that these are valuable numbers to have and verify against the different manufacturers. But, his calculations for a single F value at this stage is premature. Yes, he limits variables and potential sources of error. The fuel, water, pot, set-up conditions, etc read like the first paragraph of a lengthy research paper.

    But, his implication that we could then just calculate and enter the Fvalue into a database is incorrect as is his assumption that the raw data should be combined to produce an F value that has meaning beyond the collection of data. As one of the first steps in database normalizations, you NEVER enter a “calculated” value into a database. If needed, you can always just produce this number on a spread sheet (or from a group of tables in a report.) It becomes an unimportant number to enter and track, rather it belongs on the LAST page(or as a member of further calculations) on a report if needed at all.

    Does this protocol answer what stove is best for the conditions each of us hikes in? No. It is a lab protocol for testing stoves…a really good start on a standardized procedure. Using icewater as a starting point is a great idea. A known starting point will help us all…easy to do, easy to use. I highly recommend this as a starting point for any stove tests and/or assume a 1C/34F using the ice water. I am not sure about the ending point. As Ryan demonstrated, you have to know your elevation. Storm systems can effect air pressure. Boiling points can vary due to superheating, too.  Typically in a lab, a magnetic stirrer would be used but in a home lab, this is a bit much. I would suggest a simple 76C/169F ending point. This avoids all the altitude issues and minimizes vaporization/heat losses due to near boiling temps. (Note the range works out to an even 75C degrees.) Icewater is a good start.

    Defining efficiency is a matter of comparing actual work done with potential work done. For that, we are not getting a good comparison between stoves. As Eric mentions, there are a LOT of variables involved there. Even with the pot Ryan suggest we use, it is easily overdriven on high with some wider burners with flames spilling over the sides. (Indeed, I have seen this with some stoves actually increasing boil times/fuel usage.) On the lowest possible setting, you get heat losses from the actual pot sides and differences in BTU(KW/h) outputs from the stove itself needed to maintain the low setting. In between, settings are arbitrary and not well defined, nor, easily reproducible. Both increased time-to-boil and fuel-use on high, as used in the F calculation, can lead to misleading values. Not good…I don’t have any good suggestions there.

    Most of us are more interested in a good high power stove system (capable of winter use) that has high efficiency on low settings (capable of three season use) and is extremely light in weight (capable of all season use) and takes up a small volume in a pack (there isn’t that much room in my pack.) What stove should I buy (or make?) This is perhaps a more important question.  Given the number of variables even with this, it is far from a simple answer. Let us not loose site of the goal.

    Jan 5, 2019 at 4:17 pm #3571609
    Ken Thompson
    BPL Member

    @here

    Locale: Right there

    So with the nearly endless combinations of pots, stoves, temps, etc.. what is the practical application here?

    I know that I get routinely 14-15 boils per canister. What am I missing?

    Edit: I see my education is lacking.

    Jan 5, 2019 at 4:39 pm #3571612
    Gary Dunckel
    BPL Member

    @zia-grill-guy

    Locale: Boulder

    Ryan, the final draft looks great. While some of the readers might have doubts that this type of testing protocol reflects their own personal use of canister stoves, the StoveBench approach allows one to rather accurately compare the performance of every stove system in his/her arsenal. One of the challenges of any scientific inquiry is to assure that the results can be standardized, and that they can be replicated. The StoveBench Protocol essentially minimizes human error and subjectivity when comparing various stoves.

    While it is true that almost none of us start with 32* F (0* C) water, and that factors like altitude and wind will alter a stove’s efficiency, at least we can objectively compare the performance of each stove.

    Several years ago I devised my own (amateurish) study of my favorite stove/pot systems. I was mainly interested in knowing how they all compared at the same ambient temperature and altitude with no wind, using a standard water temperature (2 cups/boil from my refrigerator). I was most interested in time to boil, grams of fuel consumed, and also the total system weight (stove, pot, and canister). I simply multiplied (time to boil) X (fuel consumed to achieve the boil) X (system weight). The lower the number, the better. Later I decided that the time to boil wasn’t all that important to me – 3 minutes, 4 minutes, who cares? So I just settled upon (grams to boil) X (system weight). Anyway, this gave me a rough idea of how my stoves compared.

    But the StoveBench Protocol now offers a definitive method of accurately evaluating the differences in performance of any stove out there. The user can determine which stove will best meet his/her requirements. We can all work with the variables of altitude, wind, ambient temperature, and starting water temperature, etc., but at least we will know with some confidence which stove should perform the best for our intended purposes.

    Jan 5, 2019 at 9:04 pm #3571650
    Ken Thompson
    BPL Member

    @here

    Locale: Right there

    Thanks Gary.

    Jan 5, 2019 at 9:14 pm #3571656
    Scott Chandler
    BPL Member

    @blueklister

    Locale: Northern California

    The bottom line to these initial tests are “How much fuel does it take this stove to raise the temperature of the water 1 degree?” That’s the constant we are all looking to compare. Next comes the detailed testing of pot size, flame height, throttle opening, fuel types, weight of the stove itself, etc.. Great first step. Look forward to seeing the results within the scope of this test.

    Jan 6, 2019 at 12:53 am #3571702
    Jon Fong
    BPL Member

    @jonfong

    Locale: FLAT CAT GEAR

    Next question.  What about sample size?  When I test stoves, I typically use a sample size of at least 6 runs.  Without really getting into statistics, that sample size is pretty low from a confidence/reliability standpoint.  It does give a reasonable, first look output if you are looking at mean and standard deviation.  Then there is the question of what value are you reporting?  Mean minus one standard deviation to account for maybe 67% of the population?  It starts getting steep and deep if you need statistical information however, it is pretty solid information.  The bottom line is a sample size of one is not very valuable.  My 2 cents.

    Jan 6, 2019 at 4:18 am #3571741
    Ryan Jordan
    Admin

    @ryan

    Locale: Central Rockies

    @jonfong – sample size is *very* important! It’s probably the elephant in the room for most kitchen-counter stove-testing enthusiasts and cottage/garage stove makers.

    The short answer is that 2 is an absolute bare minimum – IF you have instrumentation with good measurement resolution. Two samples tell you qualitatively that your method might be OK.

    Three samples are required for rudimentary statistical analysis, and 6 are required for rigorous statistics that have a believable standard deviation.

    If your measurement accuracy is not great, the sample size may need to go up in order to get a valid average.

    I don’t get too worked up with people not having fancy instruments when I’m reviewing others’ stove tests, but I do wish they’d provide repetitive samples and disclose their statistical data so we all have a feel for how much scatter there is in the tests. I’ve purposely performed rudimentary, poorly-measured tests in the field under a range of conditions (ie not a controlled environment), and then use that scattered data to ballpark my probably low and high fuel needs for a trip. That’s a very valuable exercise for any stove enthusiast.

  • Author
    Posts
Viewing 25 posts - 1 through 25 (of 73 total)
1 2 3 →
  • You must be logged in to reply to this topic.
Log In

Want outdoor gear and skills info you can really trust?

Get our Handbook - the resource you need to make intelligent decisions about gear, safety, comfort, and pack weight.


Today's Gear Deals

Guide’s Gear Recommendations

Find out what gear our guides recommend if you want lightweight gear that is durable and versatile.

guide's gear logo

Gear Recommendations

  • Publisher’s Gear Guide
  • Staff Picks
  • Guide’s Gear Recommendations
  • Our Lightweight Gear Recommendations for REI Members
  • Today’s Gear Deals
  • Search for Gear on Sale

Subscribe Right Now

Receive new Members-only content, gain access to 2,000+ articles in the archives, and become a part of the most passionate community of backpacking experts in the world.
Subscribe Now
  • Backpacking Gear Reviews
  • Backpacking Skills
  • Backpacking Trips
  • Backpacking & Outdoor News
  • Outdoor Recreation Science & Technology
  • Backpacking Courses, Webinars & Other Events

Follow Us

Get outdoor skills and gear info you can trust.

Download the Backpacking Light Handbook to help you make intelligent decisions about gear, skills, ultralight philosophy, and reducing your pack weight.

Join Now: Support fair and objective product reviews.

Something for everyone: Basic, Premium, and Unlimited Membership options available.

View Subscription Options

More @ Backpacking Light

  • About Us
  • Jobs
  • Advertise with Us
  • Write for Us
  • Submit a Product for Review
  • Diversity Grants
  • Help / Support / Contact
  • Terms & Policies

Call Us

Membership Sales & Support: 406-640-HIKE (406-640-4453) | About

© Copyright 2001-2021 BEARTOOTH MEDIA GROUP, INC. | U.S. Library of Congress Serial Registration ISSN 1537-0364
BACKPACKING LIGHT® and the FEATHER/MOUNTAIN icon are registered trademarks granted for exclusive use to Beartooth Media Group, Inc. Subscribe here.

  • Subscribe
  • Log In
  • My Account
  • Forum Profile
  • Private Messages
  • Newsletters
  • My Course Enrollments
  • Unlimited Membership Portal
  • Help / Support / Contact