StoveBench: A Stove Testing Protocol for Comparing the Performance of Backpacking Stoves

[Jon Fong / Flat Cat Gear]

The F metric will give you an idea of stove performance, that being said what F value is important to you will depend upon your goals.  Dan comments points out that a higher F value may not add value for certain needs.  I recently tested an MSR WindPro II which is a flamethrower: it is a great stove for snow melting and heating up lots of water.  I suspect that it will have a high F value.  That being said, the lowest that I could turn the stove down to was about 250 watts and it was too hot for dry baking (even with a diffuser).  So, the F value even on the lowest setting was too high for me.   The Kovea Spider performed much better for my needs even though the F value will probably be significantly lower than the MSR WindPro II.  My 2 cents.

[James Marco]

Jon, Dan. Yes. Those are a couple of the things that happen with a single calculated value and strictly testing on high.
Here is a brief of my main objection:
I believe what is happening is that the F value is basically a linear calculation. Fuel use/time to boil(efficiency) is not. It is on a curve. Though, the exact nature of that curve is indeterminant (after well over a thousand tests, I never did determine that.) Most of the varibles (pot size, materials, heat shield, insulation, flame to pot distance, fuel, etc) are the ones we use to optimize for efficiency. At the narrow range we are interested in, it almost seems linear. It isn’t really, though. Indeed, we generally use an average to “smooth” things out.
Ryan is making the same assumption with his Fvalue. Unfortunately, at the high heat levels, he really cannot get away with that, hence some rather profound problems/data conflicts. It also shows up at very low heat levels, though these are usually ignored and not in Ryan’s testing… simple logic dictates that letting a pot simmer at 80C for hours means you just burned a LOT of fuel to no effective purpose…the water never boiled, it simply evaporated away. In other words, this is an effective infinity in fuel use. Our stoves and fuels effectively limit the amount of actual heat they can deliver, so we don’t see this at super high levels…ie a VERY large amount of fuel burned to heat the water in a VERY short time…think about sun temps/plasma disassociation. Anyway, so much for the outer limits…(pun intended.)
The efficiency of a heating system has a lot of variability, but generally follows the blue line on the graph. I don’t know the exact nature of the curve, hyperbolic, exponential, or other. From very high, turning down the stove, it gets more efficient to a point, then decreases in efficiency.
The red line is the hypothetical Fvalue calculation. (No, I didn’t try to compute the general slope. I know, lacking rigor.)
The green line is the general range of testing most of us use, or even bother with. This is not really linear, but within minor limits, can be approximated by a simple average. I suppose it really shouldn’t extend beyond the use line, sloppy drawing on my part.

While this is ill-defined in the exact values, it clearly shows why the Fvalue is not a good number. The Fvalue may show very good boil times and fuel usage to achieve the boils. But it says nothing about somewhat lower heating settings and overall system efficiency. It is not even accurate to use it in any other calculation, though “like a stopped watch, it is accurate twice per day.”
As a max heating value of a stove, StoveBench is a good protocol. Many will find it useful for solo snow camping where max heat is wanted to boil water from ice cold water temps as it sits. As far as a normalized number, though, I think Fvalue falls short of being able to back-calculate the best stove system for three season use, or, the most efficient stove to use in any conditions.

Do not throw out the good with the bad behind the protocol. Regulated water temp, regulated volumes, regulated pot material (and type & size,) regulated fuel delivery, regulated temp measurement, regulated timings, regulated data entry, etc. All make the protocol a leap in the right direction and demonstrate a truly good methodology. All parts of a good test protocol. Ryan amply demonstrates his technique.

[Jon Fong / Flat Cat Gear]

James,

I tend to agree with you and I do support what Ryan is trying to accomplish.  I have already submitted data on an alcohol stove and an ESBIT stove.  Hopefully, more people will enter data.

Typically when a protocol is set up, it needs to be targeted towards a design center.  In some cases, this is being done by referencing a 0.85 l pot.  That being said, using a stove like the MSR WindPro II at max setting will probably be inefficient as the flame pattern will be huge compared to the pot size.  The BRS-3000 will probably do ok.

I drafted these graphs (guestimates) to demonstrate a point, I believe that they are similar to your charts

For a small pot diameter, I suspect that the efficiency will drop off with stoves outputting over 1000 watts (assuming a 50% efficiency so 500 W delivered to the pot)

As the stove overpowers the pot, time to boil probably levels off as the efficiency drops off.

 

As I said, I do support Ryan’s efforts and have submitted data.  Alcohol & Esbit don;t have a max setting so it is easy to test.   I do suspect that running the test on max for canisters will push the results far to the right.  My 2 cents.

 

[Jon Fong / Flat Cat Gear]

Some Preliminary data using a Kovea Spider in a Bobcat SS Windscreen and an Evernew Ultralight pot.  F value are for various fuel settings (% turns).  In all cases, 1 liter was used.

For my own use, I never run it even a 1/4 open.  The efficiency numbers are

59.6% to 59.1 % to 51.8%

My 2 cents

 

[DAN-Y/FANCEE FEEST]

Roger, which is the better canister stove valve design?

Door #1

 

Door #2

[Roger Caffin]

Hi Dan

Without any hesitation at all: the first one. It is simpler and more reliable.
The SOTO pressure regulator is mainly a marketing gimmick. It was full discussed in our article:
https://backpackinglight.com/pressure_regulators_myth_part1/

All it does is REDUCE the available temperature range the canister can handle, because there is an extra pressure drop across the servo valve (and it makes the stove more expensive). SOTO et al claim that the regulator means you don’t have to adjust the valve as the canister cools down (over a half hour burn), but who cares if you do have to adjust the valve while you are cooking dinner. For little burns – a cup of coffee, there is precious little effect to be seen.

You may find many vendors (retail shops) which claim that the regulator extends the temperature range the canister can handle. That is absolutely FALSE, and is due to a combination of ignorance on the part of the retailer and a failure to read the SOTO specs. SOTO themselves do NOT make this claim.

Yes, I do have one and I have tested it thoroughly.

Cheers

[James Marco]

Jon, yeah, your graphs are consistent with mine. Some good test data…

[DAN-Y/FANCEE FEEST]

A photo taken at a sports store along the AT back in the year…..by me and posted on bplite

[URL=https://smg.photobucket.com/user/obijiwa/media/DSCF2252.jpg.html][IMG]https://img.photobucket.com/albums/v228/obijiwa/DSCF2252.jpg[/IMG][/URL]

[Ryan Jordan]

I crunched some data to recreate Jon’s estimate graphs above about how stove output power (gross wattage) affects efficiency and boil times. This data was generated using a needle-valve canister stove, standard protocol as described in the StoveBench article.

And gross wattage vs. StoveBench Score:

[Vance P]

Ryan –

I’m understanding your “Gross Wattage” as meaning the cumulative energy expended across the entire testing time (from chilled to boiling) for the boil – as opposed to a a specific wattage reading (measured at any given point during the test) that happens to be sustained from chilled to boiling – Correct?

Vance

[Ryan Jordan]

Vance – yes – the first one. Each wattage value represents a particular test performed at some level of output as controlled by a fixed position on a fuel valve adjustment handle.

Here’s how I calculate gross wattage:

W = mass of fuel used in test ⨉ standard heat of combustion of the fuel ÷ test duration

[Vance P]

Ryan –

Gotcha – So it is energy expended over time (watts per hour, joules/second (j/s?), etc.)

I think the exploration in the comments so far have to do with whether the “F” number actually expresses efficiency – meaningfully – per se. Punching the accelerator pedal in my GTO from the stop sign in order to hit 60 mph before the next stop sign (yeah, used to do that) will not provide particularly accurate or meaningful measurements of efficiency. (In radio parlance it’s called a ‘blowtorch’ when you live near an a 50,000 watt AM station that’s able to be licked up 1,000 miles away — essentially a true waste of power (aka inefficient) for your little crystal set 2 blocks away…) You note in your report that “Efficiency is a function that balances fuel economy with flame power”. Yes and no. Pure flame power is very deceptive in how it can skew what we really want to know – which I believe is the balance between fuel economy and power/work/energy >delivery<.

Jon’s characterization of the “F” number being more like an expression of a subjective “effectiveness” is more accurate (semantically). But even so, I’d like to add a second factor to my one of “per gram” suggestion above: specific wattage capability. That would be the range of specific energy output. [A Camping Gaz Tristar has three click positions to it’s control valve which my son and i describe as “high”, “blowtorch” and “volcano”. There is no medium, low or any tapering. It’s hideously inefficient in my experience, but might have a decent “F” number in the right set-up (probably soon-to-be badly-annealed aluminum)] The idea is that a stove needs to have the capability to expend up to a certain specific wattage at any time in order to effect/conduct our desired phase change (boiling). My Tristar can do that in spades – AND it can sustain it. A 14g Esbit tablet in a gram cracker stove with no windscreen is not able to do that to 2 liters of 33-degree water pretty much at any point during its burn time.

Now, with that in mind, is the stove not only capable of hitting that specific wattage, can it sustain it? Then, to my point, is the stove able to provide >THE< appropriate/correct (most efficient) specific wattage for the given pot/windscreen/water volume/delta-temp-range – sustained all the way through the entire phase change/delta-temp. Now the question we need to use to calculate efficiency is: How much fuel (in mass/weight) was expended to perform that above-described task?? THAT is an efficiency measurement.

Even so, my quest is always toward – further – finding that efficiency level noted above for any given cooking rig as measured on a per-gram/per-ounce basis.

I believe that the proposed “F” calc is a good start – and I look forward to the first chart of findings. But I also agree that Ryan’s comment above that “Once we have a full throttle benchmark, we can evaluate from there how to optimize the systems accordingly.” means that we need to find the real stove efficiency measurement and then calculate the weight-efficiency from that point.

 

Just sayin’…. Vance

[James Marco]

Vance, yes. The Fvalue does have some meaning at the high end. But the actual “real stove efficiency” is still not effected much. Controlled testing is always a problem. And, making those tests seem valid across a wide range of stoves and different fuels is a problem.
Thanks Ryan for the additional testing/calculations. It seems to show that low Fvalues will skew a stove system’s overall efficiency more towards higher fuel efficiency. Expected.
Around 10:15 seems to be the best fuel efficiency based on Ryans new graphs. And ten minutes is not an extreme amount of time to wait for boiled water once the stove system is set up and operating at camp. This pretty much jives with my tests. 7-10 minutes (in that range) seems good for overall fuel efficiency. Longer times means the pot radiates heat fast enough to hurt efficiency. Shorter times seem to mean heat is wasted trying to boil the water. Expected.
Plotted with the heat, this is just over 500J/s, again taken from Ryan’s graphs. But, looking at the fvalue, there is little relationship between the most efficient water boils (least fuel used) and Fvalues. It seems to show an inverse relationship, again expected, but also a curve, unexpected. The value of testing!

[Jon Fong / Flat Cat Gear]

Ryan,

Great Chart!  This is what I see: The green slope represents efficiency and at low to nominal wattages, it is approximately linear.  The brown line is when the F value is less sensitive to stove wattage.  The intersection of the 2 lines tends to indicate a optimal point for speed w.r.t. efficiency.

As a side note, it may add value to change the X axis to grams/minute as it is probably more meaningful than gross watts.  It may be easier to internalize the trade off of F versus fuel consumption.   My 2 cents.

 

[Jon Fong / Flat Cat Gear]

Vance,

I have done a lot of testing with ESBIT and Alcohol and the stove output over time is highly non-linear.  That being said, joules are joule and heating up water is an energy integrator so the net effect has a minor impact.  The biggest difference with ESBIT & Alcohol is when you want to test large differences in water volume.  There are few people that I know of that heat 4 cups of water with either of these fuel.  In general, Alcohol & Esbit users are in the 2 cups and under group.

Isobutane stove in general have a pretty constant output so the average wattage is reasonable.  Yes, without a windscreen, the effective output will vary, but this is not in the test protocol yet (and it is difficult to make it repeatable).  That being said, I think that we are learning a lot already when evaluating the F metric.  The more you burn, the more you learn.  My 2 cents.

[Jon Fong / Flat Cat Gear]

SVEA 123 using the Sigg Tourist (Aluminum):  F Value of 0.343

Now, I pulled the SVEA 123 down off the shelf and it hadn’t been run in about a year.  It fired up pretty quickly, I had to remember which direction was off and forgot about self cleaning the jet.  This run did not really include the normal prime as I ran the test about 5 minutes after firing it up.  My stove is probably not nearly as optimized as James’ stove, but these are my results.

 

[Eric Blumensaadt]

This will continue to interest me in that I have several stoves to test myself, hopefully using BPL’s protocol but my own pot(s).

My stoves:

1. SVEA 123 – white gas (gotta include this icon)
2. MSR Whisperlite Universal – butane, white gas (petrol) and kerosene   (remote fuel stove)
3. MSR Windpro – butane (remote fuel stove)
4. Brunton CRUX (canister top stove)
5. TD Sidewinder Ti stove using ESBIT sized cubes of C4 (Just kidding, but something like de-tuned rocket fuel has been my hope to replace ESBIT. Maybe somebody in a country with lax safety standards (Moldova?) can experiment on this.)                                                                         OK now some NSA (No Such Agency) software will pick up my mention of this substance and tag me to be watched in cyber space. They will contact Ryan and my posts will not only stop appearing but past ones will mysteriously disappear. HEY, even paranoids have enemies. ;o)

 

[Roger Caffin]

my posts will not only stop appearing but past ones will mysteriously disappear.
Nah, nothing of the sort.
Of course, Eric B might mysteriously disappear … launched into space by a packet of rocket fuel.
:)

Cheers

[Jon Fong / Flat Cat Gear]

Whatever happened to the Stove Bench?  Lots of time and energy went into it.  Just curious.

[Ryan Jordan]

We’re now working on the “integrated canister stove” tests. Jetboils, Reactors, Windburners, etc. Gonna be interesting, probably even more so than the upright stoves. Lots more variability, esp. in inclement (cold, wind) conditions. I’m excited about this one! We hope to publish the review in February, we really want to get some “ice-cold” temp testing in for these.

[DAN-Y/FANCEE FEEST]

StoveBench defines a protocol that is used to measure two important backpacking stove performance features (power and efficiency) in order to determine a single, quantifiable performance factor called the StoveBench Score.

Ryan, stick with the indoor testing, make it simple.

[edited by MK forum guidelines here]

[Jon Fong / Flat Cat Gear]

Well, time for an annual update.  Where the heck is F?

F = [ 0.5 L ⨉ 100°C ] ÷ [ 4 min ⨉ 15 g ] = 0.83 L•°C/min•g

[Roger Caffin]

Hiding behind Wally.

[Kin C]

Was there an update on the stovebench program for integrated stoves?

[Jon Fong / Flat Cat Gear]

StoveBench: is this a dead topic? Is there a conclusion to this effort?

[Author]

Posts