LightBench: A Laboratory Testing Procotol for Comparing the Performance of Flashlights and Headlamps

Updates

• December 13, 2024: A quantitative measure of effective brightness (B_eff) was introduced. It represents the time-weighted average brightness of a light during its battery discharge cycle. It provides a more representative measure of brightness performance than manufacturer specifications of brightness measured at single points in time.
• December 13, 2024: LightBench tests described in this report were repeated with laboratory-grade, calibrated instrumentation for greater accuracy. All reported data and interpretations have been updated.

Introduction

Comparing the performance of handheld flashlights and headlamps objectively is challenging because there are so many design attributes that contribute to lighting performance. Not only are we concerned about brightness, efficiency, weight, and battery life (easily measurable attributes), but we also care about ergonomics, aesthetic design, beam quality, and ease of use. These attributes are more difficult to correlate with objective performance measures because different users have different needs.

In a recent survey of Backpacking Light users (November 2024, 1370 respondents), the most important features considered when selecting a light were identified as weight (78%), the presence of a rechargeable battery (57%), and maximum burn time (55%). In addition, brightness at maximum power (42%) and battery life at maximum power (32%) were not insignificant considerations.

Unfortunately, light manufacturers are not required to disclose technical performance details about their products, although some opt-in to standards such as the ANSI (Plato) FL1 standard. In that particular case, the Light Output ANSI standard for measuring lumens is reasonable. However, there is controversy over the usefulness of Beam Distance, Run Time, and Peak Beam Intensity standards in the context of how lights are used in the field by backcountry, tactical, and military users.

In addition, standardized methods only measure brightness at specific points in time (usually with a fully charged battery). Consequently, they fail to capture the overall average brightness of the light during its entire battery discharge cycle.

In this technical brief, we present a laboratory (bench-scale) method for quantitatively evaluating (and comparing) the performance of lights using an objective, measurable, and normalized metric (the LightBench Index) that considers brightness, how long a battery lasts during a single discharge cycle (a.k.a. runtime), and product weight. In general, the LightBench Index (LBI) is defined as follows:

LBI = brightness × runtime ÷ weight {eq. 1}

where brightness × runtime is an approximation of the total volume of light output by a product during a battery discharge cycle (we refer to this as light volume or LV in this report).

The LightBench Index (LBI) was developed to address the lack of standardized, weight-normalized metrics for evaluating lighting performance in backcountry environments.

In addition, the center of mass of the area under the curve is calculated, which provides a time-weighted measure of effective brightness (B_eff). This metric is useful for comparing the overall lighting performance of different products by quantifying typical amounts of light delivered throughout the runtime.

Two lighting products are used to illustrate the concepts: the Petzl Actik Core (v1 model, ca. 2019) and the Fenix HM50R (v1 model, ca. 2019). Updated (2024) versions of both lights are also being tested; those results will be released in upcoming test reports that compare several different brands and models.

Table 1. Physical specifications for the Fenix HM50R vs. Petzl Actik Core

Defining Performance Efficiency

Performance efficiency can be defined as the amount of output (i.e., performance) per unit of input (e.g., product weight or battery capacity). The two most common quantifiable measures of performance for lighting products are brightness (usually measured in lumens) and runtime (usually measured in hours).

Brightness and runtime are inversely related, as higher brightness requires more power. Given this correlation, overall performance must account for both brightness and runtime. This is a strategy we’ve used before in StoveBench, where both stove power and fuel efficiency (which are also inversely correlated) are used to identify an overall performance metric called the StoveBench Index.

Because of this inverse correlation, overall performance that considers these two measures should be the product of their individual values. Since brightness is measured in lumens (light per unit area) and runtime is measured in time (i.e., how long a battery lasts), the product of brightness and runtime is representative of the total “volume” of light that can be delivered during a single battery discharge cycle:

light volume = brightness × runtime {eq. 2}

Manufacturer specifications can reveal some clues about how much light volume their light can produce. Here’s an example of a typical specification table (for the Petzl Actik Core v1), and the resulting light volume for each mode (calculated using equation 2).

Table 2. Light volumes calculated from manufacturer brightness and runtime specifications at various brightness modes for the Petzl Actik Core v1.

For this product, light volume decreases as a function of brightness mode. This is expected, and the reasons for this are well-known:

• LEDs are not perfectly efficient at all brightness levels. Luminous efficacy (lumens per watt) decreases as current drops. Consequently, at the lower currents used to produce dimmer lighting, the LED requires more energy per lumen.
• At lower power draws, the LED power regulation driver (typically, a solid-state circuit) must step up battery voltage more aggressively, an inherently inefficient process that may place additional strain on the driver and draw more incremental power from the battery.
• Power-regulated lights become less efficient at lower light levels because a larger fraction of energy is allocated to powering the regulation driver.

Interestingly, not all lights (or power regulation drivers) follow the behavior that light volume always decreases as brightness drops (see Table 3).

Table 3. Light volumes calculated from manufacturer brightness and runtime specifications at various brightness modes for the Fenix HM50R (2019).

Table 3 suggests less efficiency (lower light volume) in the Turbo (highest) brightness mode of the HM50R. There are two possible explanations for this:

1. The light volume calculated from manufacturer specifications for brightness and runtime is inaccurate. This is because most modern rechargeable lights use advanced power regulation programming that results in stepped-down brightness as the battery is discharged (to improve battery life).
2. A significant amount of energy is lost to heat. This occurs when LEDs are operating near their maximum intensity.

The solid-state power regulation circuitry that is built into most rechargeable lights programmatically controls the rate of power discharge at different brightness levels. In some lights, that circuitry is programmed to step down brightness levels as the battery discharges to avoid heat accumulation and to extend runtime (“battery conservation regulation”). In other power regulation strategies, the circuitry is programmed to maintain a constant power discharge rate and, consequently, a (relatively) constant brightness level (“constant brightness regulation”).

Despite whatever programmatic algorithms that manufacturers build into their power regulation circuitry, there is a case to be made for evaluating a light’s overall performance by doing so at the light’s maximum power output. The reasons for making this case are:

1. The design and engineering of the lens and reflector assembly, the effectiveness of heat sink construction, and the optimization of power regulation circuitry are more stress-tested by brighter vs. dimmer lighting.
2. Lights are often most critical in emergency situations requiring sustained maximum brightness.

Relying on the manufacturer’s specifications to calculate light volume requires constant brightness regulation – a power regulation strategy that is somewhat rare in modern lights. There is so much variability in how brightness levels and power discharge rates are regulated by power regulation circuits that there is no gold standard. Therefore, calculating light volume from manufacturer specifications is not recommended.

Measuring Light Volume (LV)

Direct measurements, unlike manufacturer data, capture real-world performance, accounting for variables like heat loss and dynamic power regulation. Thus, a more accurate representation of light volume (LV) requires directly measuring real-time brightness levels during a battery discharge cycle. Accurate LV requires that brightness be measured in lumens (which integrates the entirety of light emitted by a lamp rather than just a spot in the beam, as might be measured by a lux meter). Thus, all tests are performed in a calibrated integrating sphere photometer (a spherical device with a highly reflective interior surface that measures the total optical power of a light source or sample) at an ambient temperature of 20 °C (± 1 °C).

A typical test results in a graph of time vs. brightness: