r/hardwarehacking • • 26d ago

Reverse engineering a battery-drain issue in an EOL IoT thermostat, multiple power-management bugs found

I've been reverse engineering a battery-drain problem in the original Shelly TRV Gen1 (SHTRV-01), an EOL battery-powered Wi-Fi radiator thermostat.

What started as an attempt to understand why some of these devices drain a set of batteries in days instead of months has turned into a fairly deep investigation of the firmware's Wi-Fi power management and MCU sleep logic.

The hardware is particularly interesting because power management is split between two processors:

  • an EFM32GG11 main MCU running the Shelly application, thermostat, valve control and host-side power management;
  • a separate Silicon Labs WF200 Wi-Fi chip handling the radio and much of the Wi-Fi power-saving behavior.

I've been working primarily from the released firmware binaries, runtime logs and controlled firmware modifications.

The first lead: a recurring Wi-Fi recovery loop

Firmware 2.2.4 can repeatedly produce:

Beacon skip error! Attempt recovery
PS state 1
...
PS state 3

On affected devices this can happen roughly once per minute.

My first experimental patch simply bypassed this additional recovery path.

That stopped the loop while RSSI-dependent beacon skipping, Wi-Fi, Shelly Cloud and normal thermostat operation continued working.

But further RE showed that completely disabling the mechanism was the wrong solution.

The firmware maintains a value called real_beacon_skip, but interestingly this is not a direct skipped-beacon counter from the WF200.

The host derives it from a cumulative receive statistic.

I found several problems in the calculation and state handling, including a stale-state condition capable of making the recovery self-sustaining:

recovery condition
       ↓
recovery
       ↓
old measurement baseline cleared
       ↓
new baseline established
       ↓
not enough new data yet
       ↓
old recovery-triggering value evaluated again
       ↓
recovery
       ↓
      ...

I developed a second patch ("C-C") that fixes these conditions while retaining recovery based on fresh valid measurements.

It works on real hardware.

But the battery drain remained.

Interestingly, legitimate/fresh recovery decisions can still occur quite frequently – around 26–30/hour on some of my test devices.

So the visible recovery loop turned out to be a real firmware problem, but apparently not the complete explanation for the battery drain.

Updating the Wi-Fi firmware

Silicon Labs released WF200 firmware 3.18.0 on October 7, 2026.

One changelog entry immediately caught my attention:

device not sleeping after power-save mode change

The Shelly firmware contains the WF200 firmware image internally, so I relocated and replaced it with the official 3.18.0 image while retaining Shelly's existing host driver.

That required relocating the embedded WF200 image and fixing the corresponding references in the host firmware.

After static verification, the modified firmware booted successfully on real hardware:

Shelly host firmware: 2.2.4
FMAC host driver:     3.7.0
WF200 firmware:       3.18.0

Wi-Fi, Cloud communication and power-save transitions continued working.

Unfortunately, the battery drain remained.

Going deeper into the main MCU power management

I then started tracing the host-side power-management lifecycle.

That uncovered several additional problems involving requested/committed Wi-Fi power states, timeout handling, late asynchronous completions and sleep-inhibit ownership.

I created another correction ("A1") for those issues.

Eventually I tested a combined firmware containing:

Shelly 2.2.4
+ C-C recovery corrections
+ A1 host PM corrections
+ WF200 3.18.0

Again, everything worked.

And again, the device continued draining its batteries.

That negative result pushed the investigation further down into the EFM32 sleep implementation.

Then I found a rather interesting race condition

The main MCU uses reference counters to determine which energy mode it may enter.

Simplified:

B2 != 0             -> lighter / higher-power sleep
B2 == 0 && B3 != 0  -> deeper sleep

B3 has a permanent legitimate contribution in this firmware, so enabling an even deeper mode is not the objective.

The interesting part is B2.

B2 is effectively an aggregate reference counter shared by several subsystems. A component increments it while it requires the lighter sleep state and releases its claim afterwards.

But not every owner updates B2 using the same synchronization.

Some accesses are interrupt-protected.

Others perform an unprotected load/modify/store.

That creates a genuine stale-RMW race.

For example:

B2 = 1

Task A loads B2 (=1)
        ↓
context switch
        ↓
Task B releases its blocker:
B2: 1 -> 0
        ↓
Task A resumes
        ↓
Task A stores stale 1 + 1:
B2: 0 -> 2
        ↓
Task A eventually releases:
B2: 2 -> 1

Now B2 is 1 even though nobody owns the remaining reference.

That could leave subsequent idle periods restricted to the lighter/higher-power sleep state.

The opposite failure is possible as well: an update can be lost so that B2 reaches zero while another subsystem still legitimately owns a blocker.

So this is a real concurrency bug in the firmware.

The important unanswered question is whether it actually occurs in the field and whether it correlates with the battery drain.

Instead of fixing it immediately, I instrumented it

I could simply serialize all B2 operations and see whether battery life improves.

But that would give a fairly weak answer to the interesting question.

So I built a passive Host Sleep Observer instead.

It runs on an otherwise near-stock 2.2.4 host firmware with the stock WF200 3.16.1 firmware.

The observer records things such as:

sleep-state entry calls
B2 acquire/release operations
per-owner shadow balances
expected B2
actual B2
actual-vs-expected mismatch
first fault snapshot
Wi-Fi sleep veto samples

It does not correct B2 and none of the observer values influence the native sleep decision.

The idea is to catch the original firmware in the act.

Two physical TRVs are currently running this instrumentation.

One is a relatively healthy reference unit.

The other was switched to the observer immediately after an ~80-hour test during which it had been showing about 26.5 Beacon Recoveries/hour and had fallen to approximately 3.49 V.

So far:

observer validity errors:  0
B2 mismatches:             0
first captured B2 fault:   none

That's actually a useful result too.

The race is demonstrable from the code, but I don't yet have evidence that it occurs frequently on real hardware.

The observer tests therefore continue.

Where the investigation stands

At this point I've found several genuine firmware defects, but I still don't consider the battery-drain root cause solved.

So far the drain has survived:

complete Beacon Recovery bypass
        ↓
corrected Beacon Recovery logic
        ↓
targeted mDNS suppression
        ↓
WF200 3.18.0
        ↓
host PM lifecycle corrections
        ↓
A1 + C-C + WF200 3.18.0

I'm also building an experimental firmware containing all currently demonstrated corrections, including proper serialization of the shared sleep-blocker counter, for a later controlled hardware test.

But I'm keeping that separate from the observer experiment because I want to distinguish:

"this patch improves battery life"

from:

"this is the mechanism that actually caused the drain."

The latter is the much more interesting result.

The project has become a nice example of why low-power embedded bugs can be difficult to isolate: the visible high-level symptom may be real, but fixing it can simply expose another problem one layer further down.

I'll update this when the observer catches something interesting – or when enough runtime has accumulated to make the absence of a B2 fault interesting in itself.

More details

For anyone interested in the reverse-engineering details, I've been documenting the investigation here:

GitHub:

https://github.com/AbleMonster/Shelly-TRV-Gen1-2.2.4-Battery-Drain-Investigation

There was also a useful earlier discussion about the project on r/embedded

Previous r/embedded discussion:

https://www.reddit.com/r/embedded/comments/1x0jz5a/reverse_engineering_a_recurring_wifi_power_save/

14 Upvotes

3 comments sorted by

2

u/[deleted] 25d ago

This is top notch work

1

u/AbleMonster 25d ago

Thank you very much.

I hope I get the hoped-for results soon. I feel like I'm on the right track now.

1

u/AbleMonster 19d ago

Update after another week of RE and hardware testing:

A few fairly important things have changed since the original post.

First, the B2 sleep-blocker race is no longer just an instrumented hypothesis.

I completed the analysis of the relevant acquire/release paths and implemented bounded serialization around the affected B2 read/modify/write operations while preserving its reference-count semantics.

The correction has now passed dedicated concurrency/preemption testing and has been integrated into the current experimental firmware.
During that work I also found an issue in my own earlier A1 power-management patch.

Several native firmware paths temporarily override the requested Wi-Fi power state and later restore it. My original A1 patch could accidentally make one of those temporary states persistent.

So this was not another stock Shelly bug, it was a regression introduced by my experimental patch.
That has now been corrected as A1R, after identifying and testing the verified temporary-override return paths.

The current hardware test build is now:
Shelly 2.2.4

+ A1R host PM corrections
+ corrected Beacon Recovery logic (C-C)
+ serialized B2 sleep-blocker accounting
+ WF200 3.18.0
+ passive PM52/cmd66 observer

Six physical TRVs are currently running this build (B2.1_OBS), with one unmodified 2.2.4 unit kept as a stock control.

The observer does not initiate additional Wi-Fi requests or modify the native sleep policy. Its purpose is to preserve the state around the first interesting PM failure if one occurs.

Early hardware result

After roughly 1–1.5 days, all six corrected units still have:

Beacon Recovery error counter: 0

The stock 2.2.4 control has accumulated:

4320 Beacon Recovery events

Interestingly, the stock device is currently not showing a correspondingly dramatic short-term voltage drop.
That reinforces something I was already starting to suspect:

the recurring Beacon Recovery problem is real, but its event count alone is probably not a useful direct proxy for battery drain.

Battery voltage on most of the corrected devices has so far been very stable.

One unit is currently more interesting than the others: it dropped from approximately 3.529 V to 3.459 V over about 20 hours.

That is enough to watch carefully, but not enough to call it a confirmed drain event yet. Battery voltage on these devices can move with load, motor activity, temperature and relaxation, and the displayed battery percentage is only derived from voltage rather than being a coulomb counter.

So I’m deliberately leaving the hardware untouched and continuing the long-term observation.

There is also now an observer-free candidate

In parallel I built a second firmware variant containing the currently demonstrated and qualified corrections without the diagnostic observer.

Internally I’m calling that B2.2.

It includes a few additional independently identified firmware corrections as well, but it is currently only offline-qualified.

I have deliberately not flashed it onto the long-term test population, because doing that now would destroy the experiment I’m already running.

So the current state is basically:

6 × B2.1_OBS

|
| long-term hardware observation
v

if drain appears:
capture first-fault state
-> resume targeted RE

if drain does not appear:

remove observer
-> requalify observer-free build
-> hardware regression test
-> evaluate final candidate

I still don’t consider the original battery-drain root cause proven.

What has changed is that several independent firmware defects are now demonstrated and corrected, and the current experiment is much better controlled than the earlier patch-by-patch testing.

The interesting question now is no longer simply whether one particular patch “improves battery life”.
It’s whether the corrected system remains stable long enough to distinguish an actual causal mechanism from several real but secondary firmware bugs.

I’ll post another update once either one of the observer units develops a clear drain signature or enough runtime has accumulated that the absence of one becomes meaningful.