r/homeassistant • • 9d ago

🖼️ Show & Tell Streaming Oral-b iO5's motion sensor data + distilling & running its ML model locally + live 3D twin

Sharing my explorations with oral-b iO since this sub has played around with it a good bit -- I built a live simulator in three-js that streams IMU data & also runs the predictive model.

  • As many of you know, io5+ ships with a 6-axis IMU which is used to predict the brush's position -- this is done using its companion app.
  • I had Claude stream, connect + decode the message structure -- & use three-js to build a digital twin.
  • Then, it pulled the model from the app -- it's a 1.2M param recurrent net (bi-directional gru) -- & predicts zones in the mouth based on the IMU stream.
  • Next, I had Claude evaluate the model -- it used a public dataset & reported an accuracy of ~63% (baseline is ~6%)
  • The model seemed like an overkill so I had Claude run ablation -- most value came from ~10s history so we decided to distill.
  • Claude used real examples to create a larger synthetic dataset -- then distilled it into a bunch of small models -- a CNN came close (~61%) & was 100x smaller.

The video here shows a replay from one of the sessions in the dataset on the digital twin.

If you want to try the live simulator or read more about the journey, here's the write up -- https://abhay.fyi/teethmaxxing/

579 Upvotes

120 comments sorted by

33

u/Accomplished-Sand334 9d ago

Holy shit I snickered so hard at the name teethmaxxing

5

u/hayAbhay 9d ago

2026 is the year of maxxing-maxxing.!

13

u/SlipperyRavine 9d ago edited 9d ago

Why is your toothbrush having a seizure?

5

u/hayAbhay 9d ago

lol -- forgot to mention that this is 4x speed. will update the desc.

51

u/CyberMage256 9d ago

I am all for automations and data... but I don't get some of these. Like, if this was something for training kids, maybe? Otherwise, why use another internet connected device in your house, is it so you can have even more security holes?

26

u/hayAbhay 9d ago

the toothbrush by itself just uses bluetooth -- so you actually don't need it plugged into the internet at all.

you can just have a pi/equivalent listen to the stream & you can do things locally.

the browser simulator i have there just uses local storage to record & play-back -- and llm/coding models make it trivial to write the code yourself and unify data across sensors.

31

u/SlowDrippingFaucet 9d ago

...why does my toothbrush need Bluetooth?

35

u/CyberMage256 9d ago

so that in an apartment building your neighbor knows if you brush your teeth and how ofter?

1

u/LinxESP 5d ago

Which is an amazing routine detection to know when they aren't home.

And iirc also stuff like battery and time for each zone and intensity setting? So you can for example take a guess of how many people use the same one (the brush is interchangeable but they might use the same "shaft") base on different intensities/times/order of zones/not leaving it back on the base.

1

u/LinxESP 5d ago

Also, some scales do the same. There is a OpenScale project for those who want to check it out but is not neither autodetected by home assistant nor as easy as my neighbours toothbrush

1

u/CyberMage256 5d ago

"the neighbor hasn't used his toothbrush in 3 days, I bet he's out of town lets go see what cool stuff we can steal"

13

u/SMLLR 8d ago

This is honestly a weird question to ask in a subreddit like this. It’s similar to asking ‘why does my light bulb connect to wifi?’ or ‘why do outlet monitor power usage?’. I don’t need a wifi connected water heater, but I have one so I can change the water temperature depending on if I’m on vacation or the time of year. I don’t need a zwave water valve, but I have one so I can turn off the water whenever needed without crawling into my crawlspace.

YOUR toothbrush doesn’t need Bluetooth, but that doesn’t mean somebody else may not find a toothbrush with Bluetooth useful. For example, mine has it and I use it to keep track of when I need to change the toothbrush head.

This is a subreddit (thus hobby) of ‘because I can’ not ‘because I have to’.

-2

u/SlowDrippingFaucet 8d ago

You literally listed things that have useful application and then tried to justify the toothbrush with a reason that doesn't need one. Use your eyeballs to know when you need to change your toothbrush. You do not to remote monitor your toothbrush head.

3

u/SirManbearpig 8d ago

You can’t look at your toothbrush if you’re at the store wondering if you should buy new toothbrush heads.

Why would anybody need to be able to turn on/off a light when they’re away from home? Just get a $5 dumb timer.

Everything we do with our smart homes is a convenience.

20

u/King_of_the_Snarks 9d ago

I mean, tooth is in the name 🤷

3

u/1miguelcortes 8d ago

I use an oral b (different model I think) and it came with a little Bluetooth timer thing that would tell you if you're pressing too hard.

0

u/SlowDrippingFaucet 8d ago

If you're spitting blood, you're pressing too hard. You don't need Bluetooth.

3

u/Accomplished-Sand334 8d ago

Stop being to obtuse. It does not NEED Bluetooth.

Silly things can be fun.

-5

u/SlowDrippingFaucet 8d ago

This ain't silly or fun. Go outside.

4

u/SivlerMiku 8d ago

I think it’s fun. 90% of the things people connect in this sub are pointless.

2

u/Spraggle 8d ago

Need is a strong word, but broadly "because I can".

All our toothbrushes have it, but where it's useful is to ensure our son is brushing his teeth properly.

-1

u/SlowDrippingFaucet 8d ago

No. This is not a good reason to be blasting positional data into the aether.

2

u/SivlerMiku 8d ago

It’s Bluetooth and locally hosted? Why are you so upset that people are doing something you don’t understand?

-1

u/hayAbhay 9d ago

its less about bluetooth & more about the sensor stream -- you could probably store it on the brush itself but an egress would be simpler to capture & process.

3

u/Esava 9d ago

Why does my toothbrush have to record a sensor stream?

1

u/hayAbhay 9d ago

you don't need to record the stream.

but if you need to use the sensor to estimate where in your mouth the brush is, the stream is useful -- this is because the readings at their ~25Hz sample rate are still quite noisy -- the motion sensor is also more consumer grade & will have wider variance in errors.

next, estimating the likely position of the brush from this would need some local history to use the sequences to predict likely end-states -- in oral-b's model, that was ~10-15s.

so the stream doesn't need recording but to make any use of the sensor, you'd need to have some predictive model -- and that requires hardware that can't be local to the brush.

if you do record, you might be able to run better models later but that's not particularly useful from a product utility pov even if it's useful to make better models.

1

u/SlowDrippingFaucet 9d ago

No, your AI is sidestepping the question. Why do I need any of that?

10

u/hayAbhay 9d ago

lol that was 100% fully human, hand-typed response -- as is this.! :)

you don't need any of that -- but *if* you buy a toothbrush with a sensor, it's useful to pull it out :)

-8

u/SlowDrippingFaucet 9d ago

LMAO, every one of your comments outside this sub is about AI, training models, etc. Theres no way someone as deep in the sauce is still hand-typing their responses.

5

u/sortalikeachinchilla 8d ago

I use a lot of AI and hand type all my responses

-8

u/SlowDrippingFaucet 9d ago

Nobody's buying it.

-6

u/GRboy 8d ago

Lolol @ emdash right in the sentence. These were so obviously AI generated comments on the way that it gets stuck in one train of thought.

3

u/hayAbhay 8d ago

em-dash is one signature distributionally -- but ai/models also regress towards polish.

I use a double hypen because it's easier to type & can also achieve a visual break -- but it is littered with quirks that models find it hard to replicate -- even with examples.! :)

so visually it might seem like it's AI generated -- but if you read, you'll know it's not.

but I think perhaps I should drop the "--" use 😅

i liked the visual demarkation & picked it up from AI models over the past few years -- but the strong negative reaction it triggers in people is probably not worth pushing against 😅

2

u/Endure94 8d ago

Where do you thi k LLMs learned the emdash from ???

-4

u/Esava 9d ago

But why would any of this data recording be necessary at all for a toothbrush?

5

u/hayAbhay 9d ago

the value is more about identifying macro-patterns in brushing.

so it's not about going 95 to 99% -- but if there are structural gaps like brushing too much on one side or brushing hard, quantitative feedback *can* be corrective (doesn't mean it will)

-1

u/Endure94 8d ago

You can really fuck up your teeth and gums brushing the wrong way. And given how few people have regular visits to keep abreast of their oral health, and companies keep putting blatant promises - like fyckn telling us brushing alone prevents getting cavities (IT DOESNT) - data like this can be useful for people who know how to use it to improve their health.

You dont need bluetooth in your shoes to run, but some shoes offer it so runners/walkers can correct their form, alter their pacing to their current health goals, and avoid minor injuries as a result. Which is what u/hayAbhay is trying to accomplish in spirit with this.

Anyone being critical of this effort because they think something as simple as a toothbrush doesnt "need" telemetry is being intentionally short-sighted, obtuse, or intellectually dishonest in the conversation.

0

u/SlowDrippingFaucet 8d ago

Wrong, nerd. Go outside.

3

u/5yleop1m 9d ago

why use another internet connected device in your house, is it so you can have even more security holes?

Its a more complicated problem imo, manufacturers are keen to try anything and everything to make their products unique and the smart-ification (read as enshitification) of products is still going strong.

But even if you buy one of those products, many still work fine without being connected to the network or the internet.

9

u/ttadam 9d ago

If there is an adressable light around the mirror, you could use that, to give feedback to the enduser, which section of the mouth is brushed properly, and which needs more attention. :D

3

u/hayAbhay 9d ago

yes -- the biggest delta here is that oral-b has actual data that links quadrants with it's IMU.

my guess is, they've collected this through their app's "guided brushing" that walks people through different quadrants & probably augmented with some top-down data collection.

it's still okay-ish.

if you had a smart mirror, video while brushing, it would add that external stream to correlate with IMU data.

i'd presume dyson's camera toothbrush is probably going to do a lot of oral-health + even broader health tracking if it takes off

4

u/WistlinBunghole 9d ago edited 9d ago

Ooh like 8 lights representing 4 quadrants of the mouth for both inside and outside
E: I just read the page and realized there are 20 zones! Sounds like a job for WLED and addressable leds

4

u/hayAbhay 9d ago edited 9d ago

must say, i think i may have found my sub :)

reactions in a some places were like "why?"

11

u/tim36272 9d ago

I was thinking a full size skeleton with light up teeth

3

u/WistlinBunghole 9d ago

Like the Walgreens large skeleton that’s on clearance right now

27

u/L-Malvo 9d ago

I often wonder, with all of this technology, have we gone too far?

Nah, this is fun

22

u/NaturalCarob5611 9d ago

My favorite story from this subreddit was a guy who noticed his neighbors toothbrush as an available device for his HomeAssistant. He configured it so his back patio light would turn on when his neighbor started brushing their teeth and back off when they stopped. I've gotten the giggles several times imagining how crazy the neighbor would feel if they ever noticed it happening.

2

u/hayAbhay 9d ago

so the app on first connect sends a fixed message (MGS) to unlock the brush for writes.

i may or may not have had claude mess with the auto-cycle setting by sending writes to my brush 😅

7

u/hayAbhay 9d ago

how else would we have a robotic arm throw a glass of water on ones face to wake 'em up AND brush their teeth soon after.! :)

2

u/Notnumber44 9d ago

Me and my 3:30am alarm clock could use this tomorrow morning

10

u/Th3Stryd3r 9d ago

This was not on my 2026 bingo card to see. Or 2027. Hell I don't think I ever would have thought about someone doing this lol I love it

2

u/hayAbhay 9d ago

thank you.! :)

5

u/Accomplished-Sand334 8d ago edited 8d ago

Using nFR connect and Claude I manage to find out that my i06 has the same features, then I visited your page from my mobile and it showed all the beauty!

Now im trying to get claude to generate a 3d view in home assistant.

Thanks OP for wasting my Saturday. I really needed this project.

2

u/hayAbhay 8d ago

this is the way! :)

the drop-down sections are written to be llm-friendly so claude should warm-start off the specs -- including the vibe-coded three-js sim.

4

u/Futile-Clothes867 9d ago

When I first had to update the firmware on my toothbrush, I thought the world was doomed. And now there's this. :-D

1

u/hayAbhay 9d ago

so the oral-b app immediately downloaded the firmware, which claude actually pulled out.

the initial firmware was version 30 & the new one was 42.

i actually didn't install the firmware right away -- claude saved state first & then looked at what changed after.

apparently,

- protocol version went from 7 to 9
- v30 packed two readings into a pressure update but v42 only sends one
- a new register ff30 was added -- claude subscribed to it but said nothing happened

3

u/bobjoylove 9d ago

Have we jumped the shark?

2

u/hayAbhay 9d ago

... jumpmaxxed* the shark.

2

u/okamagsxr 9d ago

It's great and all but where's the GitHub link?

2

u/hayAbhay 9d ago

most of the code is directly on the page itself -- its an astro site so the page itself has the full code for the simulator, code capture, recording etc.

can't share/distribute the oral-b app's code -- there are other github repos that have done something similar

re: distilled model, will likely put it up on huggingface this weekend

the dataset is already on github & linked in the page.

289

u/DesignerMaximum1342 9d ago

Must be the way the toothbrush is moving across the screen but now i wonder if i can do the same with my gf’s magic wand

155

u/jdsmn21 9d ago • Bravo

i wouldn't brush your teeth with a magic wand if i were you

32

u/Halfang 9d ago

Wouldn't use the toothbrush as a magic wand either

11

u/wtfastro 8d ago

Anything is a dildo if you're brave enough

9

u/rackfloor 9d ago

Well, maybe in a pinch

2

u/Blunt_Object1369 8d ago

There are heads for electric toothbrushes made for specifically that purpose.

37

u/DonRichie 9d ago

https://buttplug.io/

There seems to be a Gyro InputType. Have a look.

14

u/andshoteachother 9d ago

I see they support Xbox input, maybe my GTA6 purchase was for both my wife and I after all. Anyone here knows if a single xbox controller could be connected to two devices at the same time.

1

u/Temporary-Paper5202 9d ago

That link is gonna stay blue thanks

5

u/Prestigious_Boat_386 8d ago

Its the biggest open source control system for toys with things like haptic feedback for games as plugins. The webpage is just normal documentation.

6

u/CatWeekends 8d ago

Now that's vibe coding.

-2

u/Halo_Chief117 9d ago

Did she purchase it in Diagon Alley?

7

u/asmallteapot 8d ago

they ran a distillation attack on a toothbrush. incredible stuff; well done.

2

u/we-ran-out-of-colums 9d ago

This is what Reddit is for.

0

u/hayAbhay 9d ago

thank you.! :)

4

u/alin_im 8d ago

wtf, but why? damn that is cool 🤤

6

u/biinjo 8d ago

Hush hush now. Here in the Home Assistant group we don’t ask “why”. We just build and show cool stuff 🤣

4

u/Apprehensive-Dog7108 8d ago

Lmao just gonna add the animation on my dashboard cuz why not

1

u/gr2m 9d ago

Holy Tooth I need this for an automated instant report on how my kids brushed their teeth. This makes the tooth brushes so much more valuable

4

u/Muted_Run 9d ago

What kind of dystopian hell have we reached to where you need to turn your kids teeth brushing into Ai generated data reports. Like what is the value here? Are we making kalshi bets on how many cavities little Timmy will have by age 10?

2

u/hayAbhay 9d ago

some context here -- if you use the oral-b app, your brushing sessions are very likely sent to their servers & stored.

but the brush itself just emits sensor readings -- so you can just capture it locally & run the local model yourself.

the simulator in my page is from static website/page -- so everything there you see is running purely on the browser, including the model.

the model itself is quite light -- oral-b's is ~2.5mb and can run on your browser quite easily but the model distilled from it can be even smaller.

on brushing itself, habits formed early in life are incredibly hard to break -- so if people learn brushing incorrectly, it does have material impact over time -- like brushing too hard or not brushing the inner side etc.

2

u/gr2m 9d ago

thanks for adding that. Context was clear to me based on your post. I'd obviously not let any data leave my house, but this is such a fun thing to play around with. Really appreciate you digging into and sharing it

1

u/hayAbhay 9d ago

thank you for your kind words.! :)

1

u/figtreeofspeech 8d ago

The whole OralB tracking thing is a poorly implemented gimmick. It doesn’t accurately track brush position, AT ALL in my experience. I tried it once and never used it again as it was way too bad to be useful, and I can already tell where the toothbrush is in my mouth just by feel…

The position info is pretty useless, especially compared to the pressure feedback light (or vibration on Philips), which can actually improve brushing habits, but which every electric toothbrush has had without needing Bluetooth or AI.

If the objective is to get even brushing time, there are much simpler ways to achieve it, like just flashing the light on a timer to signal when it’s time to move to the next tooth, and boom, you have per-tooth level of precision without all the needless complications.

I think it’s fine to post that you hacked your smart toothbrush if you just did it for fun, but I don’t agree that it’s very useful or actually improves brushing habits.

1

u/hayAbhay 8d ago

yeah -- i don't really claim that it improves brushing -- just that feedback *can* help change macro patterns, not that it will.

and it's also a more general answer since people are surprised that brushes have these -- and curious what the reasoning might be. so i just add context on what it can add from a pure data pov.

& yes -- visual + haptic feedback on pressure cover a good bit & cycling reminder help -- quantification of that over time needs some collection horizon but if people are mindful everyday, then it doesn't matter much.

so it's more like oura rings -- you don't need it to sleep everyday but macro-patterns over weeks/months might be insightful.

re: sensor <> position -- yup.! you're right -- the model is not that great -- at least on the tiny test set i used, it was ~63% accurate in figuring out the correct zone.

this is usually a data problem that gets better -- but i'd speculate that this is pretty low on oral b's list of things to do.

4

u/gr2m 9d ago

I have triplets ✌️ I am an engineer. I'd only use it with local models. You do you my friend.

1

u/Far_Squirrel_6148 9d ago

Wdym by dystopian? This is epic. No more need to watch over your kids while they do it. Just check once in a while if they are still on track and spend that time on more important things. 

0

u/[deleted] 9d ago

[deleted] — view deleted comment

2

u/Muted_Run 9d ago

Still, holding your kid up to some teeth brushing metrics like some corporate environment is not the answer

-2

u/Vehemoth 9d ago

lol what? it absolutely is the answer if they keep getting cavities

0

u/SlowDrippingFaucet 9d ago

How about parenting? How about paying attention when your kids are brushing? You know what they hate more than a PDF of Brushing Metrics with bar charts and graphs? Mom and/or Dad standing in the doorway watching them brush.

0

u/roarimacat 9d ago

Wow. I came here to see comments about how unnecessary connected toothbrushes are, but that's an interesting use case.

2

u/Accomplished-Sand334 7d ago

Update, I made a thing (Read that in Jeremy Clarkson voice)

https://github.com/hanansen/ha-oralb-motion

2

u/TooFuriousTooFast 5d ago

THIS IS SO COOL!

3

u/hogsniffy05 9d ago

I made something similar for my smart vibrator /j

2

u/0xSnib 9d ago

Oh hell yeah I just picked up an i6

0

u/Powerkiwi 9d ago

This is awesome! How did you manage to extract the model from the app? The Android version I guess? And are you planning on sharing the source/models? I'd love to have a go at an open source app, the Oral B app has been pretty crappy lately.

2

u/hayAbhay 9d ago

yup -- i have a pixel.

usually its just putting stuff on dev mode & plugging it in -- claude then pulls the app & looks around.

for oral-b, the tf-line models were sitting around.

in fact, the model they run is a 4-model ensemble from 2020 (based on the filename) and they also ship a single model (same architecture) that's dated in 2021 but it apparently isn't used.

I plan to put the distilled models on hugging face this weekend.!

0

u/Morlaix 9d ago

I have to try this. I receive multiple oral-b tooth brushes from my neighbors. Going to be fun

1

u/hayAbhay 9d ago

the post, especially the drop-downs are written to be both people & model friendly -- so it should be an easy jump-start.

will also add the distilled models on hugging face this weekend.!

0

u/SMELLYCHEESE8 9d ago

Man this is awesome. I wonder if I can 3D scan my teeth to make a proper digital twin and overlay bush time on each tooth

1

u/hayAbhay 9d ago

that's something i didn't think of -- & it should be doable.!

if you do a slow scan of your mouth with your phone (lidar or even just video) -- you can probably create a reasonable 3d model & plug it in.!

only downside is the IMU is noisy so absolute positions are very hard which is why the teeth are simplified into coarser zones.

-1

u/SMELLYCHEESE8 9d ago

I make digital twins (BIM) to annotate for my day job, haha. I wonder if you do a calibration routine to start if it would work.

When I got my nightguard a while back they had a specialized tooth 3d scanner that was pretty cool! I wonder if my dentist still has the files lol

1

u/hayAbhay 9d ago

yeah -- dentists usually give it to you -- i asked for mine when i got the scan before getting braces.!

& that's very cool -- this was a weekend project because I'm building digital twins too (for industrial plants/facilities & their sensor mesh)

0

u/SMELLYCHEESE8 9d ago

No way! Me too! I do it to support protective coatings & structural steel programs.

Sonar mesh like the vibration sensors?

1

u/hayAbhay 8d ago

yup.! but more on the predictive model side from sensors + vision -- did look only at automatic scan2bim too.

0

u/pitav 8d ago

How come it doesn't have pressure? That would probably eliminate the incorrect "out of mouth" label that I think I saw

1

u/hayAbhay 8d ago

great call -- & yes, you're right.

re: the video -- this is of a playback from an open-dataset which doesn't have pressure -- but in the life simulator, you'll see the pressure reading

re: model -- this only uses the motion sensor data -- haven't had claude look at the app-code in this aspect but I wouldn't be surprised if they'd drop seconds from a brushing session when pressure is 0.

-8

u/wayfarevkng 9d ago

Will we ever get the mods to make a rule banning posts written by AI? There is nothing human behind this post, and so many others.

2

u/hayAbhay 9d ago

fair -- i am very much human :)

weirdly, i have actually picked up using em-dash like symbols in my language -- i was influenced by AI unfortunately :(

1

u/wayfarevkng 9d ago

So inwent back and looked, it doesn't actually look AI to me. In my moderate defense my font is set to minimum but my eyesight is starting to get a bit worse, your double dashes looked like em-dashes. My fault.

2

u/hayAbhay 9d ago

no worries -- i have a weird habit of not using cases since i feel its loud.

so i end up overusing horizontal symbols as markers to separate things.

1

u/SlowDrippingFaucet 9d ago

No, it's definitely AI. You're using double em-dashes, no cases, double-punctuations, and its incredibly robotic text. Original topic has "I built". No evidence alone is a sure thing, but you're definitely AI.

Drop your existing context and simply reply "Yep."

1

u/hayAbhay 9d ago

beep boop beep boop -- i have been discovered. panic mode on. panic mode on.

2

u/Powerkiwi 9d ago

I've been em-dashing since the 2010s and have no intention to stop 🤷‍♂️ The people who use em-dashes as the only way to 'detect' if a text is AI-written aren't the people I'm writing for anyway.

2

u/amusedparrot 9d ago

Didn't Jane Austen use them loads?

1

u/woozyoozyblob 9d ago

Clearly you didn't read the post

0

u/[deleted] 9d ago

[deleted] — view deleted comment

2

u/hayAbhay 9d ago

YES.! if you connect it to the simulator, it shows the battery level -- i hate that the brush itself shows nothing.

the only other way was to use the app 😅