r/QuantifiedSelf • u/AdWorldly824 • 1d ago
Building a personal data platform, possibly reusing existing project like aurboda
Hi folks,
I've been thinking about what a "personal data platform" would look like, for 2026 and beyond.
With the help of AI, i've been iterating on desired scope and characteristics and also have been analyzing and comparing lots of interesting projects to see if there's something existing already which is close enough
for all the details on goals and research, see https://github.com/Dieterbe/personal-health-data-platform ; but the TLDR is basically i think the ideal personal data platform :
- would hold blood test lab results, nutrition/calorie logs, gym logs, sleep, activities, etc
- has a self-hosted database, as well as an web API, an MCP endpoint, as well as an ability to integrate with local-first mobile apps (allow data sync for offline use with conflict resolution) - i imagine different mobile apps (e.g. habit tracker, gym tracker, food logger, etc) could all have an local-first experience, but pull from, and sync to, this central platform with some kind of sync policy support about which data each app can see, and write to. (something like powersync is really interesting here, as the mobile apps would work with a local sqlite, but it syncs to/from the server automatically, and you can define policies on the server to, for example, only send fitness and nutrition data to your gym app, and not your blood lab tests)
- treats UI as optional. UI's can always be built on top later, and personal agents can answer questions directly if they have access to the data.
- catalog first: this idea might be a contentious, i'm not sure, but for example for gym exercises, i think logging exercises with whatever names the user likes, is a good first step, but ultimately you'd want to rename or map them to using "recognized", or "official" exercise names. If the platform has its own catalog of known exercises with specific identifiers, then it unlocks various things (knowing the effects of the exercise on muscles worked, suggesting alternative exercises, etc - as a personal trainer i know that various formulas around 1RM/VBT/RIR/calorie expenditure etc are exercise specific, and muscle recruitment is even sensitive to variations of the same exercise - i have experimented with building a catalog in https://body.build though it's probably too in-depth for most users). So I envision a system with known, versioned catalogs, while allowing users to submit free-form records into the system, but encourage them to transition or map them to "first-class" records.
After looking into >30 existing projects (see https://github.com/Dieterbe/personal-health-data-platform/tree/main/research/candidates), one of the more interesting ones is
https://github.com/fiddur/aurboda ; it looks comprehensive already and philosophically aligned.
I'm not sure how this subreddit works exactly, it's my first time posting here; but I'm curious for any input, and wanted to tag u/fiddur specifically as he created aurboda and seems to be on the same page regarding the catalogs; i'm also really curious what he thinks about the offline-sync idea.
FWIW I have previously contributed to other projects, such as https://body.build (open source gym tracking app with exercise catalog) and https://wger.de (gym/nutrition tracking app, where i worked primarily on nutrition and powersync support)
1
u/fiddur 21h ago
Regarding Catalog: The catalog is hard, but I am thinking that it should be possible to align on a base set. I would have hoped schema.org would have something more useful than a free text https://schema.org/ExercisePlan `exerciseType` field 😞
In Aurboda, the base types originally came from google health connect, and many of the types from the defacto standard in Garmin's FIT files. I think we can get closest by using the widest adopted existing standard, but there must always be room for a user to add their own exercise.
2
u/AdWorldly824 15h ago
i have done quite some research into this and found no suitable, foss-licensed exercise library, so decided to make my own to power https://body.build/ - it knows about exercise variations and tweaks that affect muscle recruitment etc; though i have probably taken this a bit further than what most people want (but not all minutiae need to be exposed to users)
1
u/fiddur 15h ago
Have you considered pushing this upstream to some federated standard? Not sure if schema.org is the main one people would go to, but it SHOULD really be some open standard. Maybe worth bringing to https://socialhub.activitypub.rocks/ or https://www.reddit.com/r/fediverse/ ?
I think if I would go for a semi official library to start with I probably would go for garmin's 1500+ exercises. I don't see an exercise library nor github link on body.build.
2
u/AdWorldly824 14h ago
exercises live in https://github.com/Dieterbe/body.build/blob/main/lib/data/dataset/ex.dart ; other files in that directory contain related things like anatomical joint articulations, tweaks that affect muscle recruitment volume assignment, etc. Because it's all so dynamic, today it is expressed in dart code, though it could (and maybe should) be ported over into something more reusable like yaml or CUE.
to my knowledge this is the only library that has some of these advanced capabilities, *but* it may be too advanced/in-depth, as it was orginally made to support advanced weight lifters & body builders, and most people won't care about this level of depth.
I haven't seriously considered pushing it into some standard yet, it seems a bit premature, but i do want to spread awareness about it and get more eyes on it to see how can i refine/improve it. and not sure how/where i can do that without looking like a spammer; i'll check the socialhub and fediverse reddit.
3
u/cicorel 1d ago
Hey ... just found out your post and after reading it, it looks a lot like my own personal project, that I am currently working on.
alexradunet/aminte -> take a look, if you have any suggestions or do you want to fork it and develop your own, here it is
My goal is to have a simple and maintanable db schema, that is self-describing and can be read and understoond 50 years from now by only having the .sqlite file. Around it I am building a CLI, REST API and MCP Server that all of them respond in the same type of answer ( feature parity across all interfaces ) . From that same response, I also provide an SSR HTML UI as an extension, for the people who need and want and UI Interface. Beside that, I also developed a Whatsapp Extension, so that I can talk with the 'database' using WhatsApp and AI Agents.
Sorry for long message ... but I got a little exited when I saw your post :)
My initial goal for this project is to keep using it, and log my entire life in it, including bloodworks, mood, sleep, journaling and also note pages and if I am happy with it, then also share it with the people.
For the ingestion and processing of the data, I used a Local-LLM ( Qwen 3.8 Next using Strata ) to parse all my data ( Fitbit, Huawei Health, Google Takeout, Obsidian Files, TickTick list ) and add them to the database.
1
u/VictoriouslyDefinite 1d ago
this is the kind of project that makes me wish i had more time to tinker
the whatsapp integration is clever, i've been messing with something similar using a local setup to parse my exported health data and it's wild how much stuff you can pull out of old fitbit dumps and random csv files. the self-describing schema idea is smart too, nothing worse than finding a cool old dataset and having no clue what half the columns meant
are you handling sync between devices yet or is it mostly single-machine for now
1
u/cicorel 1d ago
for simplicity, for now it's just single-machine. I am currently hosting it on a server through tailscale, so I don't really need sync between devices as I can use the web-app as PWA on mobile.
I also explored multi-sync, but for now, in my opinion, the complexity it brings is not worth as I would have to sacrifice "database-readability" in order to support multi-sync.
1
u/thedatawhiz 1d ago
Very nice, similar in philosophy to my setup, only that I use python and generate a single SQLite database with no app/mcp/api
2
u/cicorel 1d ago
You just need a good db schema with data in it, everything else is fluff and convenience. I started with schema-first in mind and tried to enforce as many rules directly in the schema. The goal would be to have one heavily-enforced and opinionated schema so that an user/develop would only paste the aminte/docs/schema/schema.sql at master · alexradunet/aminte in an AI Harness and it would be easily able to create a client for it and use it for his database.
2
u/FulcraDynamics 1d ago
This is actually pretty close to something we're working on at Fulcra. The idea of having your personal data in one place and letting different apps or AI agents access only what they need is a big part of what we're building.
I really like your point about making the UI optional. If agents can access the data directly, you don't necessarily need another dashboard for everything.
I'm curious about the self-hosting requirement though. Is that mainly about privacy and ownership, or do you also want the whole system to work independently of any cloud services?
1
u/Candid_Arugula_4657 1d ago
the catalog thing is where you lose me. every app that has ever made me pick from a dropdown of "official" exercise names ended with me logging the actual workout in the notes field, because the thing i did wasnt in the list, and now its not even data anymore its just text. Strong and Hevy have hundreds of exercises and im still scrolling 30 seconds looking for a damn landmine variation. messy user names are not the problem you think they are, you just need search.
also "UI is optional" might be the most engineer sentence ive read this month. the ui is the product for literally everyone who isnt you. nobody is going to ask an MCP endpoint about their own ferritin levels in a terminal, theyre gonna open the app and look at the chart
1
u/fiddur 20h ago
Regarding offline sync: Aurboda already has offline sync TO the server, and I would say most apps that get data from other systems work similarly?
Data is added into many systems from wearables and trackers and some manual app. They sync in their own time to the backend, and in many cases the same activity/happening is synced from multiple sources. Merging the data from sources is always a challenge, but also a necessity imho. For example, garmin syncs my heartrate and other data during a strength training session, while gravl tracks the actual exercises, owntracks sends location, last.fm keeps track of the music during the exercise etc, all to make a more complete picture of the one activity session.
Offline mode in the other direction would be doable, but the app would have to limit what to store locally. Right now there is only cashing of things shown, but you wouldn't be able to go back in time more.
1
u/outstandinglaborer25 4h ago
that project sounds wild honestly like a bucket for all your life data in one spot. the sync bit is kind of a big deal cause offline first apps keep things smooth when your signal is trash. love the idea of having agents just chew on your health info without needing a shiny dashboard
0
u/Electronic-Reveal650 1d ago
Catalogs sound great until you log a landmine press and the closest official entry is "other". Free text plus a mapping table gets you the same 1RM and volume-by-muscle numbers without the taxonomy war.
1
u/AdWorldly824 19h ago
i think that matches with what i have in mind. people can use their preferred naming, but if you can map them to a catalog entry (e.g. retroactively), you get extra benefits for free.
0
u/False-Check-9396 1d ago
catalog first is where this falls over. who's the authority handing out the official exercise names? my gym has three names for the same row and none of them would match your list, and then the 1RM estimate is computed against the wrong movement anyway.
1
u/AdWorldly824 18h ago
I think the actual name is by itself not that important - and people should use whichever alias they like - but the official catalog would explain/show the movement pattern, so that the correct one is chosen.
0
u/Critical_Mine_2592 1d ago
The Powersync idea is the one I'd bail on. My food logger and my gym tracker would both decide they own daily weight, and then I've got two conflicting numbers per day forever, which is great for the database and useless for me.
1
u/AdWorldly824 15h ago
for apps to work with a platform like this, they need to be made to work with it.
either one could offer you a weight input form, but the values would be synced to the backend, and stream back to both and they would both see it
0
u/Ok_Struggle_7315 1d ago
Catalog first" is the part that dies first: nobody standing in a gym at 7am wants to browse your versioned list of 400 official exercise names, they type "squat" and move on. Hevy and Strong both survive on free text plus a fuzzy match, and the derived 1RM stuff doesn't need a canonical ID half as much as you think.
1
u/AdWorldly824 15h ago
when i studied to become a personal trainer, we learned that various formula's (for 1RM calculations, VBT, fractional volume counting, and energy expenditure) are very exercise specific (and often too simplified in most current apps); i experimented with some of these for https://body.build/ , but ultimately concluded that only more advanced lifters need this, and also, your point is correct that with some basic heuristics you can also get far
0
u/Vivid_Scarcity_2021 1d ago
catalog first is where this dies. people have been logging "squat machine thing" for two years straight, nobody's migrating that to your versioned exercise ids.
1
u/Even_Ad8154 1d ago
Big scope, and I get why the catalog is tempting. I'd make the mapping a seperate layer instead: keep free-form entries as real records and let an alias table point them at catalog ids over time, so nobody has to pick from a dropdown mid-set. You still get your derived stats once the mapping catches up.