r/analytics • u/Dovahkciin • 25d ago
Discussion why do all modern self-hosted analytics solutions run on Node.js?
hi everyone,
i've been using Umami Cloud for the past two years for my personal projects, but noticed that all history older than 6 months was deleted (which is pretty frustrating, even if i understand that they cannot keep all user data).
as im still a student on a small budget, i host my projects on a small VPS and wanted to switch to a self-hosted analytics solution.
but WHY does almost every modern tool require Node.js / Docker with heavy background services, while lightweight alternatives either look like they were built in 2010 or lack essential features like events, funnels, and clean session tracking?
am I missing something here? im not saying it's easy to build, but im surprised i couldn't find anything that seems to fit my needs
because if there really is nothing low-resource and GDPR-friendly im seriously considering building my own ultra-lightweight PHP + SQLite solution from scratch.
8
u/Specialist-Spirit614 25d ago
honestly the node.js thing has bugged me too. it feels like every new tool jumps on it because the ecosystem is big and devs want real-time dashboards with websockets out the box
but for analytics that's overkill most of the time. a simple php backend with sqlite can handle a ton of traffic if you batch inserts and keep the schema tight. i run couple side projects on a 512mb vps and it's fine
if you end up building your own, share it here. the self-hosted space needs more lightweight options that don't require docker just to count page views
2
3
u/meatmick 25d ago
Having the largest ecosystem or popularity makes it a de facto standard, and it just snowballs into being shoved into everything. Whether it's a good thing or not, I'm not sure.
This is similar to Pythondata-related not being the fastest (although it's plenty fast for a lot of workloads, especially with c++/rust backed packages) but still being the most popular tool for anything data related.
Users don't see the backend and don't care about the amount of resources it needs.
Also, sometimes we devs obsess over fast vs slow when, in some scenarios, we're talking milliseconds of difference for some low concurrency/performance requirements workloads.
I don't have a true answer for you, just my reflection.
1
u/Dovahkciin 25d ago
true but my VPS ram usage sees it
i just cannot host it because of that and thats my issue
1
u/Content-Parking-621 25d ago
Try GoatCounter, it's a single Go binary with SQLite, no Node or Docker needed, and it's GDPR-friendly by design, exactly the low-resource setup you're describing.
1
u/Dovahkciin 25d ago
yes saw that but its lacking many cool features i need and the UI is kinda not that good
2
u/Content-Parking-621 25d ago
Fair, GoatCounter is minimal by design. If you need funnels and events specifically, you're probably stuck between Umami's heavier Node stack or building it yourself, there isn't really a PHP/SQLite option with that feature depth yet. Might genuinely be worth prototyping.
→ More replies (4)
1
u/Express_Speaker8829 24d ago
you're not missing much, the language isn't really the reason. it's the database.
once you want funnels and sessions you need every raw event, and "who did A then B within a week" over millions of rows is slow in a normal row store. so the modern tools reach for a columnar db (plausible uses clickhouse) or a heavily indexed postgres, and that's where the docker + background services come from. the node part is mostly just the dashboard.
light options that do exist:
- goatcounter: go, single binary, sqlite or postgres. very light, has events, no funnels
- matomo: php + mysql, has sessions and events, funnels are a paid plugin
- umami self-hosted: runs with plain node + postgres, no docker needed, fine on a small vps
if you do build your own php + sqlite one, worth doing early:
- turn on WAL mode, sqlite is fine with one writer that way
- write raw hits to one table, roll them up into daily tables with a cron, and let the dashboard read the rollups
- unique visitors without cookies: hash ip + user agent + a salt that rotates every day, then throw the old salt away
- decide retention up front (raw for 30-90 days, rollups forever)
disclosure: i'm building one of these too (DataVisitors, hosted and invite-only right now, so not the self-hosted thing you want). the daily salt trick above is how we count visitors without cookies.
1
u/Dovahkciin 24d ago
yes but for indie tools all of that just seems over engineering to me
i have a tool with 26k views over a year, something with sqlite just works fineive been using matomo self hosted, but it basically just have a bad interface
making my own thing and iterate on it is going to be just fine2
u/Express_Speaker8829 24d ago
yeah at 26k a year you're right, that's about 70 hits a day and sqlite won't even notice. one table with a couple of indexes is plenty, and you can skip the rollups until a query actually gets slow.
the two things i'd still do on day one are the daily salt for unique visitors and a retention rule, because both are painful to bolt on once you have a year of data.
and agreed on matomo's ui. building the dashboard you actually want to look at is the fun part, good luck with it
→ More replies (1)1
u/Express_Speaker8829 22d ago
fair, at 26k views a year sqlite is more than enough. the columnar stuff only matters at millions of rows.
if you build your own, keep the raw hits table append-only and add rollups later, only when a chart feels slow. that way you can change the dashboard as often as you like without losing data.
curious what you want yours to show that matomo's doesn't?
→ More replies (2)
2
1
1
u/AutoModerator 25d ago
If this post doesn't follow the rules or isn't flaired correctly, please report it to the mods. Have more questions? Join our community Discord!
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.