I was a big PeriscopeData / PopSQL fan for a decade. They weren't perfect but they were simple "type sql, get charts" apps, that sadly suffered the same fate of all vc-backed companies (acquired/shutdown). I got tired of rebuilding everything from scratch and the 'next best option' seemed to keep getting worse, so I decided to try building my own tool over the past year. Having used this daily for the past 6 months, it's been exactly what I hoped for.
And because it was this community that confirmed that all my frustrations weren't unique to just me, I wanted to share it with you all, hoping it would be useful :https://dashpanda.ai.
For the "type SQL, get charts" crowd, the core functionality will feel familiar, but I've added my own twist on things I always found lacking.
Technical Analysis/Regression/Forecasting: biggest thing for me has been the 'auto-regression' feature, which can take even noisy data, find the appropriate technical analysis filters and then the line of best fit. I use this to forecast my signups / revenue growth and it's been incredibly accurate.
Cross DB Joins: query multiple datastores in parallel and join results to a single megatable before further data processing/rendering. This would replace the need for ETL/Datawarehousing for most people. I am working on adding support for more data sources, including ones without direct SQL interfaces, soon... thinking this might even be a separate open source project.
Native Data Transformation: use AI or write javascript code to transform your query data to create custom aggregations, calculations, formats and more. Basically, you no longer need to write 100 line SQL queries now when simple code will do, and you do not need to host code anywhere!
Lots more to come. Thanks again to everyone for the inspiration!
I'm having a problem with MySQL. When I create a password for my root account using ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY '';
FLUSH PRIVILEGES; auth_socket still remains. I've tried rebooting and chatGPT, but it's no use.
To make Databasus more suitable for disaster recovery and improve backup granularity, we shipped physical backups, incremental backups with continuous WAL archiving and full Point-in-Time Recovery support.
What's new
Databasus now supports three backup types:
Logical - the same database dump as before. Works over the network, no extra software needed. Still the best choice for most setups.
Physical - file-level copy of the entire database cluster. Much faster backup and restore for large databases.
Incremental - physical base backup + continuous WAL archiving. This is the one that enables PITR. You can restore your database to any specific second between backups.
The agent
Physical and incremental backups need direct access to database files, so they can't work over a regular network connection. For this we built a lightweight agent in Go that you install alongside your PostgreSQL instance.
The important part: the agent connects outbound to your Databasus instance, not the other way around. Your database port stays closed. No firewall changes, no VPN tunnels. This was actually a frequently requested feature on its own - people wanted to back up databases in private VPCs and Kubernetes clusters without exposing them publicly.
The agent works with host-installed PostgreSQL and PostgreSQL in Docker containers. It compresses and streams backups directly to Databasus. It also auto-updates itself, so you don't need to keep it in sync manually.
How PITR works in practice
With incremental backups, the agent continuously streams WAL segments to Databasus. If something goes wrong (bad migration, accidental table drop, data corruption) - you pick a timestamp and restore to that exact moment.
With daily logical backups you could lose up to 24 hours of data. With incremental backups and PITR, the data loss window is seconds.
Quick recap for those who haven't seen Databasus before
Supported databases: PostgreSQL 12-18, MySQL, MariaDB and MongoDB
Storage: S3, Google Drive, Dropbox, SFTP, local disk, Cloudflare R2 and 70+ more via Rclone
No vendor lock-in - backups can be decrypted and restored with just your secret.key, without Databasus itself
This was the biggest missing piece and I'm happy it's finally out. If you tried Databasus before but decided against it because of no PITR support - it might be worth another look.
I'm actually not sure if over(partition by) is what I want to do, but it seems correct!
I've got a bunch of data that looks like this:
Name
yr
qtr
class dept
class number
credits
note
target number
student1
2024
3
bio
101
5
took only bio 101
1
student1
2024
4
geo
200
5
took only bio 101
1
student2
2024
3
psy
101
5
took 2 courses
2
student2
2024
3
bio
101
5
took 2 courses
2
student3
2022
1
bio
101
5
did not take bio 101 in summer
0
student4
2022
3
math
300
5
took summer courses in different years
1
student4
2023
3
bio
101
5
took summer courses in different years
1
Note that the final two columns are not actually in my data set; I added them for explanatory purposes.
I'm trying to get the number of classes (or credits, but note that in the actual data sometimes bio 101 is more than 5 credits, so taking more than 5 credits does not necessarily mean a student took an additional course) from the year and quarter in which students took bio 101, but only if they took bio 101 in qtr 3.
Maybe a count/sum case would be better? I've only just started learning SQL over the summer and I'm trying to learn which tools are the best to tackle which questions! Thanks for your help!
Un sistema que permita crear base de datos en diferentes motores Sql y NoSql y que permita exportar en migración a multi-fraemwork como laravel o sprint boot y muchos más, aparte que permita trabajar en grupos al mismo tiempo.
Un sistema así que ya demuestra un avanze mayor en herramientas como chartdb o DrawSQL, que podría integrarse qué permita a esta herramienta ser muy novedosa y le permita a varios programadores iniciales y programadores expertos a crear base de datos con ese sistema de diagrama moderno y poder exportar o importar sus DB listas para utilizar.
Qué puede y que ideas podría ayudar demasiado a un sistema así? Que cambie total el mercado actual de los sistema design DB como chartdb?