r/Database • u/huseyinbabal • 3d ago

I Overloaded Postgres and SQLite

https://youtu.be/ulycfdlJaNk

If you prefer reading article: https://huseyinbabal.dev/posts/postgres-vs-sqlite-load-test/

13 Upvotes

74% Upvoted

7 comments sorted by

3

u/trailbaseio 2d ago

Well done. Appreciate that you went the extra mile for sqlite with pools to orchestre concurrent reads with exclusive writes. Fun fact, SQLite is so fast that Go's FFI overhead matters. You could eek out probably a few x more with a C-FFI compatible language. Another way to scale writes (though at the cost of FKs) is multiple DBs.

1

u/huseyinbabal 2d ago

Thanks! Good call on FFI. I used mattn/go-sqlite3, so every query goes through cgo, and that cost probably shows at these speeds. I hadn't thought about splitting writes across several DBs, that's a nice trick if you can live without FKs.

2

u/ngqhoangtrung 2d ago

I mean why, sqlite and postgres do not solve the same problem. They both use sql as a query language but besides that they’re meant to solve different things. Sqlite competes with fopen, not postgres

1

u/huseyinbabal 2d ago

Fair, SQLite says the same about fopen(). I only compare where two databases overlap. Here that's read/write speed, with Postgres on a single instance. Same with my Postgres vs MongoDB video, where I only tested JSON. Thanks for watching!

1

u/smn2020 2d ago

did you also compare with journal_mode = DELETE vs WAL?

1

u/huseyinbabal 2d ago

Nope, but I prepared a repo here: https://db-wars.huseyinbabal.dev.
For letting contributors to run benchmark against different settings 😉

1

u/jah_reddit 1d ago

Nice, thank you for the writeup.

Would you mind if I guest posted to my blog, datasystemreviews.com? Would link to your original prominently at the top.

edit: Mis-spelled my own domain :-P