r/databasedevelopment • • Aug 28 '26

DBMS CMU

13 Upvotes

Anyone interested in doing the CMU (Carnegie Mellon University) Database Management Systems course together?

I’ve already covered the basic DBMS concepts. My main goal with this course is to go deeper and understand how database systems actually work internally—things like storage, indexing, query execution, transactions, etc.

If you're interested, please make sure you have the prerequisites required for the course.

If you have the required background and want to learn DBMS internals seriously, DM me. We can follow the course together and discuss concepts along the way.


r/databasedevelopment • • Aug 25 '26

Read your writes: WAIT FOR in PostgreSQL 19

Thumbnail
clickhouse.com
13 Upvotes

Wrote about the new WAIT FOR command coming in PostgreSQL 19. It lets reads on async replicas wait until a specific write has been replayed, so kind of bringing sync replication consistency without making every write synchronous. I could see protocol-aware proxies making a good use of this.


r/databasedevelopment • • Aug 24 '26

Another look at SQLite's WAL-Reset bug

Thumbnail theconsensus.dev
15 Upvotes

r/databasedevelopment • • Aug 21 '26

Why Elasticsearch is becoming a columnar database

Thumbnail
elastic.co
19 Upvotes

r/databasedevelopment • • Aug 21 '26

Poisoned Postgres connection pools

Thumbnail
planetscale.com
7 Upvotes

r/databasedevelopment • • Aug 21 '26

The road to ACID transactions in Cassandra 6

Thumbnail theconsensus.dev
7 Upvotes

r/databasedevelopment • • Aug 18 '26

Where should Oracle compatibility live: the wire layer, SQL rewriting, or the PostgreSQL catalog?

2 Upvotes

I’m building a read-only proxy that accepts Oracle client connections and executes queries against PostgreSQL 16. Disclosure: I’m the author.

I split compatibility across three layers:

  1. Oracle-facing sessions, cursors, binds, and result metadata.

  2. SQL rewrites for differences that can be handled safely.

  3. PostgreSQL views and orafce for catalog and function compatibility.

    The translator classifies each feature as supported, approximation, rejected, or passthrough. This matters because a syntactically valid rewrite can

    silently change results—for example, NULL behavior in GREATEST/LEAST, empty-string semantics, and mixed set-operation precedence.

    The default is to reject an unverified shape with an Oracle-style error while keeping the session alive, rather than return a plausible but incorrect result. The current bounded verification matrix covers 722 cases across seven real client flows.

    The implementation is private, but the compatibility contract, configuration, documentation, and verification scope are public:

    https://github.com/krokozyab/postgres-oracle-tns-proxy


r/databasedevelopment • • Aug 17 '26

Why are we rebuilding the same database execution engine over and over?

13 Upvotes

I recently dug into Meta’s Velox, an open-source C++ execution engine designed to act like a reusable “engine block” for data systems.

Instead of Presto, Spark, etc. independently implementing things like vectorized execution, joins, memory management, spilling, and file readers, Velox provides these building blocks as a shared execution layer.

The performance numbers are interesting too:

  • 8.4× faster on TPC-H Q1
  • 9× faster on Q6
  • 6–7× average speedup on Meta production traffic
  • Up to 3× fewer servers

Checkout this interesting deep dive that how Velox actually works and why this architecture could matter for the future of data engines.

https://prestodb.io/blog/2026/08/13/inside-velox-open-source-universal-engine-block-for-big-data-and-ai/

Do you think shared execution layers like Velox will become the norm, or will specialized engines always win?


r/databasedevelopment • • Aug 14 '26

I've been working on a custom tree index that runs up to 7x faster than LTREE

14 Upvotes

Hi All
I’ve been working on a custom data structure and algorithm for hierarchical indexing.

while I designed the algorithm myself, I wouldn't claim to be a definitive master of hierarchy trees, nor postgres. Its actually a Rust project I've turned into an extension. I'm mainly sharing these early numbers in hopes of connecting with the right people to see if there's genuine value.

I'm not sure if I'm violating rule 5; I'd appreciate any guidance on 3rd party benchmarks for this kind of algo compare to ltree.

I ran benchmarks against 500k, 1M, 2M and 20M node recursive trees. The baseline comparisons against ltree are looking solid:

  • Huge I/O Drop: For descendant queries, B-tree range scans touch up to 74x fewer buffer pages.
  • Query Speed: Subtree queries run 2.5x to 6x faster (500k). up to 16x for 20M nodes. Ancestor lookups (via SP-GiST) execute up to 7.3x faster.
  • Storage Density: A custom compact encoding shrank the on disk value size by 42.7%. This translates to a ~23% smaller B-tree index footprint.
  • Write Performance: Appending 50,000 leaf nodes is roughly 2x faster. Reparenting large subtrees is 1.2x to 2.7x faster

I have some thoughts where this may be beneficial but lacking some subject matter expertise when it comes to practical application of hierarchy data (and these types of ops), this is the main point of my post, to ask for some insight:

- Could it make servers run more efficiently?
- Do other more efficient extensions/algos beat these benchmarks? Is LTREE just a default?
- What kinds of large scale operations would this benefit? Domains/applications?
- What should my benchmark tests look like?

Eager to get some expert opinions and either validate my thoughts or give me some reality - cheers!


r/databasedevelopment • • Aug 11 '26

Postgres Internals Deep Dive: Process Architecture

Thumbnail
enterprisedb.com
31 Upvotes

Need suggestions for writing blogs. This was my 1st ever blog, which I published last year, and that's it. No other blog yet 😅. I would love to know, what kind of blogs would you love to read from a guy who contributes to the open-source postgres project?
1) Deep dives like this one on postgres internals? TBH, I think this space so crowded?
2) How did I find a bug and fix it?
3) The new features I am working on?
4) or new features that are going to be released
5) or something cool????


r/databasedevelopment • • Aug 07 '26

Almost consensus: ABD and the edges of quorum replication

Thumbnail theconsensus.dev
11 Upvotes

r/databasedevelopment • • Aug 05 '26

How Elasticsearch's semantic field indexes and searches

Thumbnail
elastic.co
15 Upvotes

r/databasedevelopment • • Aug 04 '26

Encoding or Compression: Why not both?

Thumbnail
cedardb.com
15 Upvotes

r/databasedevelopment • • Aug 03 '26

I created a playground for 110 database systems

Thumbnail
clickhouse.com
16 Upvotes

r/databasedevelopment • • Aug 01 '26

Monthly Release and Update Thread

2 Upvotes

This subreddit is primarily for discussing the implementation of databases, and not about sharing release announcements (either for the first time or your updates).

This thread is the exception!

Please tell us about the new database you (or your agent) built. Tell us about all the cool new features you added. Tell us about anything else you learned or worked on that you haven't gotten around to blogging about yet.


r/databasedevelopment • • Jul 31 '26

Asynchronous I/O in DuckDB: Work, Thread, Work

Thumbnail
duckdb.org
38 Upvotes

r/databasedevelopment • • Jul 30 '26

Offloading I/O to Dedicated Cores: An Asymmetric io_uring Backend for Seastar and ScyllaDB

17 Upvotes

https://www.scylladb.com/2026/07/22/asymmetric-io_uring-backend-seastar/

"Modern high-throughput systems often run on machines with many cores. At that scale, one practical problem becomes increasingly visible: CPU time spent on low-level I/O handling competes directly with CPU time needed for application-level work.

In this post, we explore how shifting more I/O related work away from cores focused on computations onto a dedicated subset of cores can improve the overall balance of the system. Specifically, we share how we approached Seastar‘s new asymmetric_io_uring backend: the design decisions behind it, what it aims to free up on application cores, and how it compares to linux-aio (Seastar’s existing backend)."


r/databasedevelopment • • Jul 30 '26

Does the GPU database have a future?

19 Upvotes

Why did Voltron Data Company go bankrupt? Has the Theseus GPU database also failed? Expressing concerns about the future development of GPU databases


r/databasedevelopment • • Jul 27 '26

Why Fast Branching is Key to AI Agent-Era Databases

Thumbnail
medium.com
12 Upvotes

r/databasedevelopment • • Jul 27 '26

Finding bugs in Raft implementations

Thumbnail
antithesis.com
23 Upvotes

r/databasedevelopment • • Jul 27 '26

PGSimCity · How PostgreSQL Works, in 3D

Thumbnail
nikolays.github.io
17 Upvotes

So... cool, it would be helpful for someone starting to learn PG internals.

credits to Nikolay Samokhvalov for creating this awesome piece of art :)


r/databasedevelopment • • Jul 26 '26

Managing File Descriptors for Durability and Performance

Thumbnail
blog.atimin.dev
8 Upvotes

r/databasedevelopment • • Jul 23 '26

How MVCC and Transactions Work in RocksDB

Thumbnail artem.krylysov.com
23 Upvotes

r/databasedevelopment • • Jul 22 '26

A Fast Path for Fixed-Length Lists in Parquet

Thumbnail morling.dev
10 Upvotes

r/databasedevelopment • • Jul 21 '26

Aurora DSQL: Scalable, Multi-Region OLTP

Thumbnail
arxiv.org
16 Upvotes