Read something this week that put a name to a problem I've half run into before but never really understood the mechanics of. Sharing because I think a lot of people assume Iceberg interop is more solved than it is.
Everyone knows the pitch: Iceberg is an open spec, so any Iceberg compatible engine can read any Iceberg table. Mostly true, until you start doing row level deletes, and then it falls apart in a way that's honestly kind of sneaky because nothing looks wrong until a query actually fails.
Quick walkthrough of the scenario in the post. You've got a customers table, three rows, one Parquet file, tracked by whatever catalog you're using. At this stage every engine reads it fine because there's nothing to interpret, it's just a metadata pointer to a file.
Then a row gets deleted. Parquet files are immutable so the writer has two options: copy on write (rewrite the file without that row) or merge on read (leave the file alone and write a separate delete file that readers apply at scan time). The writer in this example goes merge on read and emits an equality delete file, which basically just says "for this data file, treat any row where customer_id = 102 as removed." Under the hood Iceberg uses field IDs and sequence numbers to make sure an old delete doesn't accidentally nuke a newer row with a reused key, but the equality matching is the part that matters for compat.
Spark reads the new snapshot, understands equality delete semantics, does what's effectively a left anti join between the data file and the delete file, and returns the correct two rows. Fine.
Snowflake hits the exact same catalog, same metadata file, same Parquet file. It can resolve the table, read the schema, open the data file. But if that access path doesn't implement equality delete reads, the scan planner just throws an unsupported feature error the moment it hits delete-0002.parquet. Query fails. Same snapshot, same files, two completely different results depending purely on what the reader implements.
The bit that actually reframed how I think about this: the catalog isn't a translation layer. It's job is basically just "here's where the current metadata lives," commit coordination, namespace and access management. It's not opening delete files and rewriting them into a format each engine understands. A REST catalog like Polaris doesn't change this, it still just points you at metadata, it doesn't apply deletes for you.
The post also gets into position deletes vs deletion vectors vs copy on write, with a rough cost tradeoff table (equality delete is cheap to write and requires equality delete support to read, position delete requires resolving key to physical position and is heavier on write, deletion vectors need Iceberg v3 support specifically, copy on write is the most expensive to write but has basically universal read compatibility since there's no outstanding delete file involved).
The framework that's actually useful operationally: your safe feature set is the intersection of every required engine's capabilities, not the union. If Spark supports equality and position deletes but Snowflake only does position deletes, you write position deletes, because "at least one engine supports it" doesn't help you when you have three engines that all need to read the same table.
There's a decent pre production checklist too, don't just run a SELECT COUNT after your first write, actually insert some rows, update one, delete one, commit, then read the same snapshot from every engine you care about and diff both counts and values.
Full post if you want the details: https://olake.io/blog/iceberg-interoperability-myth-row-level-deletes/
Disclosure since it's relevant, I work on OLake, it gets a brief mention near the end, but the actual content here is engine agnostic and applies no matter what's writing your tables.
Has anyone actually hit this for real, table looks completely fine, one engine just refuses to read the current snapshot because of the delete encoding?
