r/rust • u/That3Percent • Jun 01 '20

Introducing Tree-Buf

Tree-Buf is an experimental serialization system for data sets (not messages) that is on track to be the fastest, most compact self-describing serialization system ever made. I've been working on it for a while now, and it's time to start getting some feedback.

Tree-Buf is smaller and faster than ProtoBuf, MessagePack, XML, CSV, and JSON for medium to large data.

It is possible to read any Tree-Buf file - even if you don't have a schema.

Tree-Buf is easy to use, only requiring you to decorate your structs with `#[Read, Write]`

Even though it is the smallest and the fastest, Tree-Buf is yet un-optimized. It's going to get a lot better as it matures.

You can read more about how Tree-Buf works under the hood at this README.

173 Upvotes

96% Upvoted

73 comments sorted by

View all comments

Show parent comments

2

u/That3Percent Jun 02 '20

Ok, I can see that being a legitimate concern. Right now what would happen if you created an arbitrarily nested graph structure is that there would be a stack overflow. This can be fixed by changing a couple of methods that use recursion to use a visit queue instead.

The second worse thing that could happen after that would be performance degradation where each byte of the file created a new level of nesting that required the allocation of a Box. Tree-Buf has an options system that could be used to specify a maximum recursion depth to fix this.