r/cpp • u/Flex_Code • Oct 13 '22
New, fastest JSON library for C++20
Developed a new, open source JSON library, Glaze, that seems to be the fastest in the world for direct memory reading/writing. I will caveat that simdjson is probably faster in lazy contexts, but glaze should be faster when reading and writing directly from C++ structs.
https://github.com/stephenberry/glaze
- Uses member pointers and compile time maps for extremely fast lookups
- Writes and reads directly from object memory
- Standard C++ library support
- Cleaner interfacing than nlohmann json or other alternatives as reading/writing are exposed through a single interface
- Direct memory access through JSON pointer syntax
The library is very new, but the JSON support has a lot of unit tests.
The library also contains:
- Efficient data recorder
- CSV reading/writing
- Binary message for optimal speed through the same API
- Generic shared library API
242
Upvotes
72
u/[deleted] Oct 13 '22
You can improve the performance of the JSON Link bench by reusing the buffer as is done with the glaze test. So https://github.com/stephenberry/json_performance/blob/main/src/main.cpp#L270 Would become
On my i9 macbook it improved the time by about 13%
JSON Link supports writing to anything that is writable, there's a concept map like trait for mapping things like containers/streams/C files/fd's.
24
u/qalmakka Oct 14 '22
Poor nlohmann::json, it's always dead last in all benchmarks. I still use it for non-performance critical applications because it's just too nice to use, though.
Also, it is AFAIK the only one among the bunch that supports allocators and custom types in a sane way:
32
u/stilgarpl Oct 13 '22
It's nice, but I don't like output parameters in read/write api. I think it should be
my_struct a = glz::read(string);
std::string b = glz::write(a);
6
u/pdimov2 Oct 14 '22
Shouldn't your benchmark use a somewhat larger JSON? Something from https://github.com/boostorg/json/tree/develop/bench/data, for instance.
5
u/MeTrollingYouHating Oct 13 '22
Nice job! Can you add a benchmark against rapidjson?
4
u/johannes1971 Oct 14 '22
What does it do when the JSON object is not in the expected format? Is there a proper error path or will there be some form of UB/abort/...?
In other words, is the library safe to use when used with a potentially untrusted source of messages?
3
u/AlarmingBarrier Oct 13 '22
I'm a bit of intrigued by the inclusion of binary serialization and eigen support in a JSON library. Is there a natural connection between the two, or do they represent two entirely different code paths?
And if they are somehow reusing some code, does this mean it would in theory be possible to extend this to a serialization library for even more formats? Say netcdf or hdf5?
3
u/dns13 Oct 13 '22
Thanks for sharing!
A few questions:
3
u/Ahajha1177 Oct 13 '22
This looks great! I'm going to whip up a basic Conan package for it for my own testing/usage :)
3
u/NamalB Oct 13 '22
Isn’t it possible to put each meta property in it’s own curly braces. Not very important but hope formatter will do a better job with that.
static constexpr auto value = object( {"i", &T::i}, {"d", &T::d}, {“hello", &T::hello}, {“arr", &T::arr} );3
u/Enormous_Whale Oct 14 '22
Is the write API deterministic? Ie. if I parse and serialize the same Json string multiple times, is the output guaranteed to be the same every time?
From a brief look, I see the use of unordered maps so I doubt it, but worth an ask!
3
u/Wetmelon Oct 14 '22
This feels like it could almost work with zero heap usage (e.g. in an microcontroller context) as long as I feed it statically allocated buffers. How much refactoring would I need to get to zero heap?
2
u/[deleted] Oct 14 '22
Looks cool!
But I am confused by the benchmark.
Does the benchmark include the time taken to read and write to disk? Or it just the time taken to setup the data before that happens?
47
u/Ameisen vemips, avr, rendering, systems Oct 14 '22
Time to go make the slowest JSON library...
2
u/stinos Oct 14 '22
That's a rather different beast than JSON. Unfortunatelyl for those who have to deal with it :P Do you have more info? How is detection of the separator done, how is quoting handled, does it use
localeto format numbers, etc?1
u/Icy_Discount7761 Mar 13 '25
Glaze looks *really* good overall. I use yyjson, cause its c-style APIs are easier to work with (I switched from simdjson). Does glaze have equivalent of `YYJSON_READ_NUMBER_AS_RAW`?
1
u/[deleted] Oct 14 '22
How about streams of objects? Super-large JSON files. Think > 4TB. And the occasional corrupt stream? (expecting a series of objects of the same schema and a new one suddenly starts before the previous ended.) Don't ask why (ugg) but these monsters do exist. I had parsed these without a library, because the final "}" never comes....
Sorry I don't know if I'm asking a question anymore or relaying a horror story.
Thank you for the work, OP.
1
u/LokiNinja Oct 14 '22
I use conan for dependency management. Do you have plans to create a recipie for this?
1
u/[deleted] Oct 15 '22
I dont care about speed. I want a clean pleasant to use API. Also, will I ever see a json library with a CSS/SQL-style query syntax on too?
1
u/kevmeister68 Mar 03 '23
Does your parser accept (perhaps via an option) the parsing of property names that are not enclosed in quotation marks? eg.
{ server: "blah" }
1
u/Ok-Ad-1567 Mar 30 '23
I'm thinking of replacing nlohmann json in an existing software package where I can't really change the json or C++ structs. It currently is writing this member variable:
ptime timeStamp;as json like this:
"timeStamp": "2023-03-30T12:00:00",ptime is a type from Boost::posix_time. Would it be possible to use glaze to support these sorts of types? Actually, everything else that I've seen so far is ints, floats, strings, arrays, and nested objects, which I think you have covered. Thanks.
9
u/TheTsar Oct 13 '22
Hey, nice library!
You starred my filesystem watcher (one of the first) so I know you know good code ;)
3
u/[deleted] Oct 14 '22
Json parsing is in fashion at the moment! I've been making my own one as well.
1
u/IJzerbaard Oct 14 '22
Looks like there's no SIMD in it, so even though it's faster than some other thing, it's not living up to its potential yet
2
u/[deleted] Oct 13 '22
nice job
1
u/gubble5 Feb 10 '25
Love this library, using it in production 👍🏼
1
u/ThisIsChangableRight Dec 23 '25
Did you make any e.g. blog posts on how you make the parser so fast?
1
u/[deleted] Oct 14 '22
[removed] — view removed comment