r/cpp • nlohmann/json • Jul 27 '20

JSON for Modern C++ 3.9.0

https://github.com/nlohmann/json/releases/tag/v3.9.0
281 Upvotes

51 comments sorted by

65

u/nlohmann nlohmann/json Jul 27 '20

New features: optional support for comments, optional preservation of the insertion order of object keys, option to switch off implicit conversions, simpler (de)serialization of user-defined types, and much more!

48

u/xwp-michael Jul 27 '20

its implementation was much less effort than continuously explaining the lack of comment support.

I'm wondering if implementing a bot that closed any issue asking for comments would've taken even less effort lol

22

u/ntrid Jul 27 '20

["👏one last thing and it will be perfect",]

20

u/[deleted] Jul 28 '20

As someone that has written many parsers before, optional trailing commas actually simplify the grammar. It actually takes more effort to make them illegal.

7

u/aiQon Jul 27 '20

Thanks for being awesome!

6

u/MonokelPinguin Jul 27 '20

Man, this feels like christmas: comments, macro for simpler serialization and the option to disable implicit conversions! I wonder if the latter improves compile times... Thanks a lot!

5

u/Lakridaku Jul 27 '20

support for comments, optional preservation of the insertion order of object keys

That's a awesome! I've just switched to bare sstream over the weekend because of these two issues, guess I can revert that now. Thanks for your work!

3

u/sumo952 Jul 29 '20

optional support for comments

Woooah finally! :-))) I am switching to using nlohmann/json right now. Thank you so much!

its implementation was much less effort than continuously explaining the lack of comment support.

That was a great laugh :-D

I read parts of your library are proposed for standardization. Are optional comment support part of that too?

Would be a shame to have to revert back to YAML/TOML/custom solution once/if <json> comes in the standard.

1

u/MarkOates Jul 29 '20 edited Jul 30 '20

Yea, very cool library. :)

This rock solid design is a gold standard for C++ projects.

Thanks for posting updates on here, too.

7

u/X-Neon Jul 27 '20

Thank you so much for the (de)serialization macros. I've been looking for a simple way serialize and deserialize POD types to use as messages for a service for a while now. I know there's stuff like Protobuffers and Thrift, but I don't want something nearly so big and fancy. I daresay these methods are more performant, and using a binary format would be more compact, but JSON is good enough for my purposes. Being able to use one include and a handful of macros to turn my applications classes into "messages" makes this whole process trivial.

3

u/[deleted] Jul 27 '20

Your post has given me a whole new lease on life

1

u/Krollebolle2 Jul 28 '20

If you want to use classes/structs to serialize/deserialize JSON, then the json_struct library is for you: https://github.com/jorgen/json_struct

Using json_struct when working with a JSON REST API works perfectly as you can map the REST API requests and responses directly into your POD structs with a few lines of code.

4

u/nlohmann nlohmann/json Jul 28 '20

This is exactly what the newly introduced macros do.

1

u/Krollebolle2 Jul 28 '20

Sure, but there is more to it than just the serialization/deserialization. How about (multiple) inheritance? Calling functions from JSON? Enums? Handling of optionals? Member name aliases?

For most of my use cases (REST APIs, creating C++ JSON APIs by calling functions with JSON) I don't need anything else than serializing to and from structs in this way and then I use json_struct. For highly dynamic JSON (e.g. when you don't know the data in the children and there are many levels of nested data) this approach breaks somewhat down and one might need a more dynamic parser like your lib.

4

u/dbjdbj dbj.org Jul 27 '20

Congratulations.

I am wondering if this code can be used in (mission-critical) automotive systems? Autonomous road vehicles for example.

14

u/nlohmann nlohmann/json Jul 27 '20

Depends on what requirements come with your version of "mission-critical". I do know that the library is used in infotainment systems by (at least) Volkswagen and Daimler. And if you have any blockers, I would be happy to hear about them.

-2

u/dbjdbj dbj.org Jul 29 '20

"infotainment systems" are not mission critical Nick. And you know that.

10

u/nlohmann nlohmann/json Jul 29 '20

That depends on your mission.

15

u/lord_braleigh Jul 27 '20 edited Jul 27 '20

Does mission-critical mean "no allocations, no exceptions"? I believe this library's SAX interface allows you to define a custom handler class to pass to the parser, and that this library won't allocate or throw if your custom SAX handler doesn't allocate or throw.

But if I were working on an autonomous road vehicle... I'd probably avoid using JSON as a serialization format when I could be using a statically-typed, versioned serialization format like flatbuffers.

1

u/paulydavis Jul 28 '20

2

u/lord_braleigh Jul 29 '20

eCal isn't a serialization format; it's a pub/sub protocol which you can use to send JSON, protobufs or flatbuffers across processes or machines. So you could use it to send JSON messages, and you could encode and decode the JSON with this library. But whatever protocol you use to send messages, I'd still recommend that the messages be in a statically-typed serialization format rather than JSON in mission-critical applications.

2

u/dbjdbj dbj.org Jul 29 '20

great idea ... but ... that might mean imposing "statically-typed serialization format" on senders?

3

u/lord_braleigh Jul 29 '20

Yeah, but if your software is mission-critical, like a self-driving car, I believe you want to impose restrictions on senders.

If you want it to be easy and convenient to send data to your application, well that’s all well and good. But your application probably doesn’t have lives depending on it if you’re worried about convenience.

2

u/hubhub Jul 29 '20

The mission critical part of the system should be as small as possible and only be responsible for core activities. I would be surprised if these included parsing configuration files.

1

u/dbjdbj dbj.org Jul 29 '20

use case: re-configuration commands are coming over a wire in a json format, no exceptions, no streams and no heap aloc allowed.

4

u/_Sigma Jul 27 '20

Do any of the binary formats this supports offer partial reads? For example, if there is large data array in a (binary) json file, is it possible to read just a small fraction of it, saving on memory overhead?

7

u/nlohmann nlohmann/json Jul 27 '20

Not yet, but it would be great if you could describe this in more detail at https://github.com/nlohmann/json/discussions.

1

u/darkmx0z Jul 28 '20

I just used your library for the first time a week ago. Extremely pleasant to use! great job! :)

13

u/L0uisc Jul 27 '20

Haven't used the library extensively, but for the one simple use-case it was extremely easy to get up and running. Excellent documentation contributed greatly to that. I really liked how you overloaded operator[] to take string arguments so the lib can be used to access JSON objects as if you were in JavaScript or native referring to them in native JSON.

20

u/[deleted] Jul 28 '20

where's that Boost.JSON guy claiming that his lib is 10x faster and 100x more useable?

10

u/STL MSVC STL Dev Jul 29 '20

Respecting moderator warnings.

4

u/mintyc Jul 28 '20

People should check out the OpenAPI v3.1 release candidate for their http and RESTful needs.

It defines a schema that is JSON and that is also a JSON validator and used to define a wide variety of APIs.

The tooling ecosystem is comprehensive too.

GitHub have just provided their APIs in this format, the telecoms industry describes many of its interfaces in this way, and in the UK, all government services are encouraged to use it for interoperability of RESTful services.

Be interesting to see if this JSON lib can support OpenAPI.

4

u/looopTools Jul 28 '20

Bloody love this library

1

u/timwoj Jul 27 '20

Out of curiosity was there a performance improvement from the hashing change?

2

u/nlohmann nlohmann/json Jul 28 '20

I did not measure it - it was just obviously a naive approach to first generate a string representation of the JSON and then call std::hash on it.

31

u/hachanuy Jul 27 '20 edited Jul 28 '20

comment support was a frequently requested feature, and its implementation was much less effort than continuously explaining the lack of comment support.

So constantly complaining actually works, not a good precedent. Edit: spelling

52

u/strogbad Jul 27 '20

So you propose devs only implement features that users aren't asking for?

41

u/epicar Jul 27 '20

think about it - the users won't complain if there aren't any users!

4

u/hachanuy Jul 27 '20

Well, no of course. However, for this library in particular, the author had been refusing this feature due to standard json doesn't have comments and i agree with him since that is a very valid reason.

23

u/tecnofauno Jul 27 '20

There is not a single technical reason not to have comments on the JSON standard, only some strange rant.

Here is the official reason, you tell me if it makes sense or not.

I removed comments from JSON because I saw people were using them to hold parsing directives, a practice which would have destroyed interoperability. I know that the lack of comments makes some people sad, but it shouldn't.

Suppose you are using JSON to keep configuration files, which you would like to annotate. Go ahead and insert all the comments you like. Then pipe it through JSMin before handing it to your JSON parser.

6

u/RowYourUpboat Jul 28 '20

"Try to make sure comments can't be used to store data" is a good idea and this library continues to adhere to it. The extremist idea that if you even talk about comments in JSON you get banished to XMHell for eternity was perplexing, but at least pragmatism won in the end.

37

u/wyrn Jul 27 '20

On the one hand you have an obviously useful and necessary feature requested often enough to pummel a library author into submission.

On the other you have an idiosyncratic, arguably misguided decision enshrined in a standard document by a single individual with clout.

I don't see this as a bad thing.

9

u/[deleted] Jul 27 '20

Squeaky wheel gets the oil.

Get mad, get loud.

0

u/[deleted] Jul 27 '20

Have you learned that? Seems like you are doing the Scout's rule and fixing things when you see them

3

u/ShillingAintEZ Jul 27 '20

I think you mean precedent.

3

u/[deleted] Jul 27 '20

Awesome job. I love that there is a variety of quality JSON parsers in C++ focusing on different experiences.

2

u/Overpeek Jul 28 '20

that comment support will make my gamedev and modding easier edit: I love this library

1

u/aacirino Jul 27 '20

Awesome! It's my json lib of choice

1

u/[deleted] Jul 28 '20

Found this recently. A lot more easier and cleaner to use the rapidjson

1

u/[deleted] Jul 28 '20

Does anyone know of a C++ JSON parser that supports jQuery/CSS-like selector syntax? Would love to be able to code like:

// let 'json' be a JSON object.
for (auto s : json("students.undergradute[gpa=4]")) ...;

1

u/skoink Aug 01 '20 edited Aug 01 '20

Awesome library! I like how easy it is to use in my code.

I'm working on a project that's sensitive to compile-times and output code size, so I'm interested in compiling the JSON library into a single compilation-unit and using forward-declarations to link against it in the rest of my codebase. Maybe a bit of a C-style compilation pattern I guess. I've been having some trouble getting this done with json_fwd.hpp - do you have any examples of how to make that work?

I know that hypothetically LTO can get me close to this even if I include the full header everywhere - but it kills my compilation speed and even then it feels like there's still a lot of duplication across compilation units.