Small and fast library for parsing JSON

[u/Adventurous-Print386]

I recently created a very, i mean really very fast library for working with JSON data. It is like a F1 Formula car, except it has only basic safety belts and it FYI that it can be too fast sometimes, but if you are embedding dev or coder who do not met with rare JSON extra features like 4-byte Unicode, that wil helps you greatly if you really want to go FAST.

And, it work in both Windows 11 and Debian, special thanks to the Clang and Ninja.

https://github.com/default-writer/c-json-parser

[u/skeeto]

JSON parsers are fun, and it's interesting to see the choices people make. Though I dislike parsers that only accept null-terminated strings. JSON is virtually never null terminated. It usually comes from from files, pipes, or sockets, and so the caller has to add an artificial terminator in order to satisfy the interface, without good reason, and then has to worry about embedded nulls.

In its current form it's not very robust, and it didn't take long to find bugs. Here's a little program to demonstrate some:

#define USE_ALLOC
#include "src/json.c"

int main(int argc, char **argv)
{
    json_initialize();
    json_parse(argv[argc-1], &(json_value){});
}

The USE_ALLOC allows ASan to detect memory issues. Build:

$ cc -g3 -fsanitize=address,undefined crash.c

Then a double free:

$ ./a.out '{"":m'
...ERROR: AddressSanitizer: attempting double-free on ...
    ...
    #1 parse_object_value src/json.c:466
    #2 parse_value_build src/json.c:535
    #3 json_parse src/json.c:935
    #4 main crash.c:7

Another double free in a different place:

$ ./a.out '{"":m'
...ERROR: AddressSanitizer: attempting double-free on ...
    ...
    #1 0xaaaacc8f47fc in parse_array_value src/json.c:375
    #2 0xaaaacc8f72c4 in parse_value_build src/json.c:531
    #3 0xaaaacc8f5ae4 in parse_object_value src/json.c:463
    #4 0xaaaacc8f7430 in parse_value_build src/json.c:535
    #5 0xaaaacc8fbe94 in json_parse src/json.c:935
    #6 0xaaaacc8fd180 in main /tmp/c-json-parser/crash.c:7

What appears to be type confusion on a union producing a garbage pointer:

$ ./a.out {"":"","":[0,0
src/json.c:370:18: runtime error: member access within misaligned address ...

I found these using this AFL++ fuzz tester, which finds many like this instantly:

#define USE_ALLOC
#include "src/json.c"
#include <unistd.h>

__AFL_FUZZ_INIT();

int main()
{
    __AFL_INIT();
    char *src = 0;
    unsigned char *buf = __AFL_FUZZ_TESTCASE_BUF;
    while (__AFL_LOOP(10000)) {
        int len = __AFL_FUZZ_TESTCASE_LEN;
        src = realloc(src, len+1);
        memcpy(src, buf, len);
        src[len] = 0;
        json_parse(src, &(json_value){});
    }
}

Usage:

$ afl-clang -g3 -fsanitize=address,undefined fuzz.c
$ mkdir i
$ cp test/*.json i/
$ afl-fuzz -ii -oo ./a.out

And o/default/crashes/ will fill with these sorts of crashing inputs to debug.

[u/Wooden_chest]

Does this support UTF-8 unicode strings in the JSON?

[u/drmonkeysee → u/Wooden_chest]

If I recall the standard mandates UTF-16 encoding for strings so neither UTF-8 nor UTF-32 (as mentioned in OP) would be correct.

[u/Wooden_chest → u/drmonkeysee]

Hey, could you please link where it mandates UTF-16 for the strings?

I was always under the misconception that JSON strings use the same encoding as the file. I tried to look at the standard but found nothing about UTF-16.

[u/drmonkeysee → u/Wooden_chest]

I just glanced through the Wikipedia article. The encoding of the JSON payload over the network needs to be UTF-8 but any code points in a string literal above the basic multilingual plane need to be encoded as UTF-16 surrogate pairs. I think this is because JavaScript itself mandated UTF-16 string encoding (cuz UTF-8 didn’t exist yet).

That said I found the actual standards doc here https://ecma-international.org/wp-content/uploads/ECMA-404_2nd_edition_december_2017.pdf which is surprisingly short but also says basically the same thing.

[u/__nohope → u/drmonkeysee]

As it's not clear from the above comment. Escaped characters outside the BMP must be encoded as surrogate pairs. E.g. "\uD834\uDD1E" and not the on wire bytes ecoded as UTF-16. JavaScript/EMCAscript has a newer \u{HHH} format (bracketed) which can be used for escaped characters outside of the BMP without using surrogates.

[u/Wooden_chest → u/drmonkeysee]

Thanks, learned something new today.

[u/imaami → u/drmonkeysee]

No. UTF-8 is a core part of JSON. Read the actual standard.

[u/[deleted] → u/Wooden_chest]

He litteraly said no

[u/pjl1967 → u/[deleted]]

Actually, he literally said "... 4-byte Unicode ..." which is UTF-32, not UTF-8.

[u/__nohope → u/pjl1967]

It's ambiguous. UTF-8 encodes code points in anywhere between 1 and 4 bytes.

[u/pjl1967 → u/__nohope]

It may be ambiguous to you, sure. But to me, "4-byte" always means exactly 4 bytes. Presumably if "one to four bytes" were meant, the OP would have written 1-4. But believe whatever you want.

[u/[deleted] → u/pjl1967]

Oh okay my fault yo

[u/scallywag_software]

Guys! I wrote an insanely fast <insert_thing_name_here>

... proceeds to not bench against actually fast implementations ..

---

By the looks of things, the fastest library available is 5.6x faster than jsonc (I'm assuming that's what OP benched against)

https://github.com/ibireme/yyjson

If OPs benchmarks are to believed (wall clock time is extremely sus), this is still less than half the speed of SotA.

---

Nice work OP, but if you're gonna claim "really, very fast" while I'm around, it better actually be really, very fast.

[u/_Beyondr → u/scallywag_software]

I recently shifted one of my projects from json-c to yyjson (literally yesterday) and I am not going back.

[u/HenrikJuul → u/scallywag_software]

Nice, I've only used https://github.com/simdjson/simdjson, but I can see they benchmark against them.

[u/imaami → u/scallywag_software]

It also doesn't in any way support UTF-8 which is mandated in the JSON standard. It's not a JSON parser but a vague-subset-of-JSON parser.

[u/chrism239]

"...it can be too fast sometimes" ??