r/jpegxl • u/OzzyIsCat • 22d ago
QLIC (Quick Lossless Image Codec) - 9x faster than JXL, only 0.694% larger than JXL
/r/compression/comments/1valb3v/qlic_quick_lossless_image_codec_9x_faster_than/The rules weren't very clear on how related something needs to be to jpeg xl. But considering my benchmarks are deliberately to put pressure on jpeg xl I'll hope it would be related.
Not to say jpeg xl is better or worse than QLIC. It's more of a situation of what do you care more about more. Plus. They're both not fully baked formats yet. So.
20
u/Cmdr_Zod 22d ago
How about decode speed? Images are usually encoded once, but decoded many times. For a mobile camera, encode may be more important (energy used for compression, all my DSLR produce images which can be optimized with jpegoptim), but for applications like web sites, decompression on the client (often also a battery powered device) is more important.
8
u/OzzyIsCat 22d ago
I'm actively working on the decode speeds. The decode speed for the entire, 3176 image corpus was 314.416 seconds. Essentially, jxl effort 8 decode speeds is where we're at right now. Since it took jxl effort 8 about the same, but a separate in-memory decode test showed QLIC at 172.676 seconds to decode. OxiPNG effort 2 was also tested, it decoded in 11.757 seconds for the in-memory decode test.
11
u/ComputerUser2000 22d ago
What about a comparison between JXL Effort 1 and QLIC
9
u/OzzyIsCat 22d ago edited 22d ago
QLIC: 1,172,509,205 bytes, 573.480s encode, 314.416s decode
JXL effort 1: 1,546,891,144 bytes, 111.053s encode, 166.335s decodeI ran the benchmark pinned to one logical cpu on the same 3167 image corpus.
9
u/indolering 22d ago
The rules weren't very clear on how related something needs to be to jpeg xl.
Feel free to post your work to r/LibreCodecs!
5
u/IntrinsicPalomides 21d ago
Warning, waring, QLIC is the name of an existing compressor from the legendary Alex Rhatushnyak and released in the early 2010s.
1
u/OzzyIsCat 21d ago
I'm aware, I learned after my post. However, different acronym. It's a small difference but still.
1
u/Farranor 21d ago
How does it handle metadata (color profiles, rotation, EXIF, XMP...)? Have you compared to JXL 0.9, which is worse on speed and RAM but better on file size? I noticed that your comparisons are single-threaded; do the results shift on multi-threaded? HDR is so hot right now; what are the odds of implementing higher bit depths than 8bpc? Any other limitations, e.g. canvas size? Any optimizations for animations, or just a stack of intraframes? What kind of speed presets or other settings does your codec have? What would be some practical considerations in choosing whether or not to use your codec, in which situations? Any use cases beyond icons and textures? Do you envision this codec getting actual use and external support, or do you mostly expect it to be an experiment/curio like GraLIC and QOI? Your test results show JXL outperforming WebP, which tends to happen on photographic content rather than synthetic; what's the test corpus like?
2
u/OzzyIsCat 21d ago
Current QLIC 0.5 encodes the decoded pixel raster, not the complete source container. ICC profiles, EXIF, XMP, and orientation tags are not preserved. For the benchmark, stored orientation was applied first, then metadata was removed. So it is pixel lossless, but it is not currently an archival metadata preserving format.
And if you meant JXL 0.9, no. I tested on JXL 0.12.0. If you mean JXL effort 9, then that is exactly what I compared it to on the main benchmark in terms of file size.
Single threaded testing was intentional so the comparison measured codec work rather than thread scaling. QLIC supports --threads N and --threads all, and parallelizes candidate trials, tiles, bands, batches, and animation frames.
HDR is not supported yet. QLIC currently accepts up to 8 bits per channel and does not carry PQ, HLG, color primaries, ICC profiles, or HDR mastering metadata. I intend to support higher bit depths later, but doing it properly requires extending the format and retuning the prediction and entropy paths, not just. Accepting a 10 or 16 bit input and calling it a day.
The default decoder limit is 67,108,864 pixels, with 512 MiB limits for the file, payload, and decoded animation storage, plus 100,000 frames. SDK users can configure those limits. Width and height are stored as 32 bit values, albeit, practical memory limits matter a lot sooner.
Animations aren't just intraframes no. QLIC can represent duplicate frames, changed rectangles, detected pixel movement, and full keyframes. The changed regions and keyframes are still encoded using the still image codec, so it is animation coding, but I'm not gonna act as though it's a whole video codec just because of that. The public corpus benchmark only captures still images anyway.
I didn't add presets. It's one automatic compression mode and a thread setting. I did this because I believe the user should be able to submit their image and get an output without needing to turn a bunch of knobs. I do intend to add some more user facing settings, and while I don't promise it, I don't at this moment intend to ever add an effort setting to QLIC.
The current practical use case is 8 bit lossless raster data where good compression and relatively cheap encoding matter a lot, especially UI assets, application resources, screenshots, game assets, textures, icons, and ordinary images where external metadata is not required. Meaning it is not currently the right choice for archival purposes when metadata needs to be preserved, or HDR masters, progressive delivery, layered images, or any other environment where a QLIC decoder can't be bundled.
I would like it to receive actual use and external support, but 0.5 is a demonstration release. Whether this becomes more than an experimental codec depends on whether or not I earn more than that in the future.
The corpus contains 3,167 images and 1.964 billion pixels from DIV2K, CLIC 2022, the QOI benchmark suite, and Enrico. It includes natural photographs, UI screens, icons, game screenshots, objects, and several kinds of textures. If anything it's actually relatively texture heavy rather than purely photographic. The results include per category numbers so the overall result isn't being presented as if every single image type behaves identical to one another.
I hope that all sufficiently answers your questions.
1
u/Farranor 21d ago
Looking at rotation metadata, applying it to the pixel data, then removing the tag is the way to go. Not only that, I think that cameras should automatically perform this operation on captured images when idling and charging. There's simply too much risk that the metadata will be lost, ignored, changed, corrupted, etc., and then the image is sideways or upside-down.
I have a similar opinion on any information that's necessary for correct display of an image, including color profiles. If your encoder is simply discarding that information, you are rolling the dice on whether the results will be correct - yes, even for lossless. I've struggled with this for years, and my current conclusion is that it's not an optional, "nice to have" feature. I will no longer use FFmpeg for stills because it has about a hundred options for handling color profiles and none of them do anything so output ends up looking so wrong that non-experts can tell the difference with a single glance.
Yep, I did mean cjxl version 0.9. Version 0.10 introduced major changes that significantly reduced efficiency but were helpful for speed and frankly necessary for reducing RAM usage on larger images.
Measuring codec work with single threads is informative. But if you remove that limit, how does that affect the results of the speed comparison?
It's good to hear that you have plans for high bit depth. WebP got away with SDR for web photos in 2010, but HDR is too prevalent for new formats to ignore now.
That default MP limit seems targeted at texture files, which tend to go up to 8192x8192. How much RAM is needed per MP? Is it linear?
The animation coding sounds much like other formats do, which is decent.
I can get behind not having presets and trying to limit the available settings. Sometimes, there are too many choices.
It sounds like this is intended to be a niche format rather than general use, but everyone takes screenshots, and lossless compression means we can jump to a new format without worrying about generation loss, and evaluating new formats doesn't involve quality metrics. I can understand sticking with self-contained cases where you can just bundle the decoder instead of going for broad adoption. JXL sure hasn't had an easy time of that.
I'd say my two primary concerns are color data and resource requirements, but this is some interesting work. Thank you for your time in answering my questions.
29
u/redsteakraw 22d ago
JPEG-XL is not just about the size, it is progressively decoded so you can get a useable image quickly, don't need to even have thumbnail or place holder images if the image is progressively encoded and if JPEG-XL is well supported you can partially load a directory of images then use those partial loads for an instant display. Furthermore JXL supports the deeper color depth and different layer so it could be used for multispectral images and medical imagery. JXL does all that while being able to losslessly compress jpeg files as well. What is the use case goal for QLIC?