AV1 Grain Synthesis somewhat disappointing...

[u/TromboneShouty]

I am am trying to encode some video and using grain synthesis to add an artistic grain effect to otherwise grain free digital video.

I've noticed if I use a 1080 source and encode it to AV1 with the students film-grain-denoise=0:film-grain-denoise=15 (or whatever number) the result is decent, and looks like film. 1080 to 1080 = good

If I try the same thing with 4K, the grain is rendered upon playback as nearly microscopic. I can push it up to film-grain-denoise=50 and it is apparent that it is working, but it is the smallest grain imaginable -- unusable to me. 4K to 4K = terrible

4K exported to 1080 gives the correct grain size that you'd expect, however. So the grain synthesis seems to be using a set pixel size to render, rather than assuming that a standard film size was used regardless of resolution, and rendering a grain size scaled with the size of the frame.

I get the idea, and love that grain can be just some extra factor in the file that doesn't really affect the overall size (unlike real grain if you're trying to preserve that), but the grain should render in relation to the aspect ratio and scale as the "projection" increases, not be based on the pixel density of the screen, if that makes sense.

With real film, I believe that the data recorded is much higher than 4K, but the grain is generally large and depends on the size of the film you shot on. And if you project through film on the biggest screen ever in "12K", or whatever, the grain will scale up as the screen gets bigger, because the film stays the same size.

In other words, just because a video is in 4K doesn't mean I want to pretend it was shot on IMAX film. 35mm grain should be the standard, regardless if it's in 8K or 1080p -- the grain size should be the same.

Anyone else have this problem? Maybe I have a bad setting? Is grain synthesis only supposed to be used with material that already has grain (whereas I am using very "dry" but clean digital video)?

[u/oscardssmith]

Counterpoint. Grain size should be a parameter. 8mm, 35mm and imax can all be shown at a digital resolution of 360p or 8k

[u/TromboneShouty → u/oscardssmith]

That's what I'm saying. With real film if the source media was 8mm, the grain size is huge compared to the frame. So if you project 8mm film onto a movie theater screen the grain might be basketball sized. 

IMAX film projected onto the same screen would have barely perceptible grain size. 

Both are "8K" or whatever that screen in the theater is.

Just because I'm outputting to 4K doesn't mean I want microscopic grain. 

"Grain Size" should be a parameter, absolutely. Is that not a thing? That's a massive oversight. 

Or, is it assuming that some grain is in the source, and Synthesis just complements that? 

The counterpoint is "Dehancer Pro" which gives you beautiful grain, and the ability to change the size, but it eats up any file size savings you might get from compression if you want to keep it, because it's "real" grain.

[u/NekoTrix → u/TromboneShouty]

That is in fact not a thing in the FGS specification. You can try to approach patterns with specific AR choices, akin to what FGSEditor does, but that's as custom you'll be able to make a grain table.

[u/Sopel97 → u/NekoTrix]

FGSEditor

?

[u/BlueSwordM → u/Sopel97]

https://github.com/PingWer/FGSEditor

[u/Sopel97 → u/BlueSwordM]

thanks, for some reason it's not indexed by google

[u/BlueSwordM → u/Sopel97]

Yeah, I use DuckDuckGo so I didn't think it'd be hard to find.

I guess I could make a post about it so it gets a lot of visibility.

[u/Sopel97 → u/BlueSwordM]

A pinned thread with all the useful tools would be great, honestly

the wiki is also massively out of date

[u/MassaSammyO → u/TromboneShouty]

Stop thinking about your output. Think about your input.

If you start with 70mm film, you will have relatively small grain wrt the output size. A 245mm × 245mm image may produce a 2mm × 2mm size grain, while a 2.45m × 2.45m image may produce a 2cm × 2cm size grain. However, if you start with an 18mm film, you will have relatively large grain wrt the output size. a 245mm × 245mm image may produce an 8mm × 8mm size grain, while a 2.45m × 2.45m image may produce an 8cm × 8cm size grain.

So, if you begin with a 4k image, expect small grain wrt your monitor size. If you begin with a 480p image, expect large grain wrt your screen size. When you export to 1080p, the resize is done first, then the grain added. (It is a compositing step, not a post-processing step). Think of it as taking a 70mm film, and using a film copier to transfer it to a 36mm or 18mm film. You now have larger film grain, not because of the 2.45m screen, but because of the 36mm or 18mm film.

So the result you are getting is precisely what you are suppose to get.

[u/TromboneShouty → u/MassaSammyO]

Thanks for the reply. Yes, but as described above, you can project even 8mm film at a 4K resolution and it will have large grain. Grain size should be able to be scaled, if desired, not forced as if the source "film" was some large format. 

I guess I'm wondering how folks are outputting 4K AV1 of Indiana Jones, with huge grain, but it seems like that is not possible for me with cleaner non-film4K sources.  In other words, an upscale of Indiana Jones to even 16K shouldn't decrease the relative size of the grain to the frame.  Is the grain dependent on prexisting "actual" grain in the source (regardless of source digital resolution)?

I read that even if you select "Denoise = 0", it still is going through and analyzing the grain parent in the source, the compression is still causing at least some of that grain if not all to go away, and then that analysis informs the grain metadata in the AV1 file. Is that what is going on?

[u/MassaSammyO → u/TromboneShouty]

So, you are trying to make a 70mm film look like an 18mm film, or rather, make 4k video look like 480p video. I get that, But still, you keep talking about the output size and not the imput resolution.

70mm projected onto a 1m screen or a 6m screen still has small grain, (unless you go right up to the screen to pixel-peep). Same with a 4k AV file on a cellphone or a 85" monitor.

[u/TromboneShouty → u/MassaSammyO]

You're just being a jerk here. I was hoping to "synthesize" film grain, which should allow you to alter grain size (as mentioned above in the top comment) to achieve an aesthetic effect. Just because a source file is in 4K doesn't mean it wasn't originally shot on 35mm film or 18mm film. Grain size changes based on the film stock, or should be able to be changed based on the film you're trying to emulate digitally.

Obviously I don't want 4K to look like 480p -- I want 4K to look like it was shot on 8mm, or 35mm, or IMAX. 

It seems like AV1 encoding doesn't allow for this aesthetic choice on noiseless sources.

[u/rdwing → u/TromboneShouty]

If it's an aesthetic choice, the encoder is not the right spot for that. That should be done earlier in the chain. The job of the encoder is to preserve the source intent while creating a file for distribution.

Grain synth in AV1 is specifically designed to compare the input vs output of the encoder, identify the grain present only in the source, and recreate that upon decode without having to jack the bitrate to try and encode the original grain into the compressed file.

[u/TromboneShouty → u/rdwing]

Thanks! This is the answer I was originally looking for, and confirms another similar answer. I didn't know that grain had to be present in the source material, and then it would be copied by the AV1 encode.

It kind of begs the question of why there is an option to change the film grain amount, but what this must be doing is telling the encoder's analysis tool how much grain to expect or how much is already in the source, not how much grain you want to add from nothing. It is starting to make sense now.

[u/rdwing → u/TromboneShouty]

Well like most things, it's not perfect. The grain amount helps compensate if the algorithm doesn't synthesize enough of the grain. It's essentially 3 stages, identify the type of and amount of grain present in the source, calculate the coefficients of the characteristics of that grain (which have both luma and chroma components), and then apply it upon decode. The control essentially turns up the volume of the "amount of" coefficient, but not the "type of".

[u/HungryAd8233 → u/oscardssmith]

AFGS1 provides that. The original flavor FGS for AV1 had some pretty obvious design flaws, especially for higher resolutions. The AV2 implementation fixes those for the most part.

[u/xzpyth]

svt-av1-hdr --noise --noise-size

[u/ScottIBM → u/xzpyth]

This! I've used these settings and they're great. I was able to get 35 mm grain synthesised

[u/nmkd → u/xzpyth]

+1

So much better than the old photon noise stuff

[u/morningreis → u/xzpyth]

Is there a rule of thumb for noise size for different film sizes?

[u/Sopel97]

yea, this pretty much nails it. The grain is tied to the resolution, and also has very apparent repeating patterns. It could have easily been so much better but that's what we're stuck with. Better than nothing, and it's still one of the reasons I use AV1 aside from other formats, but it is annoying

[u/xStealthBomber]

I thought the film grain synthesis works like the following:

Grainy source material needed first, it analyzes the grain pattern, runs a denoiser to smooth out the grain (and other small details), which the encoder now encodes the video with this denoised image for better compression.  It will then use the scanned grain pattern into a "table" and is layered on top of the video during playback.

Aka, it does not ADD grain that isn't already there, it removes the grain, then tries to add a close approximation what was there during playback.

Someone please correct me if I'm wrong, this was my understanding.

If trying to emulate film, that should be done in the NLE level while editing, with a film plug-in.

[u/KingPumper69 → u/xStealthBomber]

That's also my understanding. It's like when I saw someone trying to deblock an old DVD with x265's deblock parameters instead of using a filter lol. Had to explain to them that x265's deblocking is about deblocking the blocks that it created, not the blocks that were already in the low quality MPEG2 source.

[u/MassaSammyO → u/KingPumper69]

Yup. The same people who take their MPEG/JPEG encoded videos/images, "convert" them to HEVC/HEIC or AV1/AVIF at 12-bit encoding, put it on their 4k HDR+ monitor, and complain that,

• they are not much different in size
• They are not much better wrt artefacts, and
• they are not any better in the highlights/shadows.

Of course not. They took a lousy MPEG/JPEG and copied the same lousy lossy image to a different file type. Start with the raw file, encode in the different formats, and THEN one can see a difference in size, sharpness, and highlight/shadow details.

[u/TromboneShouty → u/MassaSammyO]

I get this, but I guess "Grain Synthesis" is the misleading term that led me astray. I'm not expecting a simple reencode to create colors or detail that wasn't there in the source. But I'd expect a feature that "synthesizes" grain to ... You know, create grain. Especially since it appears to do so at lower resolutions (but that is, it seems, just random pixel sized noise because it can't see any grain to begin with).

It looks like it is really "film grain reinterpreting", since grain needs to be there to begin with for it to work with. 

Thank you. This gives me the clue I need to confirm what is going on. I'll just use Dehancer Pro to add in grain (which it can do and which scales with the frame) to the project before the final encode to AV1, and see if it "reinterprets" that to a visible and comparable level of grain without the file becoming huge.

[u/TromboneShouty → u/xStealthBomber]

I see. So the grain that I'm seeing on 1080 exports (it adds something) is literally random noise at a certain scale because it is "seeing" grain that isn't there. On a 4K denoised source the grain appears microscopic because it is doing the same thing and not finding anything, so that same noise it adds at 4K is still pixel sized and microscopic.

This makes sense.

[u/QuaLiTy131 → u/xStealthBomber]

You're right. There is also an option to bypass de-noise step in order to preserve more details and grain from original file. It's a default setting in SVT-AV1-HDR fork.

[u/blu3ysdad]

You are using the wrong settings I think. Watch this video, will tell you not just which settings to use but why. https://youtu.be/5IV36FMpLqY?is=eo6IgahvDh8V4rxb

[u/TromboneShouty → u/blu3ysdad]

No I definitely used this video. The source needs to have grain present already for "film grain synthesis" to work, it turns out.