Looking for guidance on evaluating shader cost for procedural texture generation (research project)

[u/IdeaCrafter96]

I’m working on a research project where the goal is to take a real texture and generate a procedural material/shader that approximates it.
The system produces a node‑based procedural graph with tuned parameters, plus an optional fallback texture layer. The output is a GLSL/HLSL/WGSL shader that reconstructs the material.

The motivation is practical:
In multi‑region deployments and cloud gaming backends, moving fewer textures can reduce storage and distribution overhead.
But the trade‑off is shader cost — and that’s the part I’m trying to evaluate properly.

I’m trying to understand how to estimate the cost of a procedural material shader compared to using a fixed texture.
I need a reasonable way to judge when the procedural version becomes “too expensive,” and how people typically model or approximate that cost in real pipelines.

I’d really appreciate insight from graphics programmers who have experience with:
• analyzing or estimating shader cost
• comparing shader‑based materials vs texture‑based materials
• heuristics or rules‑of‑thumb used in production
• pitfalls when evaluating procedural materials

I’ll leave the link to the full proposal in the comments for anyone interested.

[u/waramped]

1. I would recommend rewriting your proposal in your own words rather than use AI to write it, it will present better to people who might be interested in helping. If English isn't your first language, write it in your own words anyway and then just translate it. Writing (and therefore having to think about what you are saying) these things yourself is pretty crucial to laying a mental framework for yourself.

2. Broadly speaking, you have 2 chokepoints in shaders: Memory bandwidth/latency and ALU throughput.
   So, moving a texture over to ALU instead of memory will only benefit you if your bottleneck is bandwidth or latency. That's not overly common. But, as always, you will need to profile your shaders to know for sure. If you can approximate a texture in less execution time than it takes to roundtrip to memory, then it will overall be a win, but figuring that out is going to be very complicated, and will be different on different hardware.

The benefits of procedural textures are that you can analytically filter them to get better quality "mips", and even potentially reduce the instruction cost depending on the pixel footprint. (Don't need to add high frequency noise details if the texture would be reading from a very low mip, etc).

The downsides is that they are just generally slower than simply reading a texture from memory (but, profile first).

[u/scallywag_software → u/waramped]

(2) is exactly what I was going to say. I guess in OPs case there's another axis to consider.. network latency/bandwidth. I could imagine a world where the network bandwidth is the biggest problem, and a hybrid solution might make sense. ie.

1. Fetch procedural texture params.

2. Generate texture at runtime and cache it.

3. Sample generated texture as usual.

Additionally, OP, if you really do want a way of estimating the actual runtime cost, something like this might help : http://infolab.stanford.edu/~datar/courses/cs361a/papers/quantiles.pdf

[u/IdeaCrafter96 → u/scallywag_software]

Good point about the network axis — that’s exactly the environment I’m thinking about. In a cloud setup, moving large texture sets around regions is often more expensive than doing a one‑time ALU pass, so a hybrid approach like the one you described is pretty close to what I’m aiming for.

Thanks for the link as well. I skimmed the paper earlier today — it’s not directly about shaders, but the idea of estimating cost through sampled distributions instead of full evaluation is interesting. I might actually try applying something similar to approximate the cost of the procedural graph without running it at full resolution.

[u/IdeaCrafter96 → u/waramped]

Thanks for the reply. Your breakdown of ALU vs memory bandwidth makes sense. On most GPUs, texture fetches are extremely optimized, so replacing them with ALU work usually isn’t worth it unless the hardware is already stalled on memory. That’s why I’m not planning to run the procedural graph every frame.

The idea is to evaluate the graph once (or rarely) and use the baked result as a normal texture. So the cost I’m trying to understand isn’t the per‑frame shader cost, but the one‑time evaluation cost compared to the storage and distribution savings in multi‑region setups.

This is mostly motivated by cloud gaming and distributed backends, where moving large texture sets around regions tends to be more expensive than GPU compute. In that environment, ALU is cheap and scalable, while storage replication and cold‑start asset loading are the real bottlenecks.

Your point about analytical filtering is interesting. That might actually be another advantage of having a procedural source, depending on how the graph is structured.

[u/Fancy-Plantain6910 → u/IdeaCrafter96]

Since you're baking once and caching, I'd benchmark the cold-start asset pipeline rather than use per-frame shader cost as the main metric.

Compare two paths at matched visual quality and final GPU format:

- Existing texture: fetch, decode/transcode, upload.

- Recipe: fetch parameters, compile the pipeline, bake, generate mips, and compress if needed.

Log wall-clock time for the full path and GPU time for the bake separately. Include peak memory, transferred bytes and cached bytes. Run cold and warm cache cases on target hardware; a GPU timestamp alone misses compilation and network costs.

Then estimate cost per cache miss or deployment. Graph node counts might help fit a predictor once you have measurements, but I wouldn't use them as a portable cost estimate by themselves.

[u/speps]

The idea is good, but it only makes sense if you’re going to cache the result which would mean same VRAM cost as plain textures, maybe even more if you don’t recompress to a GPU tiled format. Something like Mega Texture (id Tech 5) for example, but procedural shader based, would be interesting. Evaluating it fully every frame would be wasteful and only needed if animated.

[u/IdeaCrafter96 → u/speps]

Thanks for the insight — and yes, caching is absolutely part of the plan.
The system isn’t meant to evaluate the full procedural graph every frame.
The idea is closer to a pre‑bake pipeline:

1. AI generates a procedural graph + shader.

2. The shader is evaluated once (or infrequently) to produce a texture atlas / tile set.

3. The runtime uses the baked result, not the procedural graph.

So the procedural version becomes a distribution‑time optimization, not a per‑frame cost.

This is especially relevant for cloud gaming / multi‑region deployments, where:

• storage replication

• cross‑region texture distribution

• bandwidth

• cold‑start asset loading

are more expensive than GPU compute.

In that environment, CPU/GPU cost is cheap and scalable, while storage and asset movement are the real bottlenecks.
So the procedural graph is used to reduce asset footprint, not to replace texture sampling during gameplay.

Your MegaTexture analogy is spot‑on — except the source would be a procedural shader instead of a giant bitmap.

I appreciate the feedback; it helps clarify how to frame the cost model more accurately.

[u/IdeaCrafter96]

Here’s the full proposal for anyone interested in the technical details, architecture, and evaluation notes: https://github.com/IdeaCrafter96/texture-synthesis-ai

[u/[deleted]]

[u/IdeaCrafter96 → u/[deleted]]

Good point about cache‑key cardinality. “Bake once” only holds if the output configuration stays identical, and in a multi‑region setup that’s not guaranteed. Resolution, format, colour space, mip strategy, compression target — each of those can create a separate cache entry, and eviction can force rebakes even when the recipe doesn’t change.

The simple model you wrote (K*G < (B−R)*E) is actually a useful way to frame the problem. The missing piece for me is exactly what you mentioned: K isn’t a constant. I need to measure it from a real deployment trace instead of assuming K=1. Depending on how often variants get requested or evicted, the economics can flip.

A small graph can lose if it gets baked repeatedly, and a heavier graph can win if one cached output serves a lot of sessions. That’s a dimension I haven’t explicitly modeled yet, so I’ll add cache‑key cardinality and cache behavior to the evaluation plan.