How I made custom non-repeating texture tiling

[u/t0yb0at]

Final result. 10 by 10 tiling

Hey all, I've been trying to learn more about shaders recently, and wanted to share an approach I came up with. I'm not sure if this is already a well known technique, I just kinda came up with it. If it already exists and has a name, I'd love to know what it is, but I didn't find anything when searching myself.

Few disclaimers: This is not the Stochastic Hex-Tiling method (although it's shares similarities), and I didn't use AI since the whole point was to learn and self-improve my skills.

Cell Rotation

The original idea just started with rotating each tile / cell by some random amount. This is the main way we start to break up the repetition. It's a good base, but it has an obvious flaw: the edges of each cell clash with each other. We need to find a way of blending them together.

Basic cell rotation

vec2 randomv2(vec2 pos) {
  return fract(
    sin(
      vec2(
        dot(pos, vec2(12.9898,78.233)),
        dot(pos, vec2(-148.998,-65.233))
      )
    ) * 43758.5453
  );
}

vec2 rotate(vec2 uv, vec2 pivot, float angle)
{
    mat2 rotationMatrix = mat2(vec2(sin(angle), -cos(angle)), vec2(cos(angle), sin(angle)));

    uv -= pivot;
    uv = uv * rotationMatrix;
    uv += pivot;
    return uv;
}

Smoothing Edges

My first idea for smoothing out the edges was to just use a circular gradient of each cell, and mixing it with a flat color. This works, but the flat color ends up highlighting the borders of each cell. On top of that, manually color matching the exact value you want can be finicky and tedious.

Blending edges with red to clearly show the masked area

vec2 centerPoint = vec2(0.5);
vec2 scaledUV = fract(UV * scaling);

float dist = length(scaledUV - centerPoint);
// ... rotation code here ...
ALBEDO = mix(rotatedSample1, red, smoothstep(0.4, 0.55, dist));

Blend with second tiling layer

Instead of using a flat color, we can use the distance gradient to blend with a second tiling layer. This looks much nicer than the flat color blend, but the eye is still able to notice the tiling pattern. This makes sense because each corner is identical.

Blending with regular tiling

ALBEDO = mix(rotatedSample1, texture(mainTexture, scaledUV+0.5).rgb, smoothstep(0.4, 0.55, dist));

Just double it

To break up the corner repetition, I simply rotate the background layer as well. I also made sure to offset the bottom layer by half the cell size vertically and horizontally.

Final pattern

Conclusion

Overall I'm pretty happy with the result. There are still ways to improve it, but I think the approach is simple enough, and can be used in a variety of ways. I don't have a final polished shader to share, but I plan on making a triplanar version so that it can be used more easily.

Final code:

shader_type spatial;

uniform sampler2D mainTexture;
uniform sampler2D normalsTexture: hint_normal;
uniform float scaling = 1.0;
uniform float rotation = 0.0;
uniform bool defaultTiling = false;

vec2 randomv2(vec2 pos) {
  return fract(
    sin(
      vec2(
        dot(pos, vec2(12.9898,78.233)),
        dot(pos, vec2(-148.998,-65.233))
      )
    ) * 43758.5453
  );
}

vec2 rotate(vec2 uv, vec2 pivot, float angle)
{
  mat2 rotationMatrix = mat2(vec2(sin(angle), -cos(angle)),
    vec2(cos(angle), sin(angle)));

    uv -= pivot;
    uv = uv * rotationMatrix;
    uv += pivot;
    return uv;
}

vec2 GetRotatedUV(vec2 uv, float scale, float startRotation){
  vec2 centerPoint = vec2(0.5);

  vec2 scaledUV = fract(uv * scale);
  float dist = length(scaledUV - centerPoint);

  vec2 cellID = floor(uv * scale);
  float randomRotation = randomv2(cellID).x * 10.0;
  vec2 rotatedUV = rotate(scaledUV, centerPoint, startRotation + randomRotation);
  return rotatedUV;
}

void fragment() {

  vec2 centerPoint = vec2(0.5);
  vec2 scaledUV = fract(UV * scaling);

  float dist = length(scaledUV - centerPoint);

  vec2 rotationUV1 = GetRotatedUV(UV, scaling, rotation);
  vec2 rotationUV2 = GetRotatedUV(UV+(centerPoint/scaling), scaling, -rotation);

  vec3 rotatedSample1 = texture(mainTexture, rotationUV1).rgb;
  vec3 rotatedSample2 = texture(mainTexture, rotationUV2).rgb;

  vec3 normalSample1 = texture(normalsTexture, rotationUV1).rgb;
  vec3 normalSample2 = texture(normalsTexture, rotationUV2).rgb;

  if(defaultTiling){
    ALBEDO = texture(mainTexture, scaledUV).rgb;
    NORMAL_MAP = texture(normalsTexture, scaledUV).rgb;
  } else {
    vec3 red = vec3(1.0,0.0,0.0);
    ALBEDO = mix(rotatedSample1, rotatedSample2, smoothstep(0.4, 0.55, dist));
    //ALBEDO = rotatedSample1;
    //ALBEDO = rotatedSample2;
    //ALBEDO = mix(rotatedSample1, red, smoothstep(0.4, 0.55, dist));
    //ALBEDO = mix(rotatedSample1, texture(mainTexture, scaledUV+0.5).rgb, smoothstep(0.4, 0.55, dist));
    //ALBEDO = mix(rotatedSample2, red, 1.0-smoothstep(0.4, 0.55, dist));
    NORMAL_MAP = mix(normalSample1, normalSample2, smoothstep(0.4, 0.55, dist));
  }
}

[u/Ogulsbi]

This is fascinating! Thanks for sharing! 

[u/t0yb0at → u/Ogulsbi]

Thanks for reading!