How to Create Distortion and Grain Effects on Scroll with Shaders in Three.js

Posted on by

With shaders, we can, with just a few lines of code, a bit of math, and a lot of experimenting and trial and error success, create amazing stuff. In this tutorial, I’m going to show you my setup and how I got an interesting distortion and grain effect working on scroll using shaders in Three.js.

For quite some time, I had a big block in creating these effects because I simply didn’t get how to sync the position of native HTML images with WebGL. Turns out (as always) that once I figured it out, it’s pretty simple.

Setup

By now, I have my own little boilerplate with Nuxt.js where I collect such base functionalities, but I’ll try to explain briefly what’s going on for the demo without Nuxt.

Media and meshes

The basis of how I manage the media and meshes is the following function:

// this gets all image html tags and creates a mesh for each
const setMediaStore = (scrollY) => {
  const media = [...document.querySelectorAll('[data-webgl-media]')]

  mediaStore = media.map((media, i) => {
    observer.observe(media)

    media.dataset.index = String(i)
    media.addEventListener('mouseenter', () => handleMouseEnter(i))
    media.addEventListener('mousemove', e => handleMousePos(e, i))
    media.addEventListener('mouseleave', () => handleMouseLeave(i))

    const bounds = media.getBoundingClientRect()
    const imageMaterial = material.clone()

    const imageMesh = new THREE.Mesh(geometry, imageMaterial)

    let texture = null

    texture = new THREE.Texture(media)
    texture.needsUpdate = true

    imageMaterial.uniforms.uTexture.value = texture
    imageMaterial.uniforms.uTextureSize.value.x = media.naturalWidth
    imageMaterial.uniforms.uTextureSize.value.y = media.naturalHeight
    imageMaterial.uniforms.uQuadSize.value.x = bounds.width
    imageMaterial.uniforms.uQuadSize.value.y = bounds.height
    imageMaterial.uniforms.uBorderRadius.value = getComputedStyle(media).borderRadius.replace('px', '')

    imageMesh.scale.set(bounds.width, bounds.height, 1)

    if (!(bounds.top >= 0 && bounds.top <= window.innerHeight)) {
      imageMesh.position.y = 2 * window.innerHeight
    }

    scene.add(imageMesh)

    return {
      media,
      material: imageMaterial,
      mesh: imageMesh,
      width: bounds.width,
      height: bounds.height,
      top: bounds.top + scrollY,
      left: bounds.left,
      isInView: bounds.top >= -500 && bounds.top <= window.innerHeight + 500,
      mouseEnter: 0,
      mouseOverPos: {
        current: {
          x: 0.5,
          y: 0.5
        },
        target: {
          x: 0.5,
          y: 0.5
        }
      }
    }
  })
}

This script collects all images (that we want in WebGL) in an array, adds event listeners to them (so we can manipulate uniforms later), creates the meshes with Three.js, and adds necessary uniforms like dimensions. There are more details to it, like checking if the images are in view or if they have a border radius, but you can check these in the code yourself. The code has grown over time and is very helpful to minimize the effort needed. Essentially, we just want to say, “use this image and let me apply a shader.”

Camera and geometry

The camera and geometry setup is also rather easy:

// camera
const CAMERA_POS = 500
const calcFov = (CAMERA_POS) => 2 * Math.atan((window.innerHeight / 2) / CAMERA_POS) * 180 / Math.PI

const camera = new THREE.PerspectiveCamera(50, window.innerWidth / window.innerHeight, 10, 1000)
camera.position.z = CAMERA_POS
camera.fov = calcFov(CAMERA_POS)
camera.updateProjectionMatrix()

// geometry and material
geometry = new THREE.PlaneGeometry(1, 1, 100, 100)
material = new THREE.ShaderMaterial({
  uniforms: {
    uResolution: { value: new THREE.Vector2(window.innerWidth, window.innerHeight) },
    uTime: { value: 0 },
    uCursor: { value: new THREE.Vector2(0.5, 0.5) },
    uScrollVelocity: { value: 0 },
    uTexture: { value: null },
    uTextureSize: { value: new THREE.Vector2(100, 100) },
    uQuadSize: { value: new THREE.Vector2(100, 100) },
    uBorderRadius: { value: 0 },
    uMouseEnter: { value: 0 },
    uMouseOverPos: { value: new THREE.Vector2(0.5, 0.5) }
  },
  vertexShader: effectVertex,
  fragmentShader: effectFragment,
  glslVersion: THREE.GLSL3
})

An important aspect is the small math trick to calculate the field of view (FOV), which allows us to set the positions of the meshes based on the HTML dimensions and bounds. While I can’t go into every detail here, I highly encourage you to study the file and follow along step by step. It’s quite manageable and will give you a deeper understanding of the process.

The render loop

What’s left is the render loop:

// render loop
const render = (time = 0) => {
  time /= 1000

  mediaStore.forEach((object) => {
    if (object.isInView) {
      object.mouseOverPos.current.x = lerp(object.mouseOverPos.current.x, object.mouseOverPos.target.x, 0.05)
      object.mouseOverPos.current.y = lerp(object.mouseOverPos.current.y, object.mouseOverPos.target.y, 0.05)

      object.material.uniforms.uResolution.value.x = window.innerWidth
      object.material.uniforms.uResolution.value.y = window.innerHeight
      object.material.uniforms.uTime.value = time
      object.material.uniforms.uCursor.value.x = cursorPos.current.x
      object.material.uniforms.uCursor.value.y = cursorPos.current.y
      object.material.uniforms.uScrollVelocity.value = scroll.scrollVelocity
      object.material.uniforms.uMouseOverPos.value.x = object.mouseOverPos.current.x
      object.material.uniforms.uMouseOverPos.value.y = object.mouseOverPos.current.y
      object.material.uniforms.uMouseEnter.value = object.mouseEnter
    } else {
      object.mesh.position.y = 2 * window.innerHeight
    }
  })

  setPositions()

  renderer.render(scene, camera)

  requestAnimationFrame(render)
}

In this part, I update the uniforms and set the positions of the meshes to sync them with the native HTML images. You’ll notice that I only update uniforms for meshes that are in view and place those out of view outside the viewport. These improvements developed over time, so you don’t have to go through the same trial and error process 😀

You can find the entire file to study, copy, and modify here.

The shader magic

Let’s get to the fun part! Let’s explore the two base shader files.

Base vertex shader

uniform vec2 uResolution; // in pixel
uniform float uTime; // in s
uniform vec2 uCursor; // 0 (left) 0 (top) / 1 (right) 1 (bottom)
uniform float uScrollVelocity; // - (scroll up) / + (scroll down)
uniform sampler2D uTexture; // texture
uniform vec2 uTextureSize; // size of texture
uniform vec2 uQuadSize; // size of texture element
uniform float uBorderRadius; // pixel value
uniform float uMouseEnter; // 0 - 1 (enter) / 1 - 0 (leave)
uniform vec2 uMouseOverPos; // 0 (left) 0 (top) / 1 (right) 1 (bottom)

#include './resources/utils.glsl';

out vec2 vUv;  // 0 (left) 0 (bottom) - 1 (top) 1 (right)
out vec2 vUvCover;


void main() {
  vUv = uv;
  vUvCover = getCoverUvVert(uv, uTextureSize, uQuadSize);

  gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
}

Here you can see the uniforms I always include to avoid worrying about the basic setup and interactions, allowing me to dive straight into experimenting. Not all of them are always needed or used, but I will briefly explain each one:

  • uResolution: The resolution of the window size.
  • uTime: Represents the passage of time.
  • uCursor: The position of the cursor relative to the window.
  • uScrollVelocity: A value from the Lenis library, often used for smooth scrolling.
  • uTexture: The image.
  • uTextureSize: The dimensions of the image, needed to calculate an object-fit: cover behavior, for example.
  • uQuadSize: The size of the displayed element, the dimensions you see at the end.
  • uBorderRadius: Allows applying a border radius to images in HTML, which is also applied in the shader (additional code in the fragment shader is needed, link here).
  • uMouseEnter: A value that smoothly interpolates from 0 to 1 (on mouse enter of the image) or 1 to 0 (on mouse leave of the image). You can find how this is done in the JS file and how it’s passed to the shader.
  • uMouseOverPos: The position of the cursor relative to the image.
  • getCoverUvVert: A function to apply an ‘object-fit: cover’ behavior.

Base fragment shader

precision highp float;

uniform vec2 uResolution; // in pixel
uniform float uTime; // in s
uniform vec2 uCursor; // 0 (left) 0 (top) / 1 (right) 1 (bottom)
uniform float uScrollVelocity; // - (scroll up) / + (scroll down)
uniform sampler2D uTexture; // texture
uniform vec2 uTextureSize; // size of texture
uniform vec2 uQuadSize; // size of texture element
uniform float uBorderRadius; // pixel value
uniform float uMouseEnter; // 0 - 1 (enter) / 1 - 0 (leave)
uniform vec2 uMouseOverPos; // 0 (left) 0 (top) / 1 (right) 1 (bottom)

in vec2 vUv; // 0 (left) 0 (bottom) - 1 (right) 1 (top)
in vec2 vUvCover;

out vec4 outColor;


void main() {
  // texture
  vec3 texture = vec3(texture(uTexture, vUvCover));

  // output
  outColor = vec4(texture, 1.0);
}

There’s nothing really special here; it’s the same uniforms. You can see that I like to use GLSL3 because the varyings are written with the in and out keywords, and I’ve defined the outColor variable.
With the vUvCover coordinates from the vertex shader, we can show the texture with the cover behavior in just two lines.

At this point, we have HTML images displayed with WebGL, and their positions are completely synced. Now we can apply whatever effects we like.

It took some time, but I wanted to explain the behind-the-scenes a bit. Now, this solid base makes everything from now on much more fun and enjoyable.

The effect

Interestingly enough, this isn’t the longest part of the tutorial, as the effect itself isn’t that complex. But let’s walk through it.

First, let’s focus on the curve when scrolling. Since we need to adjust the vertices, we’ll work in the vertex shader.

vec3 deformationCurve(vec3 position, vec2 uv) {
  position.y = position.y - (sin(uv.x * PI) * uScrollVelocity * -0.01);
  
  return position;
}

This little function does the magic. We modify the vertices with a sin() function based on the uv.x values. The multiplication with PI (needs to be defined at the top of the file with float PI = 3.141592653589793;) makes the curve exactly one perfect bow. Feel free to remove it or change it to 2*PI to see what it does.

To apply the effect on scrolling, or based on the scroll intensity, we just need to multiply it by the uScrollVelocity uniform and adjust the factor (and direction with positive/negative signs). See how all that work earlier makes this super easy now? Yay!

Because the distortion got really big when scrolling really fast, I introduced a maximum, which is very easy to implement with the min() function. The only thing to consider here is that this will always output a positive value in the way I used it, so I had to multiply by the sign of the uScrollVelocity again to get back the bow in the direction of scrolling. GLSL has the sign() function for that, so it’s straightforward.

position.y = position.y - (sin(uv.x * PI) * min(abs(uScrollVelocity), 5.0) * sign(uScrollVelocity) * -0.01);

Now we just have to subtract that calculated value from the original y position, and we have the first part of our effect.

Here’s the final complete vertex shader:

float PI = 3.141592653589793;

uniform vec2 uResolution; // in pixel
uniform float uTime; // in s
uniform vec2 uCursor; // 0 (left) 0 (top) / 1 (right) 1 (bottom)
uniform float uScrollVelocity; // - (scroll up) / + (scroll down)
uniform sampler2D uTexture; // texture
uniform vec2 uTextureSize; // size of texture
uniform vec2 uQuadSize; // size of texture element
uniform float uBorderRadius; // pixel value
uniform float uMouseEnter; // 0 - 1 (enter) / 1 - 0 (leave)
uniform vec2 uMouseOverPos; // 0 (left) 0 (top) / 1 (right) 1 (bottom)

#include './resources/utils.glsl';

out vec2 vUv;  // 0 (left) 0 (bottom) - 1 (top) 1 (right)
out vec2 vUvCover;


vec3 deformationCurve(vec3 position, vec2 uv) {
  position.y = position.y - (sin(uv.x * PI) * min(abs(uScrollVelocity), 5.0) * sign(uScrollVelocity) * -0.01);

  return position;
}

void main() {
  vUv = uv;
  vUvCover = getCoverUvVert(uv, uTextureSize, uQuadSize);

  vec3 deformedPosition = deformationCurve(position, vUvCover);

  gl_Position = projectionMatrix * modelViewMatrix * vec4(deformedPosition, 1.0);
}

Let’s move over to the fragment shader.

Here we will work on the grain/noise effect. For the noise function, you can search for one online. I used this one.

We can create noise by simply calling the function:

float noise = snoise(gl_FragCoord.xy);

And display it with:

vec3 texture = vec3(texture(uTexture, vUvCover));
outColor = vec4(texture * noise, 1.0);

To get the final effect, I applied the noise based on a circle that follows the cursor position.

  // aspect ratio needed to create a real circle when quadSize is not 1:1 ratio
  float aspectRatio = uQuadSize.y / uQuadSize.x;

  // create a circle following the mouse with size 15
  float circle = 1.0 - distance(
    vec2(uMouseOverPos.x, (1.0 - uMouseOverPos.y) * aspectRatio),
    vec2(vUv.x, vUv.y * aspectRatio)
  ) * 15.0;

We need the aspect ratio of the displayed image because we always want a perfect circle (not an ellipse) if the image doesn’t have a 1:1 aspect ratio. We can create a circle quite simply with a distance function. If we pass in the uMouseOverPos to calculate the distance from, it will automatically follow the cursor position. You can always “debug” the steps by outputting, for example, the circle value as outColor to see what it does and how it behaves.

Now we only need to combine the noise with the circle and apply it to the texture:

  vec2 texCoords = vUvCover;
  
  // modify texture coordinates
  texCoords.x += mix(0.0, circle * noise * 0.01, uMouseEnter + uScrollVelocity * 0.1);
  texCoords.y += mix(0.0, circle * noise * 0.01, uMouseEnter + uScrollVelocity * 0.1);

  // texture
  vec3 texture = vec3(texture(uTexture, texCoords));

  // output
  outColor = vec4(texture, 1.0);

I know these lines look a bit complex at first, but let’s break it down; it’s not too hard.

At the end, we want to modify the UVs coming from the vertex shader as vUvCover. The effect itself is simply circle * noise * 0.01. You can try applying just that, and the effect will always be applied and visible. To make it visible only on scroll and/or mouse over, we can leverage the mix function (I love that function :D).

Basically, we say either add 0, or circle * noise * 0.01 to the texCoords based on the third parameter. Now, we only need this third parameter to reflect the interaction we want. Thanks to our outstanding preparation of the uniforms, we can just use uMouseEnter (which interpolates to 1 on enter) and uScrollVelocity as this parameter.

Now, if we either enter the image with the cursor or scroll the page, the mix function will result in circle * noise * 0.01, and we’ll get the effect we want. If we don’t interact, it will result in 0, thus not adding anything to the texCoords, and we’ll see only the image.

Here’s the final complete fragment shader:

precision highp float;

uniform vec2 uResolution; // in pixel
uniform float uTime; // in s
uniform vec2 uCursor; // 0 (left) 0 (top) / 1 (right) 1 (bottom)
uniform float uScrollVelocity; // - (scroll up) / + (scroll down)
uniform sampler2D uTexture; // texture
uniform vec2 uTextureSize; // size of texture
uniform vec2 uQuadSize; // size of texture element
uniform float uBorderRadius; // pixel value
uniform float uMouseEnter; // 0 - 1 (enter) / 1 - 0 (leave)
uniform vec2 uMouseOverPos; // 0 (left) 0 (top) / 1 (right) 1 (bottom)

in vec2 vUv; // 0 (left) 0 (bottom) - 1 (right) 1 (top)
in vec2 vUvCover;

#include './resources/noise.glsl';

out vec4 outColor;


void main() {
  vec2 texCoords = vUvCover;

  // aspect ratio needed to create a real circle when quadSize is not 1:1 ratio
  float aspectRatio = uQuadSize.y / uQuadSize.x;

  // create a circle following the mouse with size 15
  float circle = 1.0 - distance(
    vec2(uMouseOverPos.x, (1.0 - uMouseOverPos.y) * aspectRatio),
    vec2(vUv.x, vUv.y * aspectRatio)
  ) * 15.0;

  // create noise
  float noise = snoise(gl_FragCoord.xy);

  // modify texture coordinates
  texCoords.x += mix(0.0, circle * noise * 0.01, uMouseEnter + uScrollVelocity * 0.1);
  texCoords.y += mix(0.0, circle * noise * 0.01, uMouseEnter + uScrollVelocity * 0.1);

  // texture
  vec3 texture = vec3(texture(uTexture, texCoords));

  // output
  outColor = vec4(texture, 1.0);
}

Disclaimer

I think this is a very robust and solid base for creating such effects, but I also know that there are many more talented developers out there. If you have any ideas to optimize things or know better ways, I’m always open to hearing about them. Likewise, if something is not clear or if I can help with anything, I’m here for that as well.

Real-World-Shader Project

Thank you for reading this tutorial! I’m also currently working on a fun little project I call Real-World-Shader, where I aim to collect examples of nice shader effects that are useful in real client projects, all in one place. I’d love to hear your feedback and suggestions, or even contributions. Feel free to check it out here: https://real-world-shader.jankohlbach.com/

How to Create a Liquid Raymarching Scene Using Three.js Shading Language