VideoTexture / useVideoTexture

  • Video textures
    Video textures
  • useVideoTexture - url and mediaStream
    useVideoTexture - url and mediaStream

Summary

A convenience hook that returns a THREE.VideoTexture and integrates loading into suspense.

By default it falls back until the loadedmetadata event. Then it starts playing the video, which, if the video is muted, is allowed in the browser without user interaction.

export function useVideoTexture(
  srcOrSrcObject: HTMLVideoElement['src' | 'srcObject'],
  {
    unsuspend = 'loadedmetadata',
    start = true,
    hls = {},
    crossOrigin = 'anonymous',
    muted = true,
    loop = true,
    playsInline = true,
    onVideoFrame,
    ...videoProps
  }: {
    unsuspend?: keyof HTMLVideoElementEventMap
    start?: boolean
    hls?: Parameters<typeof getHls>[0]
    onVideoFrame: VideoFrameRequestCallback
  } & Partial<Omit<HTMLVideoElement, 'children' | 'src' | 'srcObject'>> = {}
)
const texture = useVideoTexture("/video.mp4")
return (
  <mesh>
    <meshBasicMaterial map={texture} toneMapped={false} />

MediaStream

It also accepts a MediaStream from eg. .getDisplayMedia() or .getUserMedia():

const [stream, setStream] = useState<MediaStream | null>(null)

return (
  <mesh onClick={async () => setStream(await navigator.mediaDevices.getDisplayMedia({ video: true }))}>
    <React.Suspense fallback={<meshBasicMaterial wireframe />}>
      <VideoMaterial src={stream} />
    </React.Suspense>
function VideoMaterial({ src }) {
  const texture = useVideoTexture(src)

  return <meshBasicMaterial map={texture} toneMapped={false} />
}

NB: It's important to wrap VideoMaterial into React.Suspense since, useVideoTexture(src) here will be suspended until the user shares its screen.

HLS

useVideoTexture supports .m3u8 HLS manifest via hls.js:

const texture = useVideoTexture('https://test-streams.mux.dev/x36xhzz/x36xhzz.m3u8')

You can pass hls config:

const texture = useVideoTexture('https://test-streams.mux.dev/x36xhzz/x36xhzz.m3u8', {
  hls: { abrEwmaFastLive: 1.0, abrEwmaSlowLive: 3.0, enableWorker: true },
})

requestVideoFrameCallback (rVFC)

useVideoTexture supports requestVideoFrameCallback:

useVideoTexture(src, {
  onVideoFrame: (now, metadata) => {}
})

<VideoTexture> Component

export type VideoTextureProps = {
  children?: (texture: THREE.VideoTexture) => React.ReactNode
  src: UseVideoTextureParams[0]
} & UseVideoTextureParams[1]

You can access the texture via children's render prop:

<VideoTexture src="/video.mp4">
  {(texture) => <meshBasicMaterial map={texture} />}

or exposed via ref:

const textureRef = useRef()
<VideoTexture ref={textureRef} src="/video.mp4" />