FaceControls

Summary

The camera follows your (detected) face.

  • FaceControls (forked)
    FaceControls (forked)
  • FaceControls - AF (forked)
    FaceControls - AF (forked)

Prerequisite: wrap into a FaceLandmarker provider

<FaceLandmarker>...</FaceLandmarker>
<FaceControls />
export type FaceControlsProps = {
  /** The camera to be controlled */
  camera?: THREE.Camera
  /** VideoTexture or WebcamVideoTexture options */
  videoTexture: VideoTextureProps
  /** Disable the automatic face-detection => you should provide `faceLandmarkerResult` yourself in this case */
  manualDetect?: boolean
  /** FaceLandmarker result */
  faceLandmarkerResult?: FaceLandmarkerResult
  /** Disable the rAF camera position/rotation update */
  manualUpdate?: boolean
  /** Reference this FaceControls instance as state's `controls` */
  makeDefault?: boolean
  /** Approximate time to reach the target. A smaller value will reach the target faster. */
  smoothTime?: number
  /** Apply position offset extracted from `facialTransformationMatrix` */
  offset?: boolean
  /** Offset sensitivity factor, less is more sensible */
  offsetScalar?: number
  /** Enable eye-tracking */
  eyes?: boolean
  /** Force Facemesh's `origin` to be the middle of the 2 eyes */
  eyesAsOrigin?: boolean
  /** Constant depth of the Facemesh */
  depth?: number
  /** Enable debug mode */
  debug?: boolean
  /** Facemesh options, default: undefined */
  facemesh?: FacemeshProps
}

export type FaceControlsApi = THREE.EventDispatcher & {
  /** Compute the target for the camera */
  computeTarget: () => THREE.Object3D
  /** Update camera's position/rotation to the `target` */
  update: (delta: number, target?: THREE.Object3D) => void
  /** <Facemesh> ref api */
  facemeshApiRef: RefObject<FacemeshApi>
}

Breaking changes

9.120.0

FaceControls was simplified.

Following props were deleted:

  • autostart: now use videoTexture.start
  • webcam: instead of webcam: false, you can now manualDetect
  • webcamVideoTextureSrc: now use videoTexture.src (or instantiate your own video-texture1 outside)
  • onVideoFrame: now use videoTexture.onVideoFrame (or instantiate your own video-texture1 outside)

Following api methods/fields were deleted:

  • detect: you can now manualDetect outside and pass faceLandmarkerResult
  • webcamApiRef: if you need videoTextureRef, instantiate your own video-texture1 outside
  • play/pause: same, if you need the video object, instantiate your own video-texture1 outside

Footnotes

  1. <VideoTexture> or <WebcamVideoTexture> 2 3 4