imagination XD

realizing dream

[Unity] Realtime Lipsync

1 Comment


Back on my manual lipsync using Tagarela. This one is faster to do and also easier, but of course, you’ll get only Anime lipsync (Anime lipsync only have 1 open mouth, there’s no O, U, I or any other mouth shape than A).

The method I’m using is just get Audio Source frequency. So basically it’s like music visualisation but for mouth.

What you need

  • Head mesh with 1 mouth blendshape
  • Voice
  • Audio Source

Here’s the script you should attach to head mesh.

using UnityEngine;
using System.Collections;

public class RealtimeLipsync : MonoBehaviour {

//LIPSYNC CORE
 [Header("Lipsync")]
 public AudioSource audioSource;
 public int mouthBlendShape;
 private float[] _samples = new float[64];
 private float clampedLipsync = 0f;

//PRIVATES
 private SkinnedMeshRenderer blendMesh;

void Awake(){
 blendMesh = GetComponent<SkinnedMeshRenderer>();
 }

void LateUpdate(){
 if(audioSource) DoLipsync();
 }

void DoLipsync(){
 if(audioSource.isPlaying){
 audioSource.GetSpectrumData(_samples,0,FFTWindow.BlackmanHarris);

clampedLipsync = Mathf.Clamp(_samples[2]*3000,0,100);

blendMesh.SetBlendShapeWeight(mouthBlendShape,clampedLipsync);
 }
 }
}

And here’s how to do it

  1. Attach this script to your head mesh.
  2. Also don’t forget to add audio source and assign audio source in this script. You can put Audio Source in whereever you want, but remember to assign audio source variable on this script.
  3. To play sound, use like audioSource.clip = <your clip name>; then audioSource.Play();
  4. Assembly-CSharp - VertexScriptCOREPlayerCORE.cs - MonoDevelop-Unity
  5. Here’s my example to use this. I put it on my player script. It’ll automatically works whenever you have audioSrc.clip = clipName; audioSrc.Play();
  6. That’s it, your mouth will move whenever the audio src playing sounds.

 

Advertisements

Author: IMGVERTEX

Started making games since 13 year old. 3D CGI Weeaboo | VFX Otaku | Anime Researcher | Indie Game Dev | VR Chuunibyou

One thought on “[Unity] Realtime Lipsync

  1. If you want unityscript version, ask me XD

FEED ME

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s