PDA

View Full Version : syncing voices to 3D characters



Sentinel45
04-26-2009, 06:34 AM
Are there any tricks to syncing a voice to the lip movements of 3D characters? Right now I've been using an actor and putting it in the background of the character, and using setting keyframes to the lip movements of the actor. It doesn't seem to be working too well and it slows down my system.

There are some samples on my youtube site: www.youtube.com/kodosIV

Any thoughts would be appreciated.

RebelHill
04-26-2009, 07:08 AM
practice, basically...

Aim firstr to hit the main readable mouth shapes... these are Eee, Ooo, Aaa and the consonants, Mmm, Ppp, Bbb, Fff, Lll, and Thh..

Block in with these main shapes first, then tween as necessary to build it up.

Sentinel45
04-26-2009, 07:23 AM
practice, basically...

Aim firstr to hit the main readable mouth shapes... these are Eee, Ooo, Aaa and the consonants, Mmm, Ppp, Bbb, Fff, Lll, and Thh..

Block in with these main shapes first, then tween as necessary to build it up.

I knew I should have trusted my first instinct, if I am reading you correctly. When I first started experimenting I thought of doing the endomorphs phonetically but I thought I could create some basic lip morphs and do everything by manipulating them. Thanks for the info. I'm going to try your suggestion.

RebelHill
04-26-2009, 08:34 AM
Im not saying create the morfs phonetically... theres a couple schools of thought, one where u create the phoneme shapes and use those, another where you create the shape variety and mix them to construct ur phonemes (i go with the latter)... what Im saying is when u animate... to first go through and just hit those main shoundshapes... dont worry about the other stuff in between... focus just on those main ones for your first pass... then once thats solid, it should read well, even if the inbetween stuff is floating a bit... If it doesnt read at that first stage, then no amount of tweening will help u... but once that first pass is good, then tween in to tighten up timing, and put in other shapes where they become necessary.

stevecullum
04-26-2009, 08:44 AM
Another good practice with lip syncing is to think about the speed of light and sound. You will always see something before hearing it, so I used to slip my sound back 2 - 3 frames after creating the phoneme animation, to compensate.

RebelHill
04-26-2009, 08:56 AM
Umm... sound travels at around 700mph... for their to be a 3 frame (1/8th sec) lag between seeing something and hearing it, youd need to be around 30 odd metres away...

stevecullum
04-26-2009, 11:48 AM
Umm... sound travels at around 700mph... for their to be a 3 frame (1/8th sec) lag between seeing something and hearing it, youd need to be around 30 odd metres away...

Just from my personal experience, if I animated dead on the sound file, it never looked natural, but when I dropped the sound back a couple of frames it looked a lot better. Whether is actually has anything to do with the speed of light and sound was only an assumption to comprehend why that was working for me....maybe I was just inaccurate in the original sync! :hey:

RebelHill
04-26-2009, 12:09 PM
We make mouth shapes before we make the sound... try saying "peeee"... u put ur lips together first, then spit out the sound About a frame or 2)... and thats true of many of the hard consonants, but usually not of the soft vowels which usually reach their extreme shape at the same moment as the high point of the sound itself.

SplineGod
04-26-2009, 05:37 PM
Heres a chart with basic phonemes that RebelHill was referring to. Use this as a basic guide to create the basic mouth shapes on your character.
Heres an example. You can find other examples pretty easily on the web or in different animation books:
http://fc04.deviantart.com/fs45/f/2009/079/f/7/Phoneme_Mouth_Chart_by_CartoonistWill.png

Once you have a wav file of the spoken text you need to animate to it needs to be broken down into small parts or phonemes. Heres a couple of free but good applications that allow you to do that. They also have built in mouth shape charts and are extremely useful for getting the tiiming right which you can apply back to your character in LW.
Mike Green, aka "dodgy" has a script to convert the papagayo data to a morph mix project in LW.
http://jlipsync.lamhauge.dk/
http://www.lostmarble.com/papagayo/index.shtml
Heres also some good info on Lip sync and it explains a bit more
about Steves suggestion about sliding the timing on teh mouth shapes.
http://www.keithlango.com/tutorials/old/lipSync.htm

paulhart
04-26-2009, 08:48 PM
RebelHill, Larry Schultz, Thank you both, great resources, always helpful.

Carm3D
04-27-2009, 05:34 AM
I will agree that the mouth shape should happen a frame or two before the sound is made.. This has nothing to do with the speed of sound, however.

There was a great tool out there I used to use called JoyTrol. Unfortunately it was only compatible with Lightwave 8 (don't believe the '8 or later' on their website). It did all of those cool joystick controls, plus it came with MorphTrack. All for $20. Such a pity... I do miss using that plugin. The joystick controls can be made by hand, but the plugin had some nice features such as double-click to recenter the controller.

DigiLusionist
06-19-2009, 01:35 AM
Papagayo is awesome! It is the fastest lip synching app I've ever used. I wish there was an exporter to Lightwave for it.

SplineGod
06-19-2009, 01:39 AM
Like this?
http://www.mikegreen.name/Lscripts.html#Papagayo_Importer

DigiLusionist
06-19-2009, 01:55 AM
!o.0!

And, verily, the sun rose up in the east, and Moses said to the Promised Ones, "There! There be Papagayo for LW!"

Giggling like a school boy...

SplineGod
06-19-2009, 03:25 AM
yea verily! :)