PDA

View Full Version : Lip Sync Software



Steamthrower
01-09-2008, 08:30 AM
A pretty complex project is coming up involving character animation, so please bear with me as I will probably be posting a lot of questions regarding the finer points of Lightwave CA...

Right now I am just doing some simple tests with lip syncing.. Before now the only paying gigs/jobs I've had are arch viz and industrial sim. So...all this is painfully new to me. I'm looking for a program that can import audio and show the phoneme shapes that a mouth requires. I don't even necessarily want it to be integrated into Lightwave per se; just something to use as reference.

Do you have any tips on a software like that? Any other comments regarding lip syncing, mouth rigs, etc. would be appreciated.

benedict_ilagan
01-09-2008, 08:48 AM
as for the reference part, surf the net for all possible phoneme mouth shape pictures that may guide you through. Under lightwave, audio can be imported in and scrub into timeline.... you have the option of using either the morph mixer nor the motion mixer as your prefered settings.....

Steamthrower
01-09-2008, 09:09 AM
The main problem I have is not the mere shapes of the mouth, but the shapes of the mouth over a period of time. I've tried using actual photos, but as this particular project is cartoonish, I've found that I need exaggerated stretches and dimples.

Thanks though, I'll take your advice and find some image sequences to follow along with.

SplineGod
01-09-2008, 11:48 AM
mimic works pretty well. :)

goodrichm
01-10-2008, 09:14 AM
SplineGod,

What version of Mimic & LW are you using?

On my PC, LW9.3.1 (build 1291 win32) crashes when I send animation data from DAZ Mimic2 Pro ver 2.5.0.6.

This is with a fresh install of LW9.3.1 and Mimic is the only 3rd party plugin installed. I have not installed any betas on this PC and the LW config files are saved to their own 9.3.1 config directory. The LW Hub is disabled.

Mimic does not crash with LW9.2...MG

DiedonD
01-11-2008, 01:13 AM
The main problem I have is not the mere shapes of the mouth, but the shapes of the mouth over a period of time. I've tried using actual photos, but as this particular project is cartoonish, I've found that I need exaggerated stretches and dimples.

Thanks though, I'll take your advice and find some image sequences to follow along with.

Hey there Inigo

You can easily exagerate mouth moves in Morph Mixer by increasing the size above 100% at either poles even. I think for the cartoonish character that may have irregular mouth moves in irregular times, Morph Mixer would be ideal tool to use for you since, youll be going through manual mouth to voice lip syncing. Also consider Sliders.

If you choose to use Mimic however. It may help you finishing off long speeches (Mind you it will only help perhaps, may not finish it completely), but I was never a fan of its accuracy, and it wont finish the sentences completely too. What Ive seen is also that it also misses a few frames, making lips move later or previous to its actual letter in voice. Also, if your character makes funny mouth shapes irregular from time to time, as you mention, Mimic would do the same mouth shape you told it to do over and over whenever it occurs, to the best it can "understand" and link accordingly. So its same mouth shape for all, that you may need to exagerate later in MM.

So then... depending on the length of your characters sentences, and depending how specific details and overall quality you want to add to your characters movements, I say that for the time beeing, you have only those two options above...sadly.

Steamthrower
01-11-2008, 07:00 AM
Thanks Diedond for the tips. The sentences are pretty short and choppy and there's not that much dialogue in the clips that I need.

DiedonD
01-12-2008, 12:17 AM
Thanks Diedond for the tips. The sentences are pretty short and choppy and there's not that much dialogue in the clips that I need.

Then Id go for MM or Sliders all the way. Especialy the Graph Editor, the envelope within it per morph.
Its like color painting. You have a few basic colors that you can mix to get a new one. Its roughly the same. You should have basic mouth morphs, that then you can mix in MM and come out with a special morph for you character.

righteous
01-12-2008, 04:01 AM
Timothy Albee has a great little program called the Timothy Albee Facial Animator. I had a play around with the demo and thought if I was every going to do a character animation I would probably use that.

I don't know anything about facial animation, but that was pretty intuitive for me at least.

http://ta-animation.com/FA/

Tom Wood
01-12-2008, 05:58 AM
Thanks Diedond for the tips. The sentences are pretty short and choppy and there's not that much dialogue in the clips that I need.

You might be able to just fake it. That's what I'm doing. I used Magpie to carefully create a waveform for each bit of dialogue, then one day I loaded the wrong one. It didn't make much difference. So now I just use a generic random looking waveform to drive the mouth morphs for all my character dialogue. It works most of the time, and the parts where it doesn't I'm calling it an endearing quirk. :D

Steamthrower
01-21-2008, 02:31 PM
Seeing as it's cartoonish anyway, that might just work...thanks for the advice.

Actually, watching The Incredibles again, I looked pretty closely at the mouths and lips. And...wonder of wonders...they don't move that much. They kind of just slide around. That's what I hate about doing 3D, you can't stop putting your favorite movies under the microscope...

So thanks. I'll try it.

rdolishny
01-21-2008, 03:18 PM
Actually, watching The Incredibles again, I looked pretty closely at the mouths and lips. And...wonder of wonders...they don't move that much.

I noticed the same thing when making my little lip-sync cartoon (below). It's pretty crude but I realized that with smaller motions of the mouth and equally controlled eye motions I was able to hack together something with character.

http://www.youtube.com/watch?v=G52bOsh7Phk

Some of the software that automatically generate morph/shape targets tends to be a bit literal. In other words, keep it simple. Honestly, in about 2 hours I doped out the mouth motions for my entire 60 second film. Lightwave has a surprisingly good implementation of WAV scrubbing.

- Rick

mrpapabeis
01-21-2008, 09:46 PM
Here's one done in Mimic. Some clean up is needed in the graph editor after use. But it gets you 80% there.

Yes I know;

1) It's a Daz character. I didn't model him myself.

2) His right hand needs more snap in the gesture.

3) His left has some keyframe drift.

4) His right eyebrow pops.



BTW the voice is that of Joe LaLa.

http://www.joelala.com/filmography.htm

He does some awesome voices.

GP