PDA

View Full Version : Lip Sync software?



sami
03-16-2012, 06:23 AM
Just wondering what dopesheet/mouth shape/speech recognition/etc software people use to assist them with the laborious tasks of lipsync.

Does anyone use MagPie Pro (http://www.thirdwishsoftware.com/magpiepro.html) - lately I mean?
Their site looks quite dated and I'm wondering if it's abandonware that gets no development love. Ideally I'd like some software that is still alive.

What other software do you use for this, and how is it?

sami
03-16-2012, 07:12 AM
I just found and tried out Papagayo which is very simplistic, and can only export mouth shapes and frame #s, but I was surprised at how quick it was to visualize basic cartoon lip sync and get the phoneme positions. Might be worth a look - plus its free and quick to try.

http://www.lostmarble.com/papagayo/index.shtml

nikfaulkner
03-16-2012, 07:28 AM
its great, mike green has an lscript to import the data into lightwave. as long as your morphs are named correctly it works a treat.

http://www.mikegreen.name/Lscripts.html#Papagayo_Importer

Robi
03-16-2012, 07:46 AM
Hi,
I like this one very much: http://ta-animation.com/FA/
Its from Timothy Albee and it works great with Lightwave.
The fastest Lipsync software I know.

Robert

sami
03-16-2012, 07:57 AM
its great, mike green has an lscript to import the data into lightwave. as long as your morphs are named correctly it works a treat.

http://www.mikegreen.name/Lscripts.html#Papagayo_Importer

Cool, thanks for that link - that should save alot of work! Btw, your avatar is hilarious.

sami
03-16-2012, 08:06 AM
Hi,
I like this one very much: http://ta-animation.com/FA/
Its from Timothy Albee and it works great with Lightwave.
The fastest Lipsync software I know.

Robert

Thanks, Is that still being developed? I couldn't get to the forums for it and the site looks really stale like I remember seeing it many years ago. When was the last update? I guess it doesn't matter if it works and is solid, but I tend to feel more comfortable using software when I know there is somebody who can fix something or will show the app love once in awhile...

robyht
03-16-2012, 01:56 PM
TAFA is great, we use it for all our lip-sync, it is a little old now but is still totally usable; I do wish that Mac Reiter and Timothy Albee would get going on TAFA 2 it would be such a powerful/cool tool if it was updated.

DigitalSorcery8
03-16-2012, 03:48 PM
TAFA is discussed in this recent thread:

http://forums.newtek.com/showthread.php?t=126523

IMO, it is FARE better than ANYTHING "out there" available for lipsyncing. Super fast and super easy to use. Top notch! :thumbsup:

Greenlaw
03-16-2012, 05:57 PM
I use Magpie Pro. The program has been around a long time but it's still supported and the workflow is quite good. The output is ideal for both 3D and 2D lipsync, meaning you can use it to output Endormorph data or 2D image sequences. The program allows you to preview with live .lwo objects with endomorphs, with pre-rendered 2D images or with hand drawn 'cels'. Many stop motion artists also use Magpie Pro to generate lipsync reference sheets.

My wife and I used Magpie Pro to generate the 2D mouthshapes for the 3D characters in our first Brudders short Happy Box (http://www.youtube.com/watch?v=gy3hSz7_zb8&feature=plcp&context=C413adffVDvjVQa1PpcFMByTDb0Ua2xf5Ws9jP5yDe fzFwtn9tXUQ%3D). When we made this movie last fall, we imposed an extremely tight schedule and we literally allowed ourselves only a few minutes to lipsync each shot. All things considered, I was pleased with the results. My standard was "if it looks better than Robot Chicken, we're golden." (Not that I have anything against Robot Chicken--I absolutely love that show!) :)

Now the bad news: I don't know if other Magpie Pro users have this problem but for us the program crashed a lot during during production, and I do mean A LOT. It was especially sensitive anytime we scrubbed the timeline. This problem existed on the two different Windows 7 Pro computers that we had licenses for, a quad core x64 desktop and a duo core x64 convertible tabletPC. It's possible that the problem is OS related since MP is officially supported through Vista, and Win 7 isn't mentioned. Unoffically, the developer says it should run on Win 7 though.

The developer is actually very responsive and helpful and I have no complaints about the company. However, he was not able to reproduce our crashes on his own Windows 7 system. I really don't know what's up, and needless to say, it got a bit frustrating at the end. We kept with it though because, when Magpie Pro wasn't crashing on us, the shots were getting done unbelievably fast.

Anyway, we're going to start lip-syncing our second Brudders short soon and I'm thinking of transferring our two Magpie Pro licenses to a quad core x64 Vista desktop machine and an old duo core x32 XP Pro laptop just to see if MP is more stable on either of those computers. If not, I guess it will be time to move on.

Our second option at the moment is to use Toon Boom Animate Pro, which isn't meant for 3D at all but it can output 2D image sequences. This might work out fine for Brudders lipsync, since we're basically using Magpie in 2D mode anyway. Papagayo is another option though I think the allowed number of mouth shapes may be too limited for us. I'll have to revisit--it's been a few years since I tried it.

Our third animated short is going to be a full 3D project and we will need to lipsync animated morph targets this time. If Magpie Pro fails us there, we're probably going to switch to TAFA or Rebel Hill's system. These two systems are not technically 'lip sync' programs in the sense that Magpie is--Magpie's workflow and interface is very much 'dopesheet' oriented, and it's designed more for speed than for generating fluid facial animations. That said, if you need higher quality, you can use MP to quickly generate a 'first pass', output the data as an Endomorph file and then refine later in Lightwave. Assuming you're not crashing all the time like we did. But even without the 'dopesheet' interface, animated mouthshapes with TAFA or Rhiggit! looks pretty fast.

BTW, Magpie also features an automated mode, meaning you can teach it to recognize spoken phonemes and then have it 'read' your .wav files and generate 'matching' shapes. As with most automated lipsync, the results range from kinda acceptable to kinda bad--the raw results might be useful if you really absolutely have no time to do it right. Otherwise you could spend a lot more time tweaking the automated results than if you just used MP in manual mode to begin with.

Another alternative is DAZ Mimic for Lightwave. I used this plug-in briefly several years ago. It's kind of like a simpler Magpie but very focused on automation. Left fully automated, the results are a bit 'meh', but you can refine the results by manually tweaking the results. Like Magpie, it's designed more for cranking through a lot of stuff quickly, but I didn't think it looked as good as MP's results. It's been a long time since I tried this plug-in and I'm not sure it still works with LW.

Anyway, I'll let you know if we can get Magpie Pro running in a more stable manner. I actually really like using MP and I hope we don't have to give it up.

G.

Greenlaw
03-16-2012, 08:37 PM
Okay, I decided to look into my Magpie Pro stability problem today and just get this over with.

I don't know why I didn't think of this earlier but I tried regressing from the current version 2.2.5 to the previous version 2.1.2, and so far Magpie Pro is behaving itself. I need to test this further to be sure but so far I cannot force a crash in 2.1.2 by scrubbing or stepping frames and it's been almost half an hour. In 2.2.5 this problem could appear in just a few seconds to a few minutes and it was completely reproducible here, even with the demo files that come with Magpie Pro. Version 2.1.2, however, seems much more stable even with our 'Happy Box' files, which are far more complicated than the demo files.

Just to be clear, I'm running Magpie Pro 2.1.2 on one of the Windows 7 Pro computers we used to lipsync 'Happy Box' last fall, not an older OS like Vista or XP.

I'm checking with the developer now to see if I'll be losing any important features by regressing but I don't think I am. So far, it seems like all the features I need are present in 2.1.2.

I'll post again soon when I'm more confident that regressing the software fixes the problem. Maybe we can stick with Magpie Pro after all. Wish me luck! :)

G.

DigitalSorcery8
03-16-2012, 08:56 PM
I got decent results in Mimic Pro for Poser, Mimic Pro for LW and Magpie Pro. Depending on how good your morphs are in your characters, the results using these programs can be good. I found that using the three programs above EACH ONE added too many morphs and you had to go in and do extensive editing to get great results. I did months of testing before I decided to use TAFA. My problem initially with TAFA was I couldn't wrap my head around how it worked - after sitting down with it and watching ALL of the training vids, I imported one of my characters and within just a few minutes I had a character talking and it looked great - FAR better than using the auto-lipsync programs. And this was from someone who had NEVER done any lipsyncing before - EVER. Of course I created just about every morph for phonemes that were needed and in the morph name I included the phoneme word it emulated so it was extremely easy to pick what I needed and drag and drop the morphs onto the TAFA timeline.

IMHO, if you've got background talking with characters that aren't directly in front of the camera, the auto-lipsyncing programs are just fine. But - also IMHO - if you've got a hero character right in the camera, TAFA will give you better results much more quickly than these other programs. Fortunately I believe that there are demos for all of them. Unfortunately you can't save and seriously test them out. Just test them all out and see which one FEELS best with your workflow.

Oh yeah... I spent $495 for TAFA when it first came out. Now I think it's $200. I think it's great even at $495 - VERY much worth the price.

Greenlaw
03-16-2012, 11:05 PM
I agree. When I use Magpie Pro in manual mode I can make sure I have only the mouthshapes I actually want and the result is far preferable to 'auto' mode. And to be honest, using it in manual mode is simply not that much work.

Quick update regarding my Magpie problem: Version 2.1.2 is definitely stable here--can't crash it no matter how hard I try. The problem now is that 2.2.5 has improved Morph Mixer compatibility. I don't actually need this for our current project but will definitely need it for our next one.

Sigh. It's always something isn't it?

Just to be sure, I installed 2.2.5 again and the crashing has returned. Still waiting to hear from the developer.

G.

Greenlaw
03-16-2012, 11:10 PM
By the way, TAFA or any morph based system would not help for the style of lip animation we've been using for Brudders. In this case, Magpie Pro is actually more flexible. Just thought that was worth noting.

G.

DigitalSorcery8
03-16-2012, 11:51 PM
By the way, TAFA or any morph based system would not help for the style of lip animation we've been using for Brudders. In this case, Magpie Pro is actually more flexible. Just thought that was worth noting.

What style is that? Is it 2D?

Greenlaw
03-17-2012, 10:28 AM
What style is that? Is it 2D?

Link in my signature or click here (http://www.littlegreendog.com/movies/happyBox/happyBox.php). The mouths are hand drawn 2D rendered in 3D. I wanted the short to resemble my webcomic (http://www.littlegreendog.com/comics/brudders/brudders055.php) fairly closely and this seemed the best approach for the mouths, not to mention the technique was very fast to do--which was critical because we imposed an extremely tight deadline for this project.

We also felt it made it look a little 'stop-motion-y' (is that a word?) which we liked. :)

My wife and I will probably keep using Magpie Pro 2.1.2 for the current short film project since we don't need the morph capability and it's more stable than the current version 2.2.5. Hopefully there will be another update to Magpie Pro so we an use it on our next film, which will definitely need morphing mouths.

G.

DigitalSorcery8
03-17-2012, 03:09 PM
Link in my signature or click here (http://www.littlegreendog.com/movies/happyBox/happyBox.php). The mouths are hand drawn 2D rendered in 3D. I wanted the short to resemble my webcomic (http://www.littlegreendog.com/comics/brudders/brudders055.php) fairly closely and this seemed the best approach for the mouths, not to mention the technique was very fast to do--which was critical because we imposed an extremely tight deadline for this project.
Now I understand. Yeah, TAFA would not work well here. It could be done, but why go through the extra hassle? :)

Anyway, looks great. Best of luck!

sami
03-17-2012, 09:36 PM
Thanks for the details on Magpie Greenlaw - thats helpful. I'll check it out

geo_n
03-17-2012, 10:39 PM
TAFA is discussed in this recent thread:

http://forums.newtek.com/showthread.php?t=126523

IMO, it is FARE better than ANYTHING "out there" available for lipsyncing. Super fast and super easy to use. Top notch! :thumbsup:

hey megalodon, DSW you're back. Can Tafa output to anything else besides mdd? Can it output to morphmixer for better control.

Greenlaw
03-17-2012, 11:02 PM
TAFA outputs MorphMixer. It was designed specifically for Lightwave, meaning it reads Endomorphs and outputs MorphMixer--but it also outputs .mdd for compatibility with other 3D apps.

DigitalSorcery8
03-17-2012, 11:27 PM
hey megalodon, DSW you're back.
Keep it quiet. :)
I may have a production where I'll be needing (possibly) LW talent. I still haven't decided if we'll be going the SI route or LW route - but that short film RUIN looks great and is making me think twice. But if we DO end up using LW, this forum is the best to obtain talent so I had to sneak back in. ;) We'll be needing modelers (which can also be Modo artists) as well as Animators and someone who's worked with MotionBuilder since we'll be using mocap extensively. I'll know more probably sometime next month.


Can Tafa output to anything else besides mdd? Can it output to morphmixer for better control.
As Greenlaw indicated, yes, it puts out Morphmixer files. I've only used the MDD pipeline though, so I can't comment on how well it works. Though if I know Mac Reiter, it works just fine. And thre will hopefully be a TAFA2 sometime in the relatively near future.

geo_n
03-17-2012, 11:30 PM
Oh wow great. I didn't know that. I bought mimic pro for lw but the morphmixer it generates theres too much keys to comprehend. Its actually easier to just do it in papagayo with less keys. But for crunch time projects mimic is ok.
I'll have to check out TAFA.

geo_n
03-17-2012, 11:33 PM
it puts out Morphmixer files. I've only used the MDD pipeline though, so I can't comment on how well it works.

Can you keep me posted if ever you use TAFA with morphmixer output? How much keyframes are generated and is it less garbage.

DigitalSorcery8
03-17-2012, 11:44 PM
Can you keep me posted if ever you use TAFA with morphmixer output? How much keyframes are generated and is it less garbage.

I just ran through a recent test and exported the animation in the Morphmixer format - hopefully I attached it here correctly. I don't know if this will help you to see - let me know.

regular
03-18-2012, 12:04 AM
Could use the old "Clutch Cargo" method.

http://www.youtube.com/watch?v=tOhGDRhJ8Hs

(about 10 seconds in is the first example)

I think of that cartoon everytime someone mentions lip syncing.

heh heh

Greenlaw
03-18-2012, 03:09 AM
Could use the old "Clutch Cargo" method.
That's awesome!

When I was a kid a local TV station re-ran an old show called Space Angel that used the same technique (I assume it was made by the same production company.) It was HORRIFYING...yet strangely mesmerizing. :)

Greenlaw
03-18-2012, 03:16 AM
...so I had to sneak back in. ;)
No need to be sneaky! Good to see you in here again. :)

G.

DigitalSorcery8
03-18-2012, 01:10 PM
That's awesome!

When I was a kid a local TV station re-ran an old show called Space Angel that used the same technique (I assume it was made by the same production company.) It was HORRIFYING...yet strangely mesmerizing. :)

I remember Space Angel too! I remember thinking it looked weird, but it was still cool to watch. Now I start thinking about all of the old 1960's cartoons - like Gigantor, Eight Man, Prince Planet and AstroBoy.... brings back lots of good memories. Thank goodness for early Japanese animation!