PDA

View Full Version : Lip Sync and LW 2018



James Moore
09-10-2018, 08:29 AM
I built a new machine and now my install of MagpiePro is no longer licensed. I am looking for options for lip syncing in LW 2018

In Magpie I particularly liked the "Recognize" function where you typed in a line of text and Magpie would line it up phonetically with the waveform and then allow export to LW

Any suggestions on software that is currently available, like a magpie, or approaches using LW as is?

Thanks in advance,

James

TheLexx
09-10-2018, 10:40 AM
Hi James, try this.

Download Annosoft's Lipsync Tool demo (http://www.annosoft.com/downloads) and open. Download Punk wav (http://www.wavsource.com/movie_stars/eastwood.htm) and drag it onto the big grey open area of the demo software. Then simply go to menu bar>Lipsync>Textless Lipsync, then hit the play icon and see what happens.

Time to revisit this LW thread (https://forums.newtek.com/showthread.php?121941-Annosoft-Lightwave-Lipsync-Tool) I think. The original guys received a bit of a cool response, but I think they were really onto something. The proper version does apparently work with LW, but no one here extensively tested it as far as I know. Here is an obscure Polycount thread (https://polycount.com/discussion/68228/automated-lip-sync) - see the 4th post where M.G. is trashing the Lipsync Tool and praising Voice-O-Matic for Maya (http://www.di-o-matic.com/products/plugins/maya/VoiceOMatic/#page=features), completely unaware that his recommendation is based on a much earlier version of the same tool he is trashing.

Also, TAFA (https://www.youtube.com/watch?v=LPXWGxDl7g4) is reckoned to be excellent, but not sure if still available, so if ever Mac comes up with TAFA 2, it would be a glorious thing. Failing these, using other software to possibly help, Poser has Talk Designer (https://www.youtube.com/watch?v=pH-eMnLm29Y), and there is Mimic Live (https://www.daz3d.com/mimic-live) for Daz and Mimic Pro (https://www.daz3d.com/mimic-pro-for-carrara) for Carrara - maybe there must be some workflow to get those into LW (?).

Chuck, if you're reading, I don't expect you to simply drop everything for this, but possibly something for native Lightwave might lift CA for more users - maybe give Mac a call (!). What do others think ?

:)

raymondtrace
09-10-2018, 01:31 PM
Open source/free...


Papagayo (http://www.lostmarble.com/papagayo/)
Mike Green's Papagayo importer (http://www.mikegreen.name/Lscripts.html#Papagayo_Importer)

erikals
09-10-2018, 01:42 PM
maybe give Mac a call (!)
i did suggest it some time back to Mac, 5 years ago or so. At that time the answer was that he'd rather own/code it himself.
things change though, wouldn't hurt to ask again.

TAFA can actually produce better results than xSI Face Station, so quite alright.
i believe that is mostly due to FS being damn slow/hard/complex to use.

all auto lipsync these days sucks for 3D characters, for LEGO movie characters it works out alright.
the only good ones are in-house solutions.

ConjureBunny
09-10-2018, 03:53 PM
Unbelievable that this hasn't been addressed yet though.

TheLexx
09-10-2018, 04:29 PM
Unbelievable that this hasn't been addressed yet though.

If it is addressed by an auto-lipsync mechanism, it needs to be a good 'un to avoid "machine-gun mouth syndrome". Are there any audio guys here ? How difficult would it be for software to differentiate phonemes from a dialogue track ? I think the silent periods in speech are important to identify first, then maybe sliders to bring up the silences on the waveform (to try to differentiate between, say, cut-glass English and a Texan drawl), and also sliders to bring down the intensity of mouth movements, but I'm too inexperienced and don't have a clue. I saw this LW plugin Soundwave (http://walen.se/soundwave.html). Could it be adapted in some crazy way for speech ?

Here is an old tutorial (https://web.archive.org/web/20040205194204/http://www.robpowers.com/Tutorials/lipservicedoc.htm) from 2003 of Joe Alter's LipService for Lightwave, written by none other than.......Rob Powers (!). Only worked with LW 7 apparently. Did any of you guys ever use it ? I wonder if anyone would consider a huge LW group buy-in if Joe was interested in developing further and bringing up to date (or even releasing as is).

TAFA 2 in Lightwave would be something special though.

Oliver, if you are reading this, did you ever have any thoughts on lipsync for Lightwave (since you're probably one of the very few people who could actually pull it off) ? My credit card hovers....:D

Greenlaw
09-10-2018, 05:07 PM
Yeah, same thing happened to one my Magpie Pro licenses. It's a shame this tool is no longer supported as it was so easy to use and very flexible for a variety of 2D and 3D uses, not to mention it was fully LightWave compatible. Papagayo is similar in concept but it's not nearly fully-featured as Magpie Pro was.

I have a second license of Magpie Pro running on my Wacom Companion 2, and I have one active personal project where I'm still using it. It will probably be my last.

My other personal project is dialog heavy and I'll probably use TAFA for it or just keyframe the whole thing in LightWave.

BTW, I wasn't using Magpie's auto-lipsync feature. In general, I'm not a fan of 'auto lipsync.' I just liked the simple click-an-image-to-insert-a-mouthshape method for manually keyframing lipsync animations. Very fast and intuitive.

That said, I did use Magpie's auto-lipsync on a 'quick and dirty' freelance project almost two decades ago. I had a ton of dialog to knock out, and because there was hardly any money in the project, this feature did the trick. Personally, I thought the result was just okay but, after tweaking a few glaring errors, the client was happy with it. I guess there's a place for every tool. :)

TheLexx
09-10-2018, 05:37 PM
Probably worth mentioning the Adobe Character Animator for Lip Sync tutorials by D.W. Burman, which might be useful for some scenarios. Pt1 (http://www.liberty3d.com/2015/09/using-adobe-character-animator-for-lip-sync/) Pt2 (http://www.liberty3d.com/2015/09/7614/)

@ raymondtrace, good call :)

ianr
09-11-2018, 08:53 AM
It seems 'LIPS are TIGHT' Mac ain't saying, his forum got hacked & TAFA don't do LWO2 format

But for $200 TAFA is VG it has a great morph joysticker http://ta-animation.com/FA/orderTAFA.htm

Yes I used Joe's Shave & Haircut (lipsyncer) it was & still is the best joysticker, but Newtek & him

parted ways he went onto build great stuff for Maya. But LW plug-ins have to feed the family OK?

So TAFA running on LW2015 then exported, is the answer right now, Mind you you have build your

own Phonemes bank up as on other platforms there is no auto text recognize function.

raymondtrace
09-11-2018, 10:59 AM
...Papagayo is similar in concept but it's not nearly fully-featured as Magpie Pro was...

Free, open source, perpetually functional are some pretty spectacular features Magpie Pro never offered. The key feature is "open source" so one could certainly add features to better align with Magpie Pro.

My memory is dusty but I vaguely remember a free edition of Magpie before it went "Pro". I wonder if the feature set of that edition surpasses Papagayo...and if I still have it buried in my files.

TAFA is definitely worth poking at. Unfortunately, since Mac replaced his hacked homepage, there's now no apparent way to reach him and let him know the value of his program...unless you have a time machine. https://web.archive.org/web/20171030115834/http://www.macreitercreations.com:80/contact.php

erikals
09-11-2018, 11:50 AM
i believe this is his LinkedIn profile ?
https://www.linkedin.com/in/mac-reiter-b000b725

James Moore
09-11-2018, 11:52 AM
thank you for all the replies and suggestions!

I can't seem to get my hands on TAFA anywhere. Links are broken ect. The Annasoft Lip Sync tool looks interesting but pricey and I'm not sure how well it integrates with LW should I shell out the cash.

I've been banging my head on Papagayo and the script for importing. Papagayo seems to be quite helpful in speeding up the lip sync animation process and I love the price. I am having some difficulty with getting the data into LW though. The names of the phonemes appear to be different than the endomorphs set up in a model I'm working with. Papagayo says it works with the Preston Blair phoneme series. The Magpie docs says it also works with the Preston Blair phoneme series. My models endomorph names aren't the same as what Papagayo is producing. For example, Papagayo spits out names "etc" and "rest" which don't appear in the Preston Blair name list.

Is there a list somewhere that details Papagayo's phoneme naming conventions?

Greenlaw
09-11-2018, 12:18 PM
From what I recall the free version of Magpie was pretty stripped down compared to Magpie Pro. Pro supported output for many 2D and 3D programs, and imported .lwo with endomorphs. I think the free version just exported image sequences but I could be wrong about that.

Compared to the free version of Magpie, Papagayo probably wins there, being a newer program and currently developed by the Morevna (https://morevnaproject.org/papagayo-ng/)guys.

I don't know if Mike's LightWave plugin work with the latest Papagayo. Anybody try this lately?

ernesttx
09-11-2018, 12:53 PM
Yes, Mike's plugin works w/ Papagayo and 2018.x. I use it quite frequently. James, your mouth morphs should be similiarly named for the phonemes (ie, "AI", "E", "U", "rest", et al). I name my mouth morphs as "pap.AI", "pap.E", et al. I just like to have the mouth morphs under one roof. You may be able to name your's as just the phoneme name. I have meant to do a video on this explaining my process but have not gotten around to it. Maybe this weekend.

As far as Papayo, I have found out that it injects more 'etc' phonemes than actually needed, so you may need to edit each word to take out any unneeded etc phonemes. Or you could just edit the .dat file in Notepad to take them out.

Greenlaw
09-11-2018, 01:07 PM
Good to know! Thanks ernesttx.

James Moore
09-11-2018, 01:23 PM
How many different phonemes does Papagayo spit out and what are their names? Is there a magic word sequence I can stick into it that will generate all it's phonemes?

I'm trying to figure out how to get the naming conventions lined up with my models phonemes....

By the way, the I couldn't get the Morvena version (1.4) to install and work. The lost models version is now at 2.0b1

Greenlaw
09-11-2018, 01:27 PM
I'm not sure. I think it's more limited than Magpie Pro, which didn't have a limit to number of mouth shapes/expressions.

I could be wrong but I think Papagayo was limited to the Preston Blair set. It's been a long time since I played around with it so not sure. Let me check the Moho Pro docs...I think there's a chapter on Papagayo there. (That would cover only the Lost Marble version though, not Morevna.)

Greenlaw
09-11-2018, 01:36 PM
Not much info about Papagayo in the Moho docs...just about how to use it with Moho.

I found I do have Papagayo NG (Morevena) installed on my personal tablet computer, and it seems to run. The help files didn't open up though.

It's almost my lunchtime--I'll take a closer look then.

ernesttx
09-11-2018, 01:43 PM
I believe (I'm not in front of Papagayo at the moment and going off memory) . AI, E, O, U, MBP, FV, L, QW, etc and rest. 10 in all.

James Moore
09-11-2018, 01:50 PM
Pretty good memory. I input a whole bunch of words into Papagayo to see how many I could find. The only deviation I could find from your memory is that QW is WQ.

Greenlaw
09-11-2018, 01:53 PM
Yeah, that sounds right to me.

There's a little info about modifying the set here:

http://www.lostmarble.com/forum/viewtopic.php?f=15&t=31810&p=181202&hilit=add+shapes+to+papagayo&sid=16db203bd9e46844c65847b928b3b7da#p181202

raymondtrace
09-11-2018, 03:24 PM
...By the way, the I couldn't get the Morvena version (1.4) to install and work. The lost models version is now at 2.0b1

Was the interface mostly blank when it starts? Try resizing the window.

On my system, I find the LostMarble versions are the best at handling scrubbing of the audio file track. The Morvena version does not appear to be very accurate with the audio when you wiggle the playback head.

raymondtrace
09-11-2018, 03:31 PM
...I can't seem to get my hands on TAFA anywhere. Links are broken ect.

Archive.org still has the demo available for download. I believe some anti-virus apps will see a false positive but it was safe when I dug into it a while back.

https://web.archive.org/web/20170627085736/http://macreitercreations.com/FA/FA_trial_setup.zip

Of course, you'd still need to reach out to Mac to buy the full version.

James Moore
09-11-2018, 03:36 PM
I seem to have the lost marbles version of papagayo working pretty well now. I changed the names of some of my phonemes to match the ones pap generates and it imports quite nicely. Now I'm wishing I could get more phonemes out of papagayo but, overall, it's working quite nicely.