PDA

View Full Version : Annosoft Lightwave Lipsync Tool



bpritchard
08-26-2011, 11:27 AM
Today we at Annosoft have launched a version of the Desktop Lipsync Tool that supports exporting data to the Lightwave morph mixer format.

There is a free NON-save enabled version on the site (www.annosoft.com) that will allow you to check out the in tool voice analysis and recognition. If your interested in a save enabled demo contact through the site and we'll get you one out quick!

We're looking for people who are interested in giving it a run... we've done some limited testing with another developer and have seen some really great results!

Cheers!
Bryan

Cageman
08-26-2011, 03:15 PM
Hmm... sounds interresting for sure, but I need more to go on than this. Don't get me wrong, I think it is great that you guys support LW... but I would want to see some videodemos of the tool, the workflow and, obviously, some examples of endresult.

:)

EDIT: Looking at the site, I can't see a single word about LW-support... :confused:

Dodgy
08-26-2011, 06:25 PM
Presumably it's in Beta, so no mention till it's fully working.

Dodgy
08-26-2011, 10:19 PM
It certainly doesn't beat it on price... 3000 for a seat?

djlithium
08-27-2011, 06:54 AM
Yeah... 3K is a bit... steep to say the least. TAFA does rock thats for sure. Make your morphs and go. I have watched Tim work in TAFA in person and the ideas that he put forward to the programmer (Mac Rieter) came across pretty much flawlessly for what Tim had in mind in the first place and then some. The OpenGL performance of TAFA is sickeningly quick to.
I'm going to have to download the demo of this thing and try it sometime but its out of my price range that's for sure.

bpritchard
08-27-2011, 12:47 PM
Hey guys,

We'll post some videos over the next week. The 3000 price tag is for the SDK version (which is used in a large number of games). The lipsync tool (desktop tool that supports lightwave) is only $500.00

The tool itself is quite mature, and has been in development since 2002. If you have any questions or need any help feel free to drop me a line. In the mean time expect to see some more info soon.

cresshead
08-27-2011, 03:04 PM
i'd like to see/hear the results all i see in the video is a chap opening and closing his mouth with no audio.


http://www.youtube.com/watch?v=Xjx29PmWOCE

also has over 2 mins of black at the end of the clip...maybe edit that sometime?
in use it looks similar to voice -o- matic plugin

pstallo
03-16-2012, 10:52 PM
I would love a lip-sync tool for the Mac, but for the $3500 I would rather struggle with endomorph sliders and purchase a copy of Realflow or 6 more Mac mini's to make it all work faster.

jasonwestmas
03-22-2012, 08:50 PM
Cut that price in half and I would probably consider using this.

OnlineRender
03-23-2012, 03:40 AM
tough crowed....

Greenlaw
03-23-2012, 10:02 AM
I just found this thread. Thanks for starting it.

Coincidentally, I was playing around with Annosoft Lipsync earlier this week and ran some of the audio from one our Brudders shorts through it. The demo did not come with docs but it's so simple I figured it out in a minute: load a character, load a sound and analyze. With the default mode setting, it didn't do so well with the Brudders audio but when I switched the analysis mode to female/child and reprocessed, the results were surprisingly good. Editing is fun--it displays a familiar dopesheet style timeline and you drag clips up and down their columns to adjust morph values. Of course you can slide and stretch horizontally to make timing adjustments.

One thing I couldn't find info on was how to set up a custom character (i.e., what morphs are required, naming convention, etc.) I imagine docs are included when you pay for it.

I'm not a fan of fully automated lipsyncing software but this is by far the easiest one I've used and the results are much better than what I've come to expect from these things. I might consider getting it but it's a bit expensive for me and I don't need it for the type of lipsyncing we're doing in the Brudders short. Maybe after we're done with the Brudders stuff but we'll see.

The version I was tried was save disabled so I couldn't test it in LW but Annosoft does offer on request a 30-trial with save enabled. It is pricey ($500) but if the results were as consistent as what I got playing around for an hour, it could be worthwhile for projects with tight schedules.

If anybody has more info about it, please post!

G.

DigitalSorcery8
03-23-2012, 07:52 PM
Hey guys,

We'll post some videos over the next week. The 3000 price tag is for the SDK version (which is used in a large number of games). The lipsync tool (desktop tool that supports lightwave) is only $500.00

The tool itself is quite mature, and has been in development since 2002. If you have any questions or need any help feel free to drop me a line. In the mean time expect to see some more info soon.

I don't see this information on the site anywhere.

In fact, I don't see too many details regarding the LW implementation at all.

When will you have more information?

Greenlaw
03-24-2012, 12:16 AM
From there Prices page:


Annosoft licenses the Lipsync Tool for 500 USD per seat. This price includes all Flash export features and includes 1 year of free upgrades, and excellent technical support. Click here for more information and to download a demo.

The other prices on the page are for the developers kit version. As for LW support, yeah, I agree the info is slim but here ya go:


Annosoft is pleased to announce the Lightwave 3D® export option for Lipsync Tool 4.0.

This release brings the accuracy and reliability of Annosoft’s automatic lipsync to Lightwave 3D® users. “This is part of our effort to support more platforms and provide flexibility to artists.” Bryan Pritchard, CCO of Annosoft said.

Usage is simple and workflow does not change in the Lipsync Tool. The tool can be used to fully automate the lipsync process, but also provides a vary convenient way to adjust lipsync with the need to edit curves. The exported file can then be brought into Lightwave 3D® using the Morph Mixer.

For a free evaluation, please contact [email protected]

Something came up today and I need to make a character demo very quickly, so I'm going to ask for the full eval version. The key thing for me will be just how easily it can import LW data and how easy it will be to apply and edit the exported data. Based on the above info it sounds like you should be able to import an .lwo (or maybe FBX?) and it outputs MorphMixer data (or a LW scene file with MorphMixer applied?) It is a little unclear...I'll let you know what I find out. If my demo project is successful, Annosoft LipSync Tool will be worth getting for me.

FWIW, there are several demo movies on the Annosoft YouTube channel (http://www.youtube.com/annosoft), mainly Maya and Unity though. It would be nice to see the LW workflow demoed there...I imagine that's what's "coming soon."

G.

DigitalSorcery8
03-24-2012, 12:44 AM
Something came up today and I need to make a character demo very quickly, so I'm going to ask for the full eval version. The key thing for me will be just how easily it can import LW data and how easy it will be to apply and edit the exported data. Based on the above info it sounds like you should be able to import an .lwo (or maybe FBX?) and it outputs MorphMixer data (or a LW scene file with MorphMixer applied?) It is a little unclear...I'll let you know what I find out. If my demo project is successful, Annosoft LipSync Tool will be worth getting for me.
Just got the demo and tried it out. Not too bad. It doesn't - AFAIK - import anything from LW into the program. You have to have your character morphs in their phoneme format which is in their online help within the program. It's just a matter of simply renaming your LW phoneme morphs. Once you've got your morphs listed as they want, your LW model is ready to receive the morphmixer file.

I took one of my characters and used a wav file and typed the words - similar to how MimicPro works - and then ran the program. It was fast and you can see every phoneme in a graph along with your text showing for each section. From there you just export as a morphmixer file and then import it onto your LW object. I compared it to the exact same file I created in TAFA and it was not that bad. I think with a little manipulation within the phoneme graph I might be able to get it almost identical. Not sure about that, but I'll give it a shot later (hopefully) this weekend. I also haven't tried the Auto-Gesture feature where (I assume) we'll get blinking eyes and other things.

At this point it's an awful lot like MimicPro for LW - without many of the features in MimicPro. Though as I recall, MimicPro added allot more unnecessary morphs and the initial animation did not look as good as the Annosoft tool. I'll need to do some more testing.

Greenlaw
03-24-2012, 01:08 AM
FWIW, I think it does import models, but I'm basing this solely on the presence of the disabled 'Import' command and the 'New Character' and 'Manage Characters' commands. ('Load' is used for audio and apparently project files.) The questions of course are "What can you import? Does it accept .lwo, .fbx, .obj?" I wish the Help file came with the downloadable trial...all this could have been easily answered there. Hopefully documentation comes with the full version.

I'll know for sure after I get the full eval version and pass on the info here.

G.

Greenlaw
03-24-2012, 01:10 AM
Oh, I think I misunderstood you. You mentioned exporting MorphMixer data--does this mean you have the full eval version? I was wondering if you got the docs with it? Any more info in there?

Thanks in advance.

G.

Greenlaw
03-24-2012, 01:30 AM
At this point it's an awful lot like MimicPro for LW...Though as I recall, MimicPro added allot more unnecessary morphs and the initial animation did not look as good as the Annosoft tool. I'll need to do some more testing.

Yeah, that was my impression too. I think Lipsync's quality is a lot better than Mimic Pro, especially after I chose the appropriate Voice Quality for the audio. I need to test this with actual characters first and see for myself how easy the data is to nuance to 'final' quality before I can decide.

From what I recall Mimic Pro gave you optional 'head bobbing' and random 'blinking', which was cute but not desireable. I may be lazy but really. :)

G.

Greenlaw
03-24-2012, 03:26 AM
I took another look at the save-disabled trial. After some poking around, it appears you can import d3d (directX) and .obj. Looking at the Character Setup window, I'm guessing you need to import a separate .obj for each morph target. Not nearly as convenient as importing a single .lwo with embedded Endomorphs like you can with Magpie Pro, TAFA and Mimic Pro for LW, but I guess that's okay. The downside is that if you change the geometry, you'll need to update all the separate .obj files--that could get annoying. Of course, without docs I'm still just guessing here.

Greenlaw
03-24-2012, 10:31 AM
Wow, Annosoft is very responsive! I already received my save-enabled trial version and this version does come with docs. The developer is enthusiastic about getting getting LW users on board with Lipsync Tool, which certainly makes me enthusiastic about trying it.

Will post what I learn as I work with it. This should be fun. :)

G.

bpritchard
03-24-2012, 11:41 AM
Hey glad your ready to dive in. If you have any questions about setup or usage, ping me and i'll be more then happy to walk you through it. The lightwave integration is relatively new and we're eager to hear peoples thoughts on it.

DigitalSorcery8
03-24-2012, 01:15 PM
Oh, I think I misunderstood you. You mentioned exporting MorphMixer data--does this mean you have the full eval version? I was wondering if you got the docs with it? Any more info in there?

Thanks in advance.
Apparently yes. Mark had sent me the download link and I was able to export a morphmixer file out without a hitch.


Yeah, that was my impression too. I think Lipsync's quality is a lot better than Mimic Pro, especially after I chose the appropriate Voice Quality for the audio. I need to test this with actual characters first and see for myself how easy the data is to nuance to 'final' quality before I can decide.

From what I recall Mimic Pro gave you optional 'head bobbing' and random 'blinking', which was cute but not desireable. I may be lazy but really. :)
Yes, MimicPro did give you those options. Well it's nice for background characters, but hero characters it would look odd IMO.


I took another look at the save-disabled trial. After some poking around, it appears you can import d3d (directX) and .obj. Looking at the Character Setup window, I'm guessing you need to import a separate .obj for each morph target. Not nearly as convenient as importing a single .lwo with embedded Endomorphs like you can with Magpie Pro, TAFA and Mimic Pro for LW, but I guess that's okay. The downside is that if you change the geometry, you'll need to update all the separate .obj files--that could get annoying. Of course, without docs I'm still just guessing here.
Yes, this is something I do NOT like - separate obj files. I seriously like TAFA uing lwo and endomorphs - plus it automatically splits morphs so (for example) you wouldn't have to create a separate left eyebrow raise - right eyebrow raise morph. Bringing in separate objs is (IMO) not a good workflow.

Wow, Annosoft is very responsive! I already received my save-enabled trial version and this version does come with docs. The developer is enthusiastic about getting getting LW users on board with Lipsync Tool, which certainly makes me enthusiastic about trying it.

Will post what I learn as I work with it. This should be fun. :)
Yeah, I got the same impression as well - VERY enthusiastic. Hopefully we can get better LW integration. Also exporting as mdd would be a nice option as well.

Julez4001
03-26-2012, 09:35 AM
Just a recap.

No Mdd export (not sure if I want that anyway)
Morph Mixer export


Does it read endomorphs?

Does it read ,lwo natively or are we exporting out .obj files

Greenlaw
03-26-2012, 11:26 AM
You probably don't want to export .mdd from Lipsync as .mdd may prevent you from editing and enhancing the data easily in your 3D app. It's far more flexible to work with exported morph data in your 3D program and write out the .mdd from there. (Assuming you're rendering in another app or need to add some special effect.)

As far as I can tell, no Endomorph support (yet?) Annosoft seems open to suggestions and learning how LW users prefer to work, so be sure to write them. (I'm doing that right now.)

This weekend I didn't get to do as much testing with Lipsync Tool as I wanted but I'm trying a few tests this morning and I will post what I learn soon. (My pre-school guinea pig...um, I mean daughter... provided me with some fun audio this weekend to try with Lipsync Tool.) :)

G.

calilifestyle
03-26-2012, 12:43 PM
never mind

bpritchard
03-26-2012, 10:16 PM
No, currently there is no endomorph support. Our first pass only supports using Morph Mixer, which gives the user a much cleaner (and simpler) interface for editing the data post too (for instance if you wanted to apply any overage animation or extra stylization passes to it). But we are definitely open to suggestions for sure... feel free to contact us if you have any suggestions or questions on lightwave usage.

Also we only support obj's for import currently. We are looking at alternatives at the moment. Remember we only import mesh data, but do not export it... only channel data. Getting the data in is pretty simple, and can also be automated (aka a lscript that writes out the obis and the config file), its definitely something we can look at adding to the pipeline.

If its helpful i can also workup a walk through on how to get data in and out easily. There are a lot of little tricks in the tool to getting good data out, but generally speaking the biggest key is having clean audio and really good base poses. Make sure they are exaggerated (the poses), you'll be much happier with the results. If its any help, there's a viseme example page included in the documentation that can help with layout of your poses.

Cheers
Bryan

DigitalSorcery8
03-26-2012, 10:28 PM
No, currently there is no endomorph support. Our first pass only supports using Morph Mixer, which gives the user a much cleaner (and simpler) interface for editing the data post too (for instance if you wanted to apply any overage animation or extra stylization passes to it). But we are definitely open to suggestions for sure... feel free to contact us if you have any suggestions or questions on lightwave usage.
I think you MAY be misunderstanding endomorphs. A single .lwo file can have a HUGE number of endomorphs - like smile, eyelids closed, eyebrows up, mouth open, phoneme IY - etc. The .lwo object with endomorphs can work with morphmixer and .mdd files. I'm not sure - not having worked with Maya - but I think a Lightwave endomorph object is similar to blendshapes in Maya - one object can contain LOTS of morphs.

For Annosoft Lipsync, it would be WONDERFUL for it to accept a single .lwo with lots of endomorphs instead of having to load lots of .obj files. Some head objects I have over 100 endomorphs - in a single file.

Hopefully this helps. :)

Greenlaw
03-27-2012, 12:53 AM
Hi Bryan,

I'm having fun with Lipsync Tool today. My daughter's voice seems so surreal coming out of the default head. By the way, the lipsync itself looks pretty good, especially considering how insanely fast Lipsync Tool processed the data.

Some of the mouth transitions could weighted a little better though. Is there a way to adjust the weighting between blends? I was just wondering since there doesn't appear to be a curves editor available. Or is this level of tweaking best left for editing in the target animation application? (In our case Lightwave.)

Regarding the workflow walkthrough for setting up a Lipsync Tool project for Lightwave, yes, I'd be very interested in seeing this. A text document or video would be very helpful. I have a proxy object of my daughter's 'head' that I'd like to drop into Lipsync Tool. I'm going to try this tonight but it will be good to see how this is intended to be done, just to be sure I'm doing this correctly.

Regarding importing .lwo, yes this would be preferable for LW users since .lwo can contain embedded morph data called Endomorphs, which are basically vertex maps and not separate geometry. The nice thing about using Endomorphs for Lightwave users is that when the base object is edited, all the associated morph maps are dynamically updated--very convenient! FYI, another lipsync program I've used called Magpie Pro supports .lwo and Endomorphs.

Thanks for dropping in by the way! Your participation here is very welcome. :thumbsup:

G.

Greenlaw
03-27-2012, 06:09 PM
Here's a quick video of what I was playing with yesterday:

Automated Lip Sync Test, Part 1 (www.youtube.com/watch?v=lzLqzSW8Hh0)

Annosoft Lipsync Tool is really fast and it's fun to use. I won't say the results here should be taken as 'final quality' but the program does remove a lot of the grunt work from the process. Some manual editing to the lip sync can be done within the program, and further enhancements and facial expressions can be added by loading this data into MorphMixer in Lightwave.

I don't have recent experience with TAFA so I was hoping somebody here could answer this for me: Can I import MorphMixer data to TAFA for tweaks and adding facial expressions? Thanks in advance for any helpful info.

In part 2, we'll see how this works with a custom head.

G.

XswampyX
03-27-2012, 06:18 PM
That looks great, how much time would that save?

The url is mangled.

www.youtube.com/watch?v=lzLqzSW8Hh0

DigitalSorcery8
03-27-2012, 07:01 PM
Annosoft Lipsync Tool is really fast and it's fun to use. I won't say the results here should be taken as 'final quality' but the program does remove a lot of the grunt work from the process. Some manual editing to the lip sync can be done within the program, and further enhancements and facial expressions can be added by loading this data into MorphMixer in Lightwave.
Perhaps it's my familiarity with TAFA, but getting a better final result in TAFA is faster (at the moment) than in Annosoft Lipsync. Annosoft does a GREAT job at getting an acceptable initial result. To get something that would compare with what I get in TAFA takes about as much time as creating it entirely in TAFA. I think that (for me) in order for this product to compete with TAFA it needs to load .lwo objects with all of the endomorph data - thought I think that the creators of this software are willing to seriously improve this LW version.

I don't have recent experience with TAFA so I was hoping somebody here could answer this for me: Can I import MorphMixer data to TAFA for tweaks and adding facial expressions? Thanks in advance for any helpful info.I don't see any place to import MorphMixer files into TAFA. That said, I would bet that it's something that Mac could implement in the next version. There are a number of things I would like to see in TAFA2 - including an automatic setup so that - like in Annosoft - you can then tweak the results.

Oh yeah... and the price difference IS a big issue. TAFA right now is $200 while Annosoft is $500. :eek:

Greenlaw
03-27-2012, 07:23 PM
That looks great, how much time would that save?

The url is mangled.

Thanks for catching that! Link is fixed now.

Well, considering that getting this much done took less than a minute, I'd say Lipsync Tool may potentially save me a huge chunk of time. Of course, the time savings you gain from using this tool is diminished the more you feel a need to tweak the results. It all comes down to how much time you can afford to spend on the task and the quality you're willing to accept.

In my view, the result of this simple test is a very good base to work from and all I've done so far is click a few buttons. My hope is that by having this 'raw' data as starting point I'll be able to focus more of my time on refinement and working on facial animation.

Where I really want to use this tool is on a short film we have planned for later this year, where one character speaks for a entire minute without any cuts. Because Little Green Dog's animation 'crew' is usually just me, this program is looking pretty good right now.

I'll let you know how it goes.

G.

Greenlaw
03-27-2012, 07:32 PM
Well, considering that getting this much done took less than a minute...
I just rewatched my own video...it actually took 38 seconds. Hmm. I wonder if they can add GPU support to get better processing time.

(Yes, that's a joke.)

G.

sami
03-27-2012, 10:38 PM
I don't see any place to buy the Tool that is LW capable and save enabled. I found a link somewhere that said it's $500. But where do you buy it? They look like they are selling the SDK quite expensively (which would be useful maybe for realtime phoneme tracking which I've always wanted to use - like that Kinect full body avatar we did, we were just doing mic input volume checking to move the jaw bone when the character spoke. This would mean much better lip sync in realtime).


Also Greenlaw can you confirm does this tool also do 2D shape-switching as well or just 3D morphs? thx

Greenlaw
03-27-2012, 11:06 PM
When you launch the Lipsync Tool trial, a 'Buy Now' button pops up. When you click it, it takes you to a checkout page. You can pay using PayPal. The price is $500 for the version I used in the video.

FYI, I haven't purchased yet. I'm waiting until I complete this test and to see if it will really help our workflow. So far it's looking good though.

Regarding 2D export, according to the docs, it sounds like Lipsync Tool can export bitmap sequences in Flash format. This doesn't help me at all for Brudders of course but I'm thinking about using this software for non-Brudders projects, like our next short film RVJ (after we complete the second Brudders film.)

Again, regarding 2D export, because I've been having so much crashing with Magpie Pro for the second Brudders movie, we're looking into switching to another application. It's a shame because otherwise we had a great Magpie Pro workflow on the first movie. However, because there's a lot more dialog to lip sync in the second film, the constant crashing is getting too frustrating. This doesn't have anything to do with Lipsync Tool but I just felt like mentioning it. :p

G.

sami
03-29-2012, 04:50 AM
When you launch the Lipsync Tool trial, a 'Buy Now' button pops up. When you click it, it takes you to a checkout page. You can pay using PayPal. The price is $500 for the version I used in the video.

FYI, I haven't purchased yet. I'm waiting until I complete this test and to see if it will really help our workflow. So far it's looking good though.

Regarding 2D export, according to the docs, it sounds like Lipsync Tool can export bitmap sequences in Flash format. This doesn't help me at all for Brudders of course but I'm thinking about using this software for non-Brudders projects, like our next short film RVJ (after we complete the second Brudders film.)

Again, regarding 2D export, because I've been having so much crashing with Magpie Pro for the second Brudders movie, we're looking into switching to another application. It's a shame because otherwise we had a great Magpie Pro workflow on the first movie. However, because there's a lot more dialog to lip sync in the second film, the constant crashing is getting too frustrating. This doesn't have anything to do with Lipsync Tool but I just felt like mentioning it. :p

G.
Thanks. Let us know how stable you find Lipsync Tool also good to know Magpie isn't the bees knees. I'm finding bugs in Papagayo when the framerate is higher than 24.

Btw does Lipsync Tool have online or PDF docs available?

Greenlaw
03-29-2012, 12:10 PM
Thanks. Let us know how stable you find Lipsync Tool also good to know Magpie isn't the bees knees. I'm finding bugs in Papagayo when the framerate is higher than 24.

Btw does Lipsync Tool have online or PDF docs available?

Hmm...long hours at work and unexpected 'life stuff' happening all week, so I haven't been able to focus as much attention as I like. But I finally got to spend a little more time with this last night.

So far Lipsync tool has been very stable. I'm starting to grasp how to edit in the program and get nicer results. I also understand how to import a 'Lightwave' object and should be testing that today, if I can make time that is. BTW, does anybody know of a Layout or Modeler script that will export LW morphs as individual objects (.lwo or .obj)?

As far as I can tell, you only get the manual if you put in a request for the unlocked 31-day trial. Just ask them for the trial...they're very friendly people. :)

Regarding Magpie Pro, yeah. I hate to be critical of this program because it's actually quite wonderful...that is, when it's not crashing. BTW are there other Magpie Pro users here? How stable is the current version of Magie Pro on your system? I'm curious because the developer was not able to reproduce the crash. I've been able to reproduce the crashing on multiple computers in my home studio (various processors and graphics cards) but because I haven't heard anybody else complaining, I'm left to wonder if there is something unique about our home setup that's not compatible with Magpie Pro.

(FYI, my Magpie Pro crashing does not happen with the previous release of Magpie Pro but the new version has improved Lightwave features that I would like to use. Sigh! I can't have everything I guess.) :p

Anyway, I will try post more Lipsync Tool stuff soon.

G.

jasonwestmas
03-29-2012, 12:50 PM
BTW, does anybody know of a Layout or Modeler script that will export LW morphs as individual objects (.lwo or .obj)?

G.

you mean for a hundred morphs quickly? You can apply the morph and then use "save trans object" of course. Probably would need something for batch saving eh?

http://forums.newtek.com/showthread.php?t=120941

Greenlaw
03-29-2012, 01:53 PM
Yeah, it's really not a huge deal to convert a dozen or so shapes. Just feeling lazy I guess. :p

Thanks for the link...this sounds like what I'm looking for!

G.

Greenlaw
03-29-2012, 01:54 PM
Oh, and thank you Sami for creating this tool! :thumbsup:

Greenlaw
03-29-2012, 02:23 PM
Morphs2Objects is fantastic! Thanks guys...now I can keep on being lazy!

G.

Greenlaw
03-29-2012, 03:38 PM
This might sound like a very basic question but I'm at a loss: How do you import an .obj into Lipsync Tool with textures applied? Or maybe the question should be: what's the proper way to export an .obj from Lightwave so that the textures are still associated with it for other applications (like Lipsync Tool)?

I thought dropping the textures into the same folder with objects would do the trick but that didn't work. Maybe it's the image format I'm using? Is there a particular flavor Lipsync Tool prefers?

Thanks in advance for any helpful info.

G.

Greenlaw
03-29-2012, 03:39 PM
Ah, okay. I just looked at the .mtl files and I think I have my answer. More later.

calilifestyle
03-29-2012, 04:03 PM
How often is TAFA updated?

Greenlaw
03-29-2012, 04:14 PM
Nope, I'm not having any luck with this at all. What I get in Lipsync Tool is a completely white object with no textures. I'm guessing other surface properties are off too because I can't even see shading, which makes this not very useful. I'm sure it's an issue with my .obj exports and settings but unfortunately I don't have much experience in this area.

BTW, the image formats I've tried so far are .png and .bmp, and the .mtl files do show the correct path for the images.

bpritchard, do you have any .obj files that are known to be Lipsync Tool compatible that we can look at for comparison?

Thanks in advance to anybody for any assistance.

Greenlaw
03-29-2012, 05:00 PM
OKAY! I got it. This must be a Lightwave 11 bug because Lipsync Tool works fine with objects exported from Lightwave 10.1. I'm guessing this is an .obj exporter bug in LW 11. I'll make some content and fogbugz it.

bpritchard, one thing I noticed after I got the textured Lightwave object imported: the object is facing 'backwards'. That's probably okay because I can't imagine how this could hurt the performance of Lipsync Tool, but I was wondering if you guys could add a button in Preferences that flipped the default object orientation. Thanks for listening.

G.

calilifestyle
03-29-2012, 05:03 PM
Yeah this was my problem, trying to open OBJ. i wasn't sure if that was a limitation of the demo version.

sami
03-29-2012, 06:19 PM
... BTW, does anybody know of a Layout or Modeler script that will export LW morphs as individual objects (.lwo or .obj)?

I've written one... it should work:

http://forums.newtek.com/showthread.php?t=120941&highlight=megafiers

sami
03-29-2012, 06:27 PM
It must be the mtl file paths to the images. If there's something I can change in my script let me know. But since I use an FBX for the neutral morph and the OBJs for the morphed ones, it's really just using point data from them so I didn't care about their textures. But it might be related to 11 - I tested this with 9.6.1 and it worked fine.

I'm happy to tweak something in my script if it will help.

Greenlaw
03-29-2012, 07:21 PM
Hi Sami,

Thanks for the reply! As you may have already read, I did sort the problem out (buggy .obj export from LW 11. I need to fogbuz this when I get home tonight.) Your script works perfectly fine with LW 10.1. :thumbsup:

G.

Greenlaw
03-30-2012, 01:55 AM
I put in a couple more hours into this tonight. To recap: I was able to get objects into Lipsync Tool using .objs exported from Lightwave 10.1 but not .objs from 11.

Here's where I am now:

1. Objects seem to import to Lipsync Tool but only with one texture map applied. Apparently, Lipsync Tool only imports the last texture listed in the .mtl because all I see is the texture for the skin but not the eyes (spooky.) I'm not sure if this is an import limitation with Lipsync Tool or a bug. And if it's a bug, I'm not sure if this one belongs to Lipsync Tool or a Lightwave 10.1. Sorry I'm not very knowledgeable about the .obj format.

2. Just for kicks, I decided to export the Lightwave object from Modo as .obj. The Modo version did not import to Lipsync Tool with any textures applied. When I examined the .mtl files I noticed that Modo lists full paths where the Lightwave version only listed the filename (the textures had been placed where the .objs would be saved.)

I'm not surprised that there are variants of the .obj format and Lipsync Tool might not recognize all .obj files equally. So the next thing I tried was changing Modo's .mtl in a text editor to list just the filename (like Lightwave's .mtl did,) and this time the skin texture imported to Lipsync Tool. But like the Lightwave .obj, the Modo .obj only imports with the last texture applied (the 'skin' texture but not the 'eyes'.)

3. I decided to ignore the missing textures and just get on with the lip sync test. Lipsync Tool seems to load the .objs and allowed me to start the process. I loaded the same .wav file I used in my previous test and processed the lip sync. The timeline showed the proper morph clips...so far so good. But when I clicked Play, Lipsync Tool immediately crashed.

I tried this several times tonight, sometimes with many variations (Merge Points On/Off, One Layer On/Off, One VMap On/Off, etc,) but I'm always getting the crash on Play. BTW, This is a fairly low-poly mesh (7431 polygons) and there is only one UV map. Is anybody else having luck with this? I was wondering if it's just me having these issues.

bpritchard, I feel like I'm shooting in the dark here. Any information on what Lipsync Tool prefers for textured .obj is appreciated. Even better, if you can provide sample files that are known to be compatible with Lipsync Tool then I could compare what's different and make adjustments. I can even send you my files if you guys want to take a look.

That first test was so promising--I hope we can get this working soon. :)

G.

Greenlaw
03-30-2012, 02:29 AM
Just so you know what I'm looking at:

http://forums.newtek.com/attachment.php?attachmentid=103055&stc=1&d=1333095966

The top image is what my proxy object looks like in Modeler 10.1. The second image is what I see in Lipsync Tool. Audio has been loaded and analyzed. At this point, if I click Play, the program crashes.

G.

DigitalSorcery8
03-30-2012, 11:27 AM
1. Objects seem to import to Lipsync Tool but only with one texture map applied. Apparently, Lipsync Tool only imports the last texture listed in the .mtl because all I see is the texture for the skin but not the eyes (spooky.) I'm not sure if this is an import limitation with Lipsync Tool or a bug. And if it's a bug, I'm not sure if this one belongs to Lipsync Tool or a Lightwave 10.1. Sorry I'm not very knowledgeable about the .obj format.
I'm not certain, but I think that TAFA is the same - only one texture map comes through the process. And that's with using .lwo objects.

Greenlaw
03-30-2012, 12:28 PM
I'm wondering if the 3D display is intentionally limited in these programs (Lipync and TAFA) to a single texture for performance reasons.

It looks like I need to make a special surface for Lipsync by rebaking the image to a new UV layout so that everything fits on a single texture map. Not a huge deal but it is extra work I was hoping to avoid. I understand that having textured eyes in Lipsync is just 'cosmetic' and not really necessary for lipsync...but, hey, why not?

FWIW, I think Magpie Pro recognized all textures in an .lwo but I could be totally wrong. I'll dig up some old Magpie projects this weekend to confirm.

Anyway, what's important now is figuring out why Lipsync is crashing with my imported geometry. I wish I had something to check my files against and get this test done because I'll have move on to something else pretty soon. :(

G.

bpritchard
03-30-2012, 04:05 PM
Heya Greenlaw,

First off apologies for the delayed response on this. So to answer a couple of qs for ya..

1) textures - we support only diffuse textures currently (aka no normal maps or anything) but it should be bringing in every texture tied to a material. I'm going to PM you here in a few and give you my email, if you wouldn't mind sending over the obj/mtl we can look at it this weekend and see how they are being written differently. We make a few assumptions about the obj format, so we'll need to perhaps case it.

2) Normals seem backwards on your earlier example, we're going to look at that. We haven't seen this problem historically... but again thats not to say its not something we're doing. We'll look at it asap and see where the issue is.

Thanks for taking the time to post all of this.. its absolutely appreciated! The more feedback we get the more we can focus on fixes and get them back to ya. ;) I'll also gather up some samples (I do believe we never uploaded the LW obj samples so i'll add that to my list) for you to tinker with this weekend.

Cheers!
Bryan

Greenlaw
03-31-2012, 10:43 AM
No problem Bryan! I figured you guys were very busy.

Diffuse textures is all I'm interested in for this use, so that's cool. Thanks for confirming that Lipsync tool should be bringing in multiple diffuse textures though.

I hope to spend more time on this later today. I'll privately send files for you guys to look at around then.

Thanks for following this thread. If we can get Lipsync Tool to play nice with my workflow, I will be very interested in purchasing this program. My intention is to build a production pipeline at our home studio where we can quickly and routinely create animated short films and music videos with our teeny tiny crew (usually just me, my wife, and occasional collaborators,) and I can already see how Lipsync Tool may help us get there. :thumbsup:

G.

Greenlaw
03-31-2012, 01:41 PM
Re: backward normals for the eyes, that idea did cross my mind the other day and I flipped the normals for the eyes and loaded it to Lipsync Tool. Results were the same however. I'll check again to be sure but I'm almost certain this is a missing texture map problem.

Sorry, still busy with other stuff right now. Will try to get back to this by tonight. :p

G.

Greenlaw
04-02-2012, 01:10 PM
This isn't really Lipsync Tool related but it is related to the video I posted here last week.

Motion Capture Test with Small Child (1 meter tall) (http://bit.ly/HFd5oJ)

Set phasers on 'cute' and enjoy! :)

Obviously, this test delayed the Lipsync Tool testing. Sorry. I promise to get back to it tonight.

G.

bpritchard
04-03-2012, 02:57 PM
Its likely the flipped normals is actually the way the mesh is getting created, not just a normal problem. Once we get a few of your obj's we can work through it easily enough.

Greenlaw
04-03-2012, 03:14 PM
Will do. (Sorry for the delay, it's been busy at work and at home.)

tyrot
09-26-2014, 04:56 PM
ok i m a bit late to party :) but is there any animation you guys have done with annosoft? greenlaw? others? - is it worth to try?

Greenlaw
09-26-2014, 05:21 PM
Sigh! It's still on my very long list of tbings to do.

FWIW, the software showed promise when I tested it a while back. There were a few issues and the developers seemed eager to address them but I haven't been able to put much time in it. Above I think there's a link to a quick test I did a couple of years ago. From what I recall, it was really easy to use and the completely raw and unedited results were not bad--I thought it could be a good base to build from, which is generally the case with most automated systems I've worked with. (Short of actual face capture I mean.)

The lipsync animation I've been doing for the Brudders (https://vimeo.com/channels/littlegreendog/68543424)stuff requires a different method so I didn't get to use Annosoft for it. Right now, I'm creating a series of tests for a fully cg character project at work and this will probably use yet another method because it features pretty cartoony creatures--I'm not sure this is the right project for Annosoft either.

One of these days, I'm going to have to design a project with normal human characters just so I can try Annosoft for it--assuming the devs are still interested in developing it then.

Hopefully, somebody else has been using it and will pop in soon.

G.

jasonwestmas
09-26-2014, 05:32 PM
How often is TAFA updated?

It isn't. But it's pretty solid.

tyrot
09-26-2014, 05:46 PM
Actually it could be pretty awesome if Mac (TAFA) could add something automatic ..... it would be pretty nice.. TAFA deserves a big update - (may be kinect import) Sorry since TAFA's forums down..here is the only place to discuss these things..

Is there a way to import from LW's morphmixer data into TAFA? i couldn't figure out.. May be we could use PAPAYAGO base morph mixer data for it ...

There are tons of facial animation for my next demo.. I just cannot cut the facial animation time.. Automated solutions are pretty miserable when i compare it with TAFA. That s why i was desperately searching for - a useful program..

jasonwestmas
09-26-2014, 06:27 PM
you can export a morph mixer file from TAFA but it does not import morph mixer files. I agree with your comment about automated solutions. while nice in the batch processing dept. they still need a ton of cleanup ime.

Greenlaw
09-26-2014, 06:28 PM
FYI, Papagayo isn't automated...it's similar to working with Magpie Pro, which is what I usually use to make my base lipsync. In Magpie, you load your audio, enter you dialog on the timeline to visually assist, and then start keying the mouth shapes by clicking on the phonemes (represented by buttons you create) as work your way through the scene. It's really fast--I can plow through a lot of material pretty quickly. Then, I bring the data into Lightwave (or whatever animation program I'm using,) and finesse the data and add face motions. You can preview using 2D renders or 3D OpenGL in Magpie, depending on your workflow setup. Then you export what you need: image sequence, morphmixer, anime studio file., etc. I exported image sequeces of mouth shapes for Happy Box and currently exporting morph mixer data for 'B2'. Papagayo is more limited in features but I think the workflow is similar.

Magpie does have an automated mode too, but it takes a bit of effort to set up because you need to teach it the voice sample. It's been about 15 years since I used that mode--I used it one crazy job where I had about four minutes to lipsync in an hour or so. I thought the results were a little mushy but the client was happy to get it so quickly. :)

I think Annosoft's auto lipsync looked better than what I did way back when with Magpie's auto lipsync, but obviously I do this sort of thing pretty rarely.

These days I mainly use Magpie Pro in manual mode for my basic lipsync, or I work directly in Lightwave with Bones+Morphmixer or Anime Studio Pro with Switch Layers+Smart Bones for lipsync. Then I tweak and enhance it in whichever animation program I'm in.

I have a license of TAFA too but I haven't had a chance to use it yet. More specifically, the right project for it hasn't come along yet--something else that's been on my ever lengthening 'to do' list.

G.