Results 1 to 12 of 12

Thread: Any Viseme/Phoneme morhmap/blendshape helpers out there?

  1. #1
    Registered User sami's Avatar
    Join Date
    Jul 2003
    Location
    TopLeft Viewport
    Posts
    1,036

    Question Any Viseme/Phoneme morhmap/blendshape helpers out there?

    Hey been searching, and usually there's a "use this obscure tool" post, but no real helper tools for Lightwave I can find. I saw this amazing polywink service that you give them a model and overnight they send you 157 blendshapes of your model with all the visemes and emotions and chewing etc of your character's face. But it's a bit pricey (probably not so for a studio who would take week(s) to do that work). But they're a bit pricey at around $750 or something for that.

    Does anyone have any good tools or plugins that can help with the actual creation of the facial morphmaps/blendshapes in Modeler? I'm hoping there's something - no YouTube tutorials exist that I can find. I'd like something that enumerates the essential visemes and emotions and either helps create them and/or tells you what tools to use (Magnet etc) and where to pull to deform the face into those shapes.

    Note: I'm not talking about animating these. That's pretty straightforward. I intend to use Lip Sync pro in Unity. I just want fast helpers or cheats to create or assist in creating them. I saw a webcam tracker to blendshapes tool facerig - but that too seemed cumbersome or expensive.

    I had heard RebelHill had a facial rig thing or was it a tutorial? and I've bought & used his excellent biped stuff for games, but I went to the site and couldn't find anything and I think I read a post that there is no responses from him currently on the site. I could use bones or a face rig to pose the face and then capture certain blendshapes maybe? But don't know what face rig to use or start with.

    Any ideas for quick creation and listing of essential visemes/emotions would be appreciated - I'm obviously familiar with basic Preston Blair mouth shapes, but a modern list with emotions showing eyes, cheeks etc would be helpful. thanks!!

  2. #2
    Registered User
    Join Date
    May 2012
    Location
    Virginia
    Posts
    545
    Hello sami!

    Robbie Choongo at Robbie's Art created a plugin in recently for lip sync in Lightwave. I'm not sure that's what you are looking for... it does rely on Adobe Character Animator to generate the visemes, but the plugin creates the lip sync.

    Check it out and see if it's something that might help your process.

    https://www.youtube.com/watch?v=yaNLqOkjJrw

  3. #3
    Founding member raymondtrace's Avatar
    Join Date
    May 2003
    Location
    Ohio
    Posts
    1,148
    As far as I know, this is just done manually as morphs in Modeler (or bones in Layout). The tool RPSchmidt mentions could be helpful after this is set up.

    If you don't already have a morph modeling template, I believe MakeHuman/DAZ/iClone should have phonemes/expressions.
    LW4, 7.5D, 2015, 2018, 2019, 2020 running portably on a USB drive on an Amiga 2500 running Wine.

  4. #4
    Super Member Kryslin's Avatar
    Join Date
    Feb 2009
    Location
    Prescott, IA
    Posts
    1,612
    Quote Originally Posted by sami View Post
    I had heard RebelHill had a facial rig thing or was it a tutorial? and I've bought & used his excellent biped stuff for games, but I went to the site and couldn't find anything and I think I read a post that there is no responses from him currently on the site. I could use bones or a face rig to pose the face and then capture certain blendshapes maybe? But don't know what face rig to use or start with.
    It's a facial rig, and there was a video series on how to do it. RebelHill's FaceFactory has been in beta for a long time, now. It's a rather comprehensive face rig, and takes some getting used to. However, due to changes in LScript, and the fact the LScript is now no longer being maintained... You'll have to do your set up in LW 2019.1.5 and under.
    --------
    My Scripts for Lightwave
    Intel Core i7 960 @3.20 Ghz, 24 GB ram, EVGA 6GB GTX980Ti "Classified" driving 2 x HP LA2405.

  5. #5
    Founding member raymondtrace's Avatar
    Join Date
    May 2003
    Location
    Ohio
    Posts
    1,148
    With LScript support fading and an unknown future compatibility with FaceFactory... why not go backward in time?

    If you're not using any face tracking techniques (which primarily require bone rigs) consider the morph method with TAFA. Morphs work in the past and should be supported well into the future. They can be portably used in LW and other programs.

    As a separate program, TAFA does not break with changes in LW's script support.

    http://macreitercreations.com/

    Messiah is also a good option for either bones or morphs
    Morphs: https://www.youtube.com/watch?v=-mBbrk-rT_8
    Bones: https://www.youtube.com/watch?v=VqEiAPJ3QWw

    Even if you don't use Messiah, these videos show some useful practices.

    (MakeHuman also includes a bone setup for the face)
    Last edited by raymondtrace; 06-22-2020 at 08:54 AM.
    LW4, 7.5D, 2015, 2018, 2019, 2020 running portably on a USB drive on an Amiga 2500 running Wine.

  6. #6
    Registered User sami's Avatar
    Join Date
    Jul 2003
    Location
    TopLeft Viewport
    Posts
    1,036
    Quote Originally Posted by RPSchmidt View Post
    Hello sami!

    Robbie Choongo at Robbie's Art created a plugin in recently for lip sync in Lightwave. I'm not sure that's what you are looking for... it does rely on Adobe Character Animator to generate the visemes, but the plugin creates the lip sync.

    Check it out and see if it's something that might help your process.

    https://www.youtube.com/watch?v=yaNLqOkjJrw
    Hi RPSchmidt!

    Thanks very much for that - that's very cool - I'm sure it saves a lot of time with keyframing. And it's great to see Robbie making such a cool plugin new now when most developers aren't these days and the prices is amazing too.

    But I don't want to animate in LW - I'm animating in Unity and the main thing is actually creating the mouthshapes and emotions. Unless I missed it in the video, I don't see how this workflow helps create those. I am not familiar with Adobe Character Animator, but I don't think it creates viseme / blendshapes of your own models - Like I can't load my own character model in it and it won't create morphs blendshapes for me based off of facial geometry??

    Appreciate the heads up with this tool, but I don't think it will work for this time. Good to keep it in mind though.

    I was mostly hoping for some thing, a tool or guide, or even a comprehensive best practices list of what the essential visemes/emotions are for a face and how to modify your neutral face to get them.

    Do you know of a comprehensive and best practice list? I haven't seen anything of what I call complete on YouTube or anywhere. Nor in paid tutorials...?

  7. #7
    Registered User sami's Avatar
    Join Date
    Jul 2003
    Location
    TopLeft Viewport
    Posts
    1,036
    Quote Originally Posted by raymondtrace View Post
    As far as I know, this is just done manually as morphs in Modeler (or bones in Layout). The tool RPSchmidt mentions could be helpful after this is set up.

    If you don't already have a morph modeling template, I believe MakeHuman/DAZ/iClone should have phonemes/expressions.
    hi raymond!

    Nope I only have the neutral expression (and no tongue - just teeth - I don't think I need tongue for now)...

    I looked into iClone and tried the demo - it was cumbersome and I couldn't figure out how to load my file into it and use their sliders. I want something that will help either weight my character face with a face rig or modify and add morphs to it for different visemes/emotions. Or at least a starting point and I can tweak. Have you ever gotten iClone to be useful for this? All their YouTube videos I've seen don't seem to show any good workflow for this or any good results. It feels like you have to use their base model and export from there? I have my own character, modeled in LW and 3DCoat - and it's a likeness of someone so I don't want to start from scratch and use a base model from somewhere else.

    I haven't used Daz in years, but again I think it only handles its own models and things that come from its base model types like Victoria or Michael or something you buy? Can it accept FBX input and generate nice FBX output with morphs that don't mess up UVs or anything on the way out?

    I also haven't used MakeHuman in a while - is it also tied to its own base? Maybe I could use MakeHuman to view the extent of the list of Visemes and the emotions - does it have those? I suppose I could export a similar shaped head from it and then put in another layer and use that as reference to model my own morphs - that may help a bit - but I really was hoping there was a tool that actually helped you modify the emotion types and mouth/lip/cheek shapes...

  8. #8
    Registered User sami's Avatar
    Join Date
    Jul 2003
    Location
    TopLeft Viewport
    Posts
    1,036
    Quote Originally Posted by Kryslin View Post
    It's a facial rig, and there was a video series on how to do it. RebelHill's FaceFactory has been in beta for a long time, now. It's a rather comprehensive face rig, and takes some getting used to. However, due to changes in LScript, and the fact the LScript is now no longer being maintained... You'll have to do your set up in LW 2019.1.5 and under.

    Hi Kryslin,
    Thanks! I'm using LW 2020 - Did you mean it only works in LW 2019.1.5 and under? or just setting it up/installing it? And do you have a link? Does it still exist? Is RebelHill still ok/around and selling product? It's a wacky time this pandemic and when you don't get a response or see a slow website you wonder if it's because of the current events and hope everyone's ok...

    I couldn't find the face rig thing on his site - is it a tutorial that comes with the rig or a separate product from a tutorial series? And do you know if that rig is lightweight enough to use in Unity? I know I had to use one of his special "game" biped rigs for Unity as the other ones had too many bones and too many bones competing per vertex. Unity only supports up to 4 bones competing for a given weighted vertex to affect it. This is a problem generally in weighting a character especially in hips or under arms and sometimes hands. Hopefully the face rig, isn't too complex - the one YouTube video I saw seemed to have a ton of bones - have you tried that rig or FaceFactory out? thx!

  9. #9
    Registered User sami's Avatar
    Join Date
    Jul 2003
    Location
    TopLeft Viewport
    Posts
    1,036
    Quote Originally Posted by raymondtrace View Post
    With LScript support fading and an unknown future compatibility with FaceFactory... why not go backward in time?

    If you're not using any face tracking techniques (which primarily require bone rigs) consider the morph method with TAFA. Morphs work in the past and should be supported well into the future. They can be portably used in LW and other programs.

    As a separate program, TAFA does not break with changes in LW's script support.

    http://macreitercreations.com/

    Messiah is also a good option for either bones or morphs
    Morphs: https://www.youtube.com/watch?v=-mBbrk-rT_8
    Bones: https://www.youtube.com/watch?v=VqEiAPJ3QWw

    Even if you don't use Messiah, these videos show some useful practices.

    (MakeHuman also includes a bone setup for the face)
    Hi Thanks for those videos - the bone one was particularly interesting. I actually have a copy of Messiah 5 or 6? somewhere on a USB somewhere - but thought it was long abandonware, and haven't tried it in Win 10 with 2020 (since LW v9 actually was the last time I used it).
    I'd feel really nervous about spending a lot of time in an app and then find out later that the export was stripping some UVs or it has a problem with layers or point order or something obscure ruining all the output from it and wasting your time. Is Messiah still a viable tool???

    Mostly the videos start out with the "magic" I'm looking for - the morphs or rig.

    TAFA is cool too, and I've used it years ago. Nowadays I almost never touch rendered output - all i do is output to realtime for game engines like Unity. So I'm less worried about the animation as I intend to animate it in this tool in Unity:
    https://assetstore.unity.com/package...sync-pro-32117

    I saw this which looks amazing but is very pricey - You send them YOUR model and the next day they give you 157+ mouthshapes and face emotions/actions named nicely which you can then animate how you like.

    https://www.polywink.com/9-automatic...on-demand.html

    I will check out MakeHuman for a bonesetup and see if it can handle external modeled faces - (my model is just a head and shoulders with accessories - no arms/legs/biped). Does MakeHuman weight the vertices for the bones for you??

    thanks!
    Last edited by sami; 06-23-2020 at 05:11 AM.

  10. #10
    Registered User sami's Avatar
    Join Date
    Jul 2003
    Location
    TopLeft Viewport
    Posts
    1,036
    They have a sample head at the polywink link I mentioned above with the 160 face shapes. I'll try that quickly as a stand in - but as they are so pricey, I'm concerned that if there's a minor error in their output, I'd have to repay to get it redone... I'll see how hard it is to use their demo head with that Unity lip sync plugin. But I don't need 160 blendshapes.... Does anyone have a favorite or best practive viseme/emotion list?

    https://www.polywink.com/index.php?c...d_attachment=5

    Like all the basic 5 phoneme shapes plus happy, angry, sad, surprised? Like what do you all use for an "essentials list" ?

  11. #11
    Registered User
    Join Date
    May 2012
    Location
    Virginia
    Posts
    545
    Quote Originally Posted by sami View Post
    Hi RPSchmidt!

    Thanks very much for that - that's very cool - I'm sure it saves a lot of time with keyframing. And it's great to see Robbie making such a cool plugin new now when most developers aren't these days and the prices is amazing too.

    But I don't want to animate in LW - I'm animating in Unity and the main thing is actually creating the mouthshapes and emotions. Unless I missed it in the video, I don't see how this workflow helps create those. I am not familiar with Adobe Character Animator, but I don't think it creates viseme / blendshapes of your own models - Like I can't load my own character model in it and it won't create morphs blendshapes for me based off of facial geometry??

    Appreciate the heads up with this tool, but I don't think it will work for this time. Good to keep it in mind though.

    I was mostly hoping for some thing, a tool or guide, or even a comprehensive best practices list of what the essential visemes/emotions are for a face and how to modify your neutral face to get them.

    Do you know of a comprehensive and best practice list? I haven't seen anything of what I call complete on YouTube or anywhere. Nor in paid tutorials...?
    I see. Well, I don't use Unity to do facial animation, but in the vein of using Robbie's plugin with a game engine, I was thinking more along the lines of using it (and Character Animator) to get your base visemes and animate them in Lightwave, then perhaps export the animations to Unity.

    So, as an example, if you used Robbies plugin and Character Animator and recorded yourself doing the viseme sounds (using a phoneme to viseme chart - like this https://www.researchgate.net/figure/...G_tbl1_4273506) then you would have your base viseme animations and could bake those animations.

    I don't know enough about Unity to know how they map them, but if you named your baked viseme animations in a way that Unity recognizes, then would that give you the base lip sync animation set?

  12. #12
    Founding member raymondtrace's Avatar
    Join Date
    May 2003
    Location
    Ohio
    Posts
    1,148
    Quote Originally Posted by sami View Post
    ...It feels like you have to use their base model and export from there?...
    Correct. My mention of MakeHuman/DAZ/iClone was just to use as a template for adjusting your own mesh in LW. It is a one-way flow out of those apps. I do not believe they would ingest something from LW and then apply any morphs.


    Quote Originally Posted by sami View Post
    ...I will check out MakeHuman for a bonesetup and see if it can handle external modeled faces...
    My thought with MakeHuman is simply to take their bone setup as a template and adjust it to your own mesh. You'll still need to improve upon the rig. It is a cheaper (but less functional) option than RH's FaceFactory.

    I'm not familiar with Unity. Your options may be limited to Unity's support for morphs or to the number of bones.

    EDIT: As I read Polywink's web page I realize what they're doing in the background. This would be a great feature request for Modeler: Imagine Modeler allowing you to set a viseme cage on a layer, adjust some control points in that cage to match your head mesh on the background layer, click a button and Modeler starts generating morphs from predefined morphs in the viseme cage.
    Last edited by raymondtrace; 06-23-2020 at 07:29 AM.
    LW4, 7.5D, 2015, 2018, 2019, 2020 running portably on a USB drive on an Amiga 2500 running Wine.

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •