Mocap in LW - how to best use with existing rigs?

[Phil]

I'm toying with the idea of using Mocap in LW, and would like to use it with existing (mostly carefully constructed) rigs on characters. I don't really know how to achieve this - anyone here who might be kind enough to share some tips, content or other insights?

I hunted through LW content and found nothing covering this scenario.

[Greenlaw]

I've been working on a getting my workflow down in a 'tutorial' format. It might be a while before it's ready though because I'm also trying to finish up the project I developed it out for. In the meantime, you might want to check out the recent Animeeple (http://newtek.com/forums/showthread.php?t=101917&page=25)and iPi Studio (http://newtek.com/forums/showthread.php?t=114498&highlight=iPi) threads here in the LW forums where I've layed out bits and pieces.

In general, I've been using FBX and Animeeple to retarget mocap data to my own rigs. There's a bit of nonsense to deal with to make this work but with the upcoming LW 10, things should get a bit smoother. Once the FBX is in Lightwave, I use MultiMotion Save and Load (http://thespread.faulknermano.com/_theoldspread/index.htm) to move all the bones animation to my custom rig. MultiMotion is less fussy about different rig structure than the built-in Load Only Motion Envelopes feature. To edit the retargeted mocap, I use Animeeple for simpler stuff; for more complicated motion, I use Lightwave's native animation tools.

Lately, I've been trying IK Booster to edit mocap. The IK-FK tools seem to work well, but for some reason I'm having trouble blending and baking keys with IK Booster. Specifically, my keyframe indicators are not showing up in the IK Booster timeline for me to blend, even though the edited keyframe is obviously there when I scrub the animation. Probably user error on my part, but I'm a a loss right now. If I can't get this to work, I might have to start looking at Motionbuilder sooner than I wanted to. (I've been avoiding MB to keep the cost down for my project but also, for at least two reasons I won't go into here, I have an aversion to giving Autodesk more $$$.)

Anyway, that's it in a nutshell. Ill let you know when I have the super detailed version of the workflow online. In the meantime...

Rebel Hill has a very good tutorial on FBX and the requirements of mocap skeletons here: Lightwave and FBX (http://rebelhill.net/html/lwandfbx.html)

And then there are Spline God's Using Mocap in Lightwave and IK Booster videos.

Cageman and dballesg have some good tutorials online too. Do a search in these forums for their names and 'mocap'.

Hope this helps.

G.

[Phil]

Oddly, animeeple hates the mocap - it doesn't seem to know what to do with the bvh file - it complains that the names are incorrect. I've attached one of the examples from TrueBones. The TrueBones examples are also odd because they all seem to have been saved with a height suited for a person 70 m tall (both as far as LW and also Animeeple are concerned).

[Greenlaw]

Oddly, animeeple hates the mocap - it doesn't seem to know what to do with the bvh file - it complains that the names are incorrect. I've attached one of the examples from TrueBones. The TrueBones examples are also odd because they all seem to have been saved with a height suited for a person 70 m tall (both as far as LW and also Animeeple are concerned).
One thing I learned is that you have to be very careful about consistent naming of right and left hierarchies. The symmetrical sets need to be exactly matched or Animeeple with not load the rig. Things to look for are naming mismatches like 'r' on one side and 'left' on the other, or characters like '_' and '-'.

Also, Animeeple and Lightwave share the same scale, but .bvh data comes into Animeeple at x100 scale by default. Set your scale to .01 for .bvh imports; Animeeple will remember this the next time you import a .bvh file. (Note: This is a new Animeeple feature so make sure you have the latest version.)

If you like, tell me which Truebones .bvh file you're having a problem with and I'll take a quick look. (I probably have a copy of that data here.)

G.

Edit: Duh! I knew you made an attachment. I'll download the file now.

[Greenlaw]

Hi Phil,

The file you posted is importing to Animeeple properly here. I had to set the scale factor was .025 though, not the usual .01 I've experienced with my own mocap data. I wish I knew why .bvh data does this. AFAIK, height information isn't part of the .bvh file format, so I'm not sure why this varies with some .bvh files.

The attached images might help you in getting data into Animeeple. The first imge shows how to characterize the rig for Animeeple. The left column is the .bvh bone names and the right column is what Animeeple expects. Start with the Hips and work your way down the list. The second image just shows the data retargeted to a different character and rig in Animeeple.

BTW, the character model seen in the second image is the new default biped in Animeeple. You can create it by selecting Create Biped from the Scene menu. It's very useful for quickly testing mocap data.

G.

Edit: One more thing: Double-click the base of the biped to open the Animation timeline, then double-click the Animation Block in the timeline. This enables the Animation tab. In the animation tab, disable Fix Feet and Blend on Ground. These features attempt to lock the feet in place, but they can be a bit tweaky. Sometimes they work well, and other times it's better to leave them off.

[Phil]

Odd. I tried again and this time see entries underneath 'Unassign'. It seems that the Animeeple UI is a little peculiar.

[Phil]

The next challenge is to figure out how to actually retarget the motion on to an underlying Maestro rig (which was carefully constructed). That part is not obvious to me at the moment. I'll take a look at your thread :)

[Greenlaw]

Yes. It took me a while to 'get it' myself.

Some fields, for example, don't accept data the way a normal Windows field does; to change the Scale in .bvh import, I usually have to delete the existing value before I can enter new value, or something weird like that. I'll have to ask the developer about it.

Apart from a few interface quirks, the program has been useful to me and the developer has been very helpful.

G.

[Greenlaw]

The most important concept for me was understanding that mocap joints and bones can't have rest rotations or other Lightwave specific 'offsets' applied to them. Tools like Rest Pivot Rotation and Bone Tools can totally hose a rig for programs like Animeeple and Motion Builder.

It's been a while since I've watched it but I think the Rebel Hill video explains why.

[Phil]

What I need to figure out is how/whether I can apply mocap to a Maestro rig. That seems to be potentially difficult. I'm summoning my courage to give it a try....

[Greenlaw]

I haven't tried that myself but I know there is a Motion Builder option in Maestro. This version of the Maestro rig probably has the properly (for mocap) zero-ed out bones. If Eric sees this post, maybe he can weigh in.

G.

[Phil]

BTW, thanks for your help and advice - it's much appreciated. I'll futz around with the Maestro rig and see if Animeeple with play with it.

I guess there shouldn't be a huge issue, but I suspect the workflow will need the load/save motion approach, due to Maestro's use of expressions on the rig. Remapping inside Animeeple would seem to break that expression implementation.

[Greenlaw]

I think you can bring the FBX of the Maestro rig into Animeeple. Animeeple should mostly ignore bones that are not part of its expected rig hierarchy. Then, when you bring the rig back into Lightwave, you can use the MultiMotion plug-in to export the motion data from all the bones (or joints as the case may be.) MM exports a .mot file for every item in a selection using the item's name for its filename. Next, open a scene with your Maestro rig, select the same bones (selection order is not important,) and use MM to load those motions (just point it to the directory and it will figure out which .mot goes where.) Now you should have the mocap on your original Maestro rig and have all the magical Maestro controls available.

Tip 1: If you do this a lot, set up bone selection filters for your rig using the Item Selection List.

Tip 2: Don't forget to add a 'dummy' joint at the end of every bone chain before exporting your rig. This solves a problem with LW FBX, where it will delete the last joint and duplicate the parent and reposition it (I think at the 'bone' tip which will probably be different from the actual joint position of the original joint.) The dummy joint will make the FBX tools do the bad stuff to this extra joint, which you can delete when you're finished. This is a pain to deal with, but hopefully it will get fixed soon.

Well, that's it in theory anyway...I've only done this using my own custom rigs.

One more thing: I don't know which version of LW you're using but if you have access to LW 10, its FBX support is better. If you use an earlier version of LW, you may want consider using the 'legacy' FBX plug-in.

G.

[Phil]

I opted to take small steps. I had a fair battle to get a set of skelegons out that I could use to match what was in the original bvh (NT have never seen fit to handle the return trip from bones to skelegons, despite many requests), but now my problem is that Animeeple seems to choke on the 25 MB FBX exported from Layout 10. Several progress bars display and then I just get OS X's dreaded pinwheel..... I've pinged the contact email address at Animeeple to see if they can advise.

My plan was to confirm that a basic pipeline would work, to some degree, before investing major effort in to the exercise.

[Greenlaw]

Hi,

I needed to get bones back to Skelegons and found a way to do it. There's a plug-in called Load Bones (http://www.dstorm.co.jp/dsproducts/lw9/FreePlugins/Bone_Skelegon/) from DStorm. Basically, you load your scene into Modeler with it, and the result is a converted Skelegon structure based on the scene's bones. There are a couple of catches though.

First, while Load Bones is fully compatible with Modeler 9.6.x (and probably 10) it can only read bones from a Lightwave 9.3 scene file. The workaround for 9.6.x is to save your rig as a LW 9.2 scene (the earliest compatible option in Layout) and then import that scene to Modeler using this Plug-in.

Second, this plug-in is not fully compatible with Joints. More accurately, it will load the points based on the rest length of Bones and not necessarily the position of Joints, which can be very different if you've moved the Joints in Layout (bones rest lengths do not adjust automatically when moving Joints. This is also the reason why you can't use Bone Tools with Joints.) I've still managed to make this work for me. What I do is, after converting to Skelegons, I connect the endpoints that I know correspond to actual Joint positions using curves, and then use Convert Skelegons to make a new Skelegon rig in Modeler, and then I delete to old Skelegon. It's not a perfect solution but this is still a huge timesaver over rebuilding the Skelegon structure in Modeler from scratch.

Load Bones is yet one more useful third-party tool that should have been part of Modeler many years ago, and the need for it is more important now than ever because the plug-in has been 'breaking' since 9.5 and it's no longer being updated for new releases of Lightwave.

G.

[Phil]

It's also not available on Intel Mac... I had to find a Windows PC and saved a 6.0 copy of the scene. Animeeple support are helping out with the FBX file.

[Greenlaw]

If I can't get (IK Booster) to work, I might have to start looking at Motionbuilder sooner than I wanted to..
Ah! Had a breakthrough today. It was user error after all.

Actually, I carefully rewatched parts of Larry Schultz's Importing and Using MoCap Data in LightWave 3D webinar and realized that I've been looking for keyframes to show up where they're not supposed to. Duh!

Geez! I wish IK Booster was better documented; the interface is soooo not Lightwave (or anything else for that matter.) But now that I know better, it seems to be working well for what I need.

In other words, it looks like I've kept Motion Builder at bay for another day. :)

G.

[Greenlaw]

Oh, by the way, you might know this from my 'rants' in other mocap threads about 9.6's FBX IO, but the orientation of the rig and mesh gets changed when you import the rig. To make it work I've had to maintain two separate rigs and meshes.

This would be a real pain if you're using Maestro, and it's the reason why I recommend either going back to the old Autodesk FBX plug-in or waiting for LW 10 if you don't already have access to it. (I've been reconfiguring my workflow to use LW 10 for at least the IO portion of my workflow.)

G.

[Phil]

Very odd. I took the exported bvh skelegon and massaged it to fit my character. I then went to Layout 10 and converted all of the skelegons to bones, exporting this to FBX on the way.

Animeeple decides it doesn't like the FBX and cannot find any bones. I'm rather bemused by this.

If I take the original bvh file and export it from Layout as an FBX, the same problem exists. It seems like the FBX exporter does something unhelpful, or I am missing a step.

DAE export results in the same kind of problem, but with an additional reported error that the skeleton does not originate from a hip.

[geo_n]

What I need to figure out is how/whether I can apply mocap to a Maestro rig. That seems to be potentially difficult. I'm summoning my courage to give it a try....

Were you able to import a maestro rig in animeeple?
When I exported the mbrig or the malebiped rig from lw via native fbx and imported it in animeeple, I couldn't assign any bones. I checked the maestro rig in lw and the names don't have any _ underscore in the bone names. Animeeple doesn't recognize any rig from the fbx.
The fbx itself is ok having checked in max and actually surprised how intact it is.

[Phil]

Were you able to import a maestro rig in animeeple?
When I exported the mbrig or the malebiped rig from lw via native fbx and imported it in animeeple, I couldn't assign any bones. I checked the maestro rig in lw and the names don't have any _ underscore in the bone names. Animeeple doesn't recognize any rig from the fbx.
The fbx itself is ok having checked in max and actually surprised how intact it is.

I'm working from another angle right now - just to try and get a basic workflow defined. As such, I took one of the bvh files in to Layout and exported it using DStorm's load bones (or whatever it is called, as NT still don't permit roundtripping skelegons) in to Modeler.

After fixing up the busted parenting in Modeler so that everything is in a sensible hierarcy (and removing the gimbal bones), I saved that out with placeholder character geometry and exported the FBX from Layout.

Importing that in to Animeeple works OK. What I'm currently failing to understand is why my character remains rooted to the spot whilst the legs and arms, etc. move around based on the mocap that Animeeple has retargetted to my new character. That's odd and something that Greenlaw's experiments have not shown him - suggesting that I'm doing something stupid.

Once I get this all figured out, I can try and then find a workflow to map this on to a Maestro rig.

Small steps, really.

[Phil]

Very odd. I took the exported bvh skelegon and massaged it to fit my character. I then went to Layout 10 and converted all of the skelegons to bones, exporting this to FBX on the way.

Animeeple decides it doesn't like the FBX and cannot find any bones. I'm rather bemused by this.

If I take the original bvh file and export it from Layout as an FBX, the same problem exists. It seems like the FBX exporter does something unhelpful, or I am missing a step.

DAE export results in the same kind of problem, but with an additional reported error that the skeleton does not originate from a hip.

Greenlaw helped me in another place to understand this issue - Animeeple expects a single hierarchy with the hip bone at the top and the rest of the structure as one might expect.

For some reason, Layout imports my .bvh files with the feet and arms decoupled from this and so Animeeple decides to give up. Fixing that up in Modeler or Layout prior to exporting the character as an FBX is sufficient to get to the point where Animeeple will find the bone structure in the imported file.

[Phil]

I've been stumped for a while with this because I could import my skeleton just fine, and the bvh, but when I applied the imported mocap to the skeleton, I hit problems. The limbs would all move around, but the character would remain firmly rooted to the spot. A quick email to Animeeple support brings this answer, which I thought I should share :

--

Hi Phil,

Thanks for contacting us and sending us your file. When you import characters and animations, it's important to click on the T-pose tab in the import wizard and make sure the T-pose is standing on the ground. If it's not, the root motion won't be scaled properly (which what's going on here - because the root is close to the ground, the movement is getting multiplied by a small scale). Clicking the button that says "Drop to floor" will take care of it. I've attached your workspace file with this fix.

Hope this helps! Let us know if you have other questions.

Best,
- The Animeeple Team

[Phil]

So that works nicely, but Layout's DAE handling in version 10.0 is utter garbage. The characters imported, but are rooted to the spot. Taking the same DAE in to Maya, everything looks fine. Exporting as FBX to Layout 10 works fine.

Exporting to DAE from Maya, to new file, though, causes all bones to be reset to world origin. Gah.

Something is also adding goofy extra bones to the end of the legs as well. *mutter*