View Full Version : MoCap affordable for LW?

Ernie Berger
06-12-2011, 01:15 PM
I saw something today on CNN that made a lightbulb turn-on in my brain.

Some company makes a game (apparently popular and now in stores) in which a player stands in front of a video screen and dances or otherwise moves his/her feet and arms. (Sorry if this inexpensive MoCap is old stuff to you, but I am 63 years old .... not a 20-something gamer.) On the screen, a character does (body, feet and arms) exactly what the player does in real time. It is clearly motion capture "on the cheap." It must not cost much, as typical consumers can afford it.

Surely someone can make the necessary technical links that would enable us LightWavers to capture those motion files and apply them to LightWave ...... using the actual components (physical devices) used in that game (probably available at Best Buy).

My brain is screaming, "I'll buy it, if someone (a person with a high-tech brain) builds one or tells LightWavers how to make the link"

I know nothing about programming, but surely many of you have the skills to use the LW CORE to make this happen. Has anyone done this?

06-12-2011, 02:09 PM
http://www.ipisoft.com/ uses the Kinect.

There are a number of folks using it with http://www.newtek.com/forums/showthread.php?t=101917 to bring the motion into Lightwave.

Ernie Berger
06-12-2011, 02:53 PM

Thank you for that reply and the links.

Best regards,

06-12-2011, 03:07 PM
even cheaper . . . http://www.brekel.com/ mocap is not as good as IPi , but i would suggest buying PS3 EYE CAMERA instead of a kinect a) cheaper b)better rez .


06-12-2011, 05:25 PM
iPi Software is about to release a dual-Kinect version of iPi DMC which is expected to resolve much of the occlusion issues, and produce results comparable to what we currently get with multiple PS3 Eye cameras. Soon, the main difference between the two systems will become the size of the capture space. (The Kinect's effective capture space is a lot smaller compared to the PS3 Eye setup.)

According to a recent post from the developer, the release of the first dual-Kinect version is maybe a month away.

I should point out that dual-Kinect is not exactly a 'cheaper' option since you can buy several PS3 Eye cameras for the price of just one Kinect but the convenience you get with using two Kinects will certainly make it worth considering.

That said, either iPi setup is significantly cheaper than any other commercial mocap system out there. If you have the space and time but don't have a lot of money, I highly recommend this system.


Ernie Berger
06-12-2011, 06:54 PM

Many thanks! This is all very helpful.


06-12-2011, 07:29 PM
You guys realize that currently the Kinect does NOT return head, neck, hands or feet bone rotations. We've had to use a webcam and FaceAPI to return head rotations, but still have to do hands/feet manually.

There are other limitations like really fast motions where you turn around quickly, plus there's a fair amount of jitter that needs to be smoothed.

06-12-2011, 09:20 PM
Actually iPi DMC's motion tracking system is able to track feet rotations. Well at least the toe bend; in all my tests, it's clear when the heel goes up, the toes are still level to the floor. I have to admit I was surprised when I first saw this, and it's been this working this way for some time. What it can't deal with is a certain amount of twisting in the thighs. So far this hasn't been a common problem for me but you can really see it breaking the rig in one of my demo videos.

It's true that iPi DMC does not capture head rotations yet but this is something they've been working on for a while now. At first I wasn't too concerned by the lack of head tracking (adding head motion is fairly easy,) but when we started working on our Brudders music video, I suddenly wished we did have it because there's been a lot head-bobbing to music in the footage we recorded. :p

The developers did figure out how to get shoulder tracking last year, which was surprising since this is a markerless system. If they can do that, I imagine head tracking isn't far behind. (Note: even though shoulder tracking is working, I still find it helpful to use Motion Builder's 'Realisitic Shoulders' option to boost this motion a bit more.)

The DMC system does not track hands and probably never will, though I'm still hopeful for capturing wrist rotations some day. Somewhat related is props tracking, which I understand is something else they are working on. I imagine if they can track hand held props, this data might be also used to interpolate wrist rotations.

Jitter is indeed an issue with raw capture data but iPi DMC's Configurable Jitter Removal system is really quite good and I haven't had much issue with jitters since last fall or winter. In general, I need to add more CJR to the feet than other parts of the body, and the results are very steady and natural without looking at all 'robotic.'

BTW, I need to add that I'm mainly speaking from experience with PS3 Eye capture. That said, the motion tracking system in iPi DMC is essentially the same for either PS3 Eye or Kinect. (I also use the Kinect version but haven't put nearly as much time into this version of the system yet. Now that things are slowing down a little at work, I hope I can finally spend more time with it.)


Edit: as for quick movements, yeah that's a big problem with Kinect. This is mainly because it's a 30fps device while PS3 Eye is a 60fps device which can reduce motion blur significantly. (Technically, PS3 Eye can do 120 fps but the result is apparently too grainy at that speed to be useable.)

06-12-2011, 11:18 PM
I should clarify, I know nothing about the iPi system - but have done a roll-your-own version with the OpenNI drivers and the Kinect. Those drivers don't report head rotations/neck bends and not really "good" spine bends either. They don't do hand/wrist rotations (forget fingers) nor feet bends (and of course not toe/ball bends). I've been using this for realtime mocap -- is iPi only for non-realtime/recorded video?

I'm waiting for the Microsoft driver release - which they promised in Spring 11 but their page still has a sign up to be notified thing and their drivers which will have a bunch more features and higher resolution for the Kinect will potentially be awesome....

06-13-2011, 12:13 AM
I've been using this for realtime mocap -- is iPi only for non-realtime/recorded video?
Yes, that's correct. The focus has been more on accuracy over realtime performance, so the purpose of the program may be a bit different. That said, for a non-realtime system DMC can be reasonably fast at motion tracking. Using a Geforce GTX 460 I'm getting it to track at about 0.6 seconds per frame. (Which is quite a big improvement over a couple of years ago in its pre-GPU days, when it took about 10 to 12 seconds per frame.)

I haven't benchmarked the tracking speed of iPi DMC with Kinect but I imagine it's faster than multiple PS3 Eye cameras since there's less data to process. (Though it might go up again once the second Kinect is added to the system. Will have to wait and see.)

Sigh! I was hoping to do Kinect tests tonight but I got distracted by other testing (again!) Maybe tomorrow night. :p


06-13-2011, 12:29 AM
I'm waiting for the Microsoft driver release - which they promised in Spring 11....
Yes, I'm curious to see if iPi Software adopts that. They started out using OpenNI but later switched to the PrimeSense drivers.

Speaking of PrimeSense, I see on Wikipedia that they've teamed up with Asus to develop a Kinect-like device for PC's for second qtr 2011. Interesting times. :)