Page 1 of 4 123 ... LastLast
Results 1 to 15 of 48

Thread: Setting up Kinect One for NevronMotion in LightWave 2015

  1. #1
    Eat your peas. Greenlaw's Avatar
    Join Date
    Jul 2003
    Location
    Los Angeles
    Posts
    7,279

    Setting up Kinect One for NevronMotion in LightWave 2015

    A lot of NevronMotion 1.1 users seem to be having trouble getting up and running with KinectOne motion capture so here's a walkthrough that may help get you started.

    Introduction

    First, a little info about the Kinect and its history.

    Kinect One is the latest version of the Kinect sensor from Microsoft. A Kinect sensor is a game controller that uses infrared rays to generate a 3D depth map of what's in front of it, in addition to recording traditional rgb video and audio. Some software can use the depth data to create point clouds which can be used to create 3D objects or to track motion in 3D space. Most notably, it can be used to capture motion from a human performer.

    There are basically three models of Kinect:

    Kinect for XBox (aka Kinect v1) was the original device created for the XBox. If you have an adapter cable, you can use this with a PC running Windows 7.

    Kinect for Windows (also called Kinect v1) was the version specifically made for Windows based computers. It was similar to the XBox version but it had better video controls and offered a special mode called Near Mode, which was meant for face capture and small object scanning.

    The original Kinect for XBox and Kinect for Windows were discontinued after the release of the Kinect One for XBox One. You might still be able to find these devices but they're becoming rare in the marketplace.

    Kinect One and Kinect for Windows 2 (aka Kinect v2) are essentially the same device. Kinect One was developed for the XBox One game console, and it was shortly followed by Kinect for Windows 2 which came with an adapter to allow PC users to use the device. Since the devices were otherwise identical, MS decided to drop Kinect for Windows 2 to eliminate some of the market confusion.

    If you get a Kinect One to use with NevronMotion, you will need to purchase a $50 Windows USB interface kit separately. You can buy this kit directly from Microsoft or from online retailers like Amazon or NewEgg. Kinect One (or v2) requires Windows 8.1 or higher--if you're still on Windows 7, you can upgrade to Windows 10 for free until next summer (2016). Kinect One also requires a dedicated USB 3.0 port and a graphics card that supports DirectX 11 or higher.

    The Kinect One (v2) differs from the original Kinect (v1) in that it supports higher resolution and a wider FOV. This means it generates cleaner tracking data and you can use it in a smaller room. The range of the IR sensor is about the same but since the lens covers a wider area, you actually get a larger capture space to perform in. Because the depth data is cleaner, the capture is more accurate and less jittery.

    However, like the original Kinect, a single KinectOne sensor is still subject by occlusion issues so some complex motions cannot be accurately tracked (like full body turnarounds.) Occlusion issues can be eliminated by synchronizing multiple Kinect sensors and placing them around the performer. The official Kinect SDK does not support capturing from multiple devices but it is possible with third party software like iPi Mocap Studio. Mocap Studio allows you to capture from up to three Kinect (v1) sensors or up to four Kinect One (v2) sensors, and additionally records data from multiple PS Move or Wii Motion Plus controllers for head and hands tracking. (We use three Kinect v1 sensors and three PS Move controllers for our animated short films and vfx work for feature films. We recently started using two v2 sensors with three PS Move controllers for our animated shorts. For more information about how we do this, see my Brudders Production Log.)

    In this article we'll cover the basics for setting up and using the Kinect One (v2) with NevronMotion 1.1 in LightWave 2015.3.

  2. #2
    Eat your peas. Greenlaw's Avatar
    Join Date
    Jul 2003
    Location
    Los Angeles
    Posts
    7,279
    Setting Up The Kinect One and NevronMotion

    As mentioned above, you will need to be running Windows 8.1 or higher to use a Kinect One sensor. (If you must stay with Windows 7, you will need to get the original Kinect v1 sensor. There is more information about setting up a v1 sensor in the NevronMotion manual.) For Kinect One compatibilty, you will also need an open USB 3.0 port (KinectOne is not compatible with USB 2.0) and a video card that supports DirectX 11 or higher. Note that if you have a really old motherboard, even if it is USB 3.0 compatible, it might not support the required throughput for Kinect One capture. For example, my five-year-old workstation can just barely support the Kinect One at 30fps but my brand new Windows tablet computer can capture Kinect One data effortlessly. We have several older computers that will recognize a v2 sensor but none of them will work adequately with it.

    In case your'e still wondering, no, you do not need an XBox or XBox One game console to use a Kinect sensor (v1 or v2) with a Windows computer.

    To run Kinect One on a Windows computer, you will need to install the Kinect 2 SDK. you can download it from here:

    Kinect For Windows SDK 2.0

    After you've installed the required software and have connected your Kinect v2 sensor, you can check compatibility by running the Kinect Configuration Verifier Tool. This tool is meant for verifiying Kinect v2 compatibility and it is not used for Kinect v1. You can download the tool from here:

    Kinect Configuration Verifier Tool

    Install the verification tool and run it. If everything checks out, you should see a panel that looks like this:

    Click image for larger version. 

Name:	KinectForWindowsConfigurationVerifier.png 
Views:	586 
Size:	39.5 KB 
ID:	131220

    Ideally, every section in this panel should have a green check mark next to it.

    Note that in my case, the verifier could not identify the USB 3.0 port in my tablet computer but it still recognized it for compatibility. If you open the bottom tab labeled Verify Kinect Depth and Color Streams, you should see the live Depth and RGB video streams from the sensor and its current frame rate. The sensor must maintain a steady rate of 30 fps, otherwise you will not be able to capture motion accurately.

    If you've gotten this far, congratulations--you're halfway there!

  3. #3
    Eat your peas. Greenlaw's Avatar
    Join Date
    Jul 2003
    Location
    Los Angeles
    Posts
    7,279
    Installing NevronMotion 1.1 for LightWave 2015.3

    NevronMotion is a retargeting tool for LightWave 11.6 and later versions. It is available as a separate add-on product for 11.6 and as of November 2015, NevronMotion comes free with LightWave 2015. On the Windows platform, you can also use NevronMotion to capture motion from a Kinect sensor. The latest NevronMotion version 1.1 adds support for Kinect One for XBox One. If you are a registered user of LightWave 2015, you can download NevronMotion and a license key from your user account.

    NevronMotion can be installed a few different ways. Please read completely through this section before installing the software.

    Note that if you have previously installed NevronMotion, you will need to uninstall it before installing version 1.1. Failing to do so may result in unpredictable behavior.

    The first installation method suggested in the documentation is to simply drag the NevronMotion folder to the Plugins folder in your user directory. This directory is located here:

    C:\Users\[your account]\.NewTek\LightWave\2015.3\plugins

    The advantage with this installation method is that the plugins will be automatically loaded to LightWave without altering your LightWave installation. However, this method of installation did not work on my tablet computer as Virtual Studio in Layout was not able to fully access the Kinect One device. I don't know why this method did not work for me. My guess is that Windows 10 may be denying LightWave permission to read the data from this directory properly--but I really don't know this for a fact. Note that this installation method apparently works for some users. If it doesn't work for you, try the next method. (Or just go straight to the next method if you don't want to risk it.)

    The method that did work for me was to install NevronMotion as a regular LightWave plugin. You can, of course install a LightWave plugin from any directory. In my case, I have a folder called Plugins where I keep all my third party plugins:

    C:\Program Files\NewTek\LightWave_2015.3\plugins

    I dragged my NevronMotion folder to this directory and added the plugins as normal. You can add the NevronMotion plugins by using the Edit Plugins - Scan Directory command and pointing it to the NevronMotion folder. After doing this, quit Layout to make sure LightWave writes the addition to your plugins config file.

    Note: LightWave 2015.3 has an annoying feature where it will sometimes create a new menu tab called Additional with the the most recently added plugins listed, and it will probably create this tab on your system after you add NevronMotion. If this happens, you may wish to open the Menu Edit window and move the two NevronMotion menu items to a more appropriate menu section. (I have a tab called User for my third party and custom tools.)

    After installing NevronMotion, you will want to copy the Nevron_Genoma_Rig.lwo and Nevron_Genoma_Rig.txt to this location:

    C:\Program Files\NewTek\LightWave_2015.3\support\genoma\rigs\ Complex_Rigs\Bipeds

    This makes the KinectOne rig immediately available to Genoma's presets in Modeler.

    Now copy the three Kinect_Skeleton rig files to your LightWave content directory. The files are called:

    • Kinect_Skeleton_SKL0.lws
    • Kinect_Skeleton_SKL1.lws
    • KinectOne_Skeleton_SKL0.lws

    These are LightWave scene files that already have the Nevron_Genoma rigs loaded and configured for Kinect capture. The first two are for Kinect v1 capture and the third file is for Kinect One (v2).

    On our home studio network, we have a shared directory called Projects where we maintain all our Lightwave project (content) directories. In Projects, we have a folder called NevronMotion specifically for these scene files. Because these are template files, you may want to lock the files so you don't accidentally overwrite them.

    Lest we forget, let's do one more thing before moving forward: Launch Layout and drag your NevronMotion License Key into the window. Layout should alert you that the license key has been added to the registry. If you didn't do this previously, NevronMotion is now officially activated on your computer.

    Now quit Layout.

  4. #4
    Eat your peas. Greenlaw's Avatar
    Join Date
    Jul 2003
    Location
    Los Angeles
    Posts
    7,279
    Accessing The Kinect One In LightWave

    You may want to quit LightWave and restart your computer at this point to make sure the necessary drivers are loaded properly.

    Now launch Layout.

    In the upper section of the toolbar, you will notice these two items: Virtual Studio and Studio Live. These are the tools you will use connect Layout and NevronMotion to the KinectOne.

    Under Virtual Studio, select Device Manager. It should look like this:

    Click image for larger version. 

Name:	DeviceManager.png 
Views:	582 
Size:	33.1 KB 
ID:	131223

    Open the toggle button next to Kinect XBOX ONE and double-click on the nested item. Rename this item KinectOne_A. The exact spelling and format is very important for proper operation because it allows NevronMotion to correctly identify the device and associate the incoming data with the Nevron Genoma skeleton. It also allows you to easily share your capture data with other NervonMotion users following the same naming convention.

    Now, make sure HID, Kinect XBox One, and KinectOne_A are enabled. Click in the corresponding boxes under Enable. A check mark should appear next to each item as in the above image.

    If you click on KinectOne_A from the list, you will see the motion data streaming in from the sensor. Note that the incoming data is shown only as numeric values, not a streaming video feed like with Kinect v1. The current release of NevronMotion (1.1) does not allow you to access the Kinect One's control and options panel like it does for the Kinect v1. However, the documentation suggests that this feature may be coming for KinectOne in a future release of NevronMotion.

    You may now close the Device Manager.

  5. #5
    Eat your peas. Greenlaw's Avatar
    Join Date
    Jul 2003
    Location
    Los Angeles
    Posts
    7,279
    Dance Monkey Dance!

    Load the file called KinectOne_Skeleton_SKL0.lws. You should see a skeleton standing in the center of the world.

    Now, under Virtual Studio, select Studio. A window will popup that lists all the connections between the joints of the internal Kinect skeleton and the LightWave skeleton in Layout. Your screen should look something like this:

    Click image for larger version. 

Name:	RigReady.png 
Views:	370 
Size:	223.5 KB 
ID:	131230

    If you're curious about how this works, double-click on any of the items and you will see the nodal connection for that item.

    Hang on, we're almost there.

    Before we can make the skeleton move, we need to activate these connections. Each item should have a check mark in the column labeled A. If the check marks are missing, shift select all the items in the list and click in the column. There should now be a check mark on every item. To activate the rig, click on the Active and Live buttons at the bottom. If you are going to record and play the capture motion, you will also want to enable Allow Record and Allow Play.

    Click image for larger version. 

Name:	Studio.png 
Views:	274 
Size:	26.5 KB 
ID:	131231

    The rig may suddenly collapse but don't worry, that's normal. What's happened is that the Kinect is trying to find and track a human form and when it can't find one, it drops the rig into a pile.

    Click image for larger version. 

Name:	Collapsed.png 
Views:	534 
Size:	223.2 KB 
ID:	131229

    So now what? Well, let's give it a human body to track. The Kinect One sensor needs to be facing the center of the room, and the space should be relatively clear of objects like chairs and coffee tables. Stand in the center of the room so the sensor can see you and assume a T-Pose. If the above instructions have been followed correctly, the skeleton in Layout should instantly snap into position and mimic your motions.

    Click image for larger version. 

Name:	2015-11-26_093717.png 
Views:	334 
Size:	196.4 KB 
ID:	131228

    This is a screen cap of my daughter waving. As a nine-years old, she's only four feet tall but the software tracked her without manual adjustment. We took turns stepping in front of the sensor, and the Kinect One immediately adjusted for our height differences. Pretty neat.

    The next steps are to record a motion and to retarget the motion to a character. The procedures for doing so are well explained in the NevonMotion user manual so please look there for further information.

    So there you have it. I hope this tutorial helps gets some users started. Now get moving!

    G.

  6. #6
    Thank you for your help and all the work it represents ! I'll study this today !
    Le Jardin Digital - Créations 3D / Design /Films - www.lejardindigital.fr

  7. #7
    Super Member
    Join Date
    Jun 2010
    Location
    Derbyshire, UK
    Posts
    456
    This looks great, thanks Greenlaw

    Going to try this out this evening.

    Do you know if there support for facial capture with Kinect v2?

    Cheers

  8. #8
    Adapting Artist jasonwestmas's Avatar
    Join Date
    Mar 2003
    Location
    North Carolina
    Posts
    11,399
    Thanks Greenlaw I didn't notice there was a KinectOne Skeleton or that there was no video feedback. Thanks for pointing those out. The original video tutorial is obviously useless for this kind of setup so I was easily confused.

    I hope they get that video feature working. It's kinda hard to visualize my own occlusion when I'm doing the motions. Merging two kinect scans for the same skeleton would make this a whole lot more useful too I think. Plus attaching geometry to the skeleton would help a ton in understanding which poses I can get away with without making the joints skip around.
    Last edited by jasonwestmas; 11-27-2015 at 10:07 AM.
    All that is powerful or long standing is first conceived in the imagination; supported by the hope of possibility and then made manifest in our commitment of our current physical reality.

  9. #9
    Super Member
    Join Date
    Jun 2010
    Location
    Derbyshire, UK
    Posts
    456
    Quote Originally Posted by jasonwestmas View Post
    Thanks Greenlaw I didn't notice there was a KinectOne Skeleton or that there was no video feedback. Thanks for pointing those out. The original video tutorial is obviously useless for this kind of setup so I was easily confused.

    I hope they get that video feature working. It's kinda hard to visualize my own occlusion when I'm doing the motions. Merging two kinect scans for the same skeleton would make this a whole lot more useful too I think. Plus attaching geometry to the skeleton would help a ton in understanding which poses I can get away with without making the joints skip around.
    You can use Kinect Studio which is part of the Kinect v2 SDK to give you the visual feedback.

  10. #10
    Adapting Artist jasonwestmas's Avatar
    Join Date
    Mar 2003
    Location
    North Carolina
    Posts
    11,399
    Quote Originally Posted by Simon-S View Post
    You can use Kinect Studio which is part of the Kinect v2 SDK to give you the visual feedback.
    Cool, that does help quite a bit thanks. It's nice to see an actual representation of the body in space so I can tell how accurate the capture is.
    All that is powerful or long standing is first conceived in the imagination; supported by the hope of possibility and then made manifest in our commitment of our current physical reality.

  11. #11
    Registered User
    Join Date
    Jul 2013
    Location
    Brisbane
    Posts
    110
    This is really helpful and much easier to follow than the native LW instructions.

    I still get a "The program can't start because Kinect10.dll is missing from you computer. Try reinstalling the program to fix this problem" after launching both layout and modeler. which is very odd.

    I can see the the sensor in Studio>device manager
    i have renamed the nested item as described
    both show as ready when ticked
    the only data stream that seems to be active below is body underscore time

    i should say i have a uber cam installed and that also placed a VR headset device in the virtual studio environment, turn this on or of seems to have no effect on the kinect sensor.

    checking the sensor with the other MS SDK sample proves the sensor is working ok,

    i am thinking i have to resolve the dll issue and the rest will fall into place.

    I will post if i get a resolution, any advice or pointers from other appreciated.
    If you cant help but make an art of it.

  12. #12
    Registered User
    Join Date
    Jul 2013
    Location
    Brisbane
    Posts
    110
    Quote Originally Posted by Zerowaitstate View Post
    This is really helpful and much easier to follow than the native LW instructions.

    I still get a "The program can't start because Kinect10.dll is missing from you computer. Try reinstalling the program to fix this problem" after launching both layout and modeler. which is very odd.

    I can see the the sensor in Studio>device manager
    i have renamed the nested item as described
    both show as ready when ticked
    the only data stream that seems to be active below is body underscore time

    i should say i have a uber cam installed and that also placed a VR headset device in the virtual studio environment, turn this on or of seems to have no effect on the kinect sensor.

    checking the sensor with the other MS SDK sample proves the sensor is working ok,

    i am thinking i have to resolve the dll issue and the rest will fall into place.

    I will post if i get a resolution, any advice or pointers from other appreciated.
    Issue resolved

    So the Kinect10.dll error is resolved by installing "NevronMotion 1.0 Kinect for Windows Driver 1.7 Setup" Silly me for assuming the already installed SDK2.0 would cover this.

    Sucess ! ! !

    Things that caught me up ....

    - did not install driver from the LW site /Face Palm
    - did not initially have HID ticked in Virtual studio > Device manager
    - did not have KinectOne_Skeleton_SKL0.lws loaded ( i had earlier version which no doubt works with V1 sensor)

    this is so much fun i am planning to project on to the side of my house and have a two story Santa waving at cars and kids real time as they drive bye, should be a blast.

    Thanks for the the above instructions way more straight forward than the read me
    If you cant help but make an art of it.

  13. #13
    Eat your peas. Greenlaw's Avatar
    Join Date
    Jul 2003
    Location
    Los Angeles
    Posts
    7,279
    Sounds like fun!

    I'm glad these instructions were helpful.

    Regarding 'face mode', I'm going to guess it's not available yet. It will probably show up when the KinectOne options and controls are added in a future update. Hopefully, somebody from LW3G (Lino?) can post a more concrete answer here.

    G

  14. #14
    Eat your peas. Greenlaw's Avatar
    Join Date
    Jul 2003
    Location
    Los Angeles
    Posts
    7,279
    Oh, one more thing:

    When you're not using Virtual Studio, it's a good idea to click off Studio Live. Leaving Studio Live active when you don't need it can seriously affect the performance of LightWave. For example, try using VPR with Studio Live active and then try it with Studio Live deactivated. There should be noticeable difference in interactivity.

    Just an FYI.

    G.

  15. #15
    Registered User
    Join Date
    Mar 2015
    Location
    Katy, TX
    Posts
    92
    I can't seem to get the darned thing out of pretzel mode. Here's a video: https://www.dropbox.com/s/vzsgwc2j3s...Fail2.mp4?dl=0

    I have 1.8 and 2.0 installed... passed the verification... put the plugins in the user directory AND tried adding them using the scan directory. Numbers are scrolling by. Named properly. HID checked (and KinectOne x 2). All active in the studio. Nada.

    It ain't my machine either. Here's my specs:
    >MSI X99 chipset motherboard
    >Intel I7-5960X (8-core)
    >32GB RAM
    >NVIDIA Titan X card
    >Windows 10-64 Pro
    >(So I have plenty of power to run it)
    >Kinect SDK 2.0, & 1.8 installed

    I should be ripping through this, but I just CAN'T get the friggin' thing to work! HELP PLEASE! What am I doing wrong?

Page 1 of 4 123 ... LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •