PDA

View Full Version : Digital Eye Replacement



gamedesign1
05-18-2017, 11:16 AM
Hi All

This isn't speciacally LightWave related, but I find this community very helpful.
I am making some silicone puppets with manual pulley eye mechanisms, but the eyes only move left and right to make it easier for one puppeteer to operate. For most scenes the left and right movements are fine, but occassionally I need the eyes to look up or down. So rather than making a more complex eye mechanism I thought it would be good to digitally replace the eyes in 3D. I am not sure if this is going to be easy enough to do or not, so I was wondering what you guys think would be a good way of tackling this accurately.

Thanks :)

Markc
05-18-2017, 11:56 AM
For starters, you will need some tracking software (Syntheyes/PFTrack) to track the footage and match the camera motion.
Then you can composite the digital eyes onto the footage.
There is possibly a tool in Hitfilm (much cheaper, there might even be a free version) that could do this.

gamedesign1
05-18-2017, 12:13 PM
For starters, you will need some tracking software (Syntheyes/PFTrack) to track the footage and match the camera motion.
Then you can composite the digital eyes onto the footage.
There is possibly a tool in Hitfilm (much cheaper, there might even be a free version) that could do this.

Thanks for your reply :)
I have used tracking software in the past, but I was unsure about how to replace the eyes wihout rotoscoping them by hand. Maybe I could make some greenscreen covers for the eyes so I can key them out.

Markc
05-18-2017, 12:21 PM
If you needed depth to the eyeballs, you would have recreate the head also.
This is assuming you have 3d digital eyes.
If it's 2d, you might get away with something in After Effects just placed over the real eyes.

The more you think about it, the more technical hurdles appear 8~

gamedesign1
05-18-2017, 12:54 PM
If you needed depth to the eyeballs, you would have recreate the head also.
This is assuming you have 3d digital eyes.
If it's 2d, you might get away with something in After Effects just placed over the real eyes.

The more you think about it, the more technical hurdles appear 8~

Really don't want to recreate the head in 3D for this project because it is going to be purely puppet based, but with CG backgrounds. Yeah it does seem tricky doesnt it. The eyebals will need to be 3D to match the footage as best as possible. But you are probably right, for some shots I might get away with 2D eyes using AE :)

Marander
05-18-2017, 02:05 PM
Hello gamedesign

You could use a DAZ model (DAZ3D itself is free and includes basic Genesis 1,2 and 3 character templates). Create a Genesis 3 male or female. You can export as FBX 2012, load it in LW and only use the eyeball geometry and UV map (and textures if you want), delete the rest. All free and pretty accurate with Cornea, Iris, Sclera etc. The Genesis 3 topology is quad only and subd friendly.

gamedesign1
05-18-2017, 04:35 PM
Hello gamedesign

You could use a DAZ model (DAZ3D itself is free and includes basic Genesis 1,2 and 3 character templates). Create a Genesis 3 male or female. You can export as FBX 2012, load it in LW and only use the eyeball geometry and UV map (and textures if you want), delete the rest. All free and pretty accurate with Cornea, Iris, Sclera etc. The Genesis 3 topology is quad only and subd friendly.

Thanks, but I have eye models already. Thanks anyway though, cheers for your reply :)

Ma3rk
05-18-2017, 06:40 PM
I just did this sort of thing but with a live action shot. Director wanted the eyeline shifted about 30 degrees from looking Cam left to more straight on. What made it really difficult though was that the actress starts off with sunglasses on & as she removes them is tossing her head up and to the right. Fortunately I didn't need to add another layer of complexity in having to use digital eyes as there as another shot from the same cam position & set that I was able to use.

Did the work in After Effects and used Mocha Plus to do the tracking. I also used MamoWorlds excellent companion script MochaImport+ for several aspects of bringing in tracking data, mask shapes, but most importantly to create a stabilized precomp that could then be corner pinned and applied to the head tracking.

For you puppet work, it sounds as if you've all sorts of options for creating elements & matting if need be.

gamedesign1
05-18-2017, 07:10 PM
I just did this sort of thing but with a live action shot. Director wanted the eyeline shifted about 30 degrees from looking Cam left to more straight on. What made it really difficult though was that the actress starts off with sunglasses on & as she removes them is tossing her head up and to the right. Fortunately I didn't need to add another layer of complexity in having to use digital eyes as there as another shot from the same cam position & set that I was able to use.

Did the work in After Effects and used Mocha Plus to do the tracking. I also used MamoWorlds excellent companion script MochaImport+ for several aspects of bringing in tracking data, mask shapes, but most importantly to create a stabilized precomp that could then be corner pinned and applied to the head tracking.

For you puppet work, it sounds as if you've all sorts of options for creating elements & matting if need be.

Thanks. Yeah there are a few options. Need to do some tests.

ianr
05-19-2017, 09:35 AM
There was A Fuse ( Plug-In) for BlackMagic Design Fusion

written specially for this, it's first Outing was the Movie 'Teen-Wolf'

If you get a bit of budget for this tricky 'Uncanny Valley' area, the eyes!

Then contact Rony Soussan @ BMG Fusion.

see this VVG @ https://www.youtube.com/watch?v=QxrNzHs5H28 (3:20 mins In)

pinkmouse
05-19-2017, 10:30 AM
I'd stick to practicals. Much, much easier and faster.

Got access to a 3d printer?

https://www.youtube.com/watch?v=Y0upri54Jhs

gamedesign1
05-19-2017, 12:38 PM
I'd stick to practicals. Much, much easier and faster.

Got access to a 3d printer?

https://www.youtube.com/watch?v=Y0upri54Jhs

Yeah I did think about that, but I need the puppet running on pulleys only (no motors) and all operated by one puppeteer. The pupeteer needs to operate the mouth and head turning, eyes and one arm. Not sure how I am going to give all that movement to the pupeteer if the eyes have two axis. The puppet hardly ever needs to look up or down anyway. Dark Crystal puppets didn't look up and down and they got away wih it :)

erikals
05-19-2017, 02:46 PM
number of shots? 20? or 2000?

SynthEyes can do an alright job, but like suggested i'd rather do this practical. easier/faster and looks better.

also, out of Cam shots are no fun in SynthEyes.

gamedesign1
05-19-2017, 04:02 PM
number of shots? 20? or 2000?

SynthEyes can do an alright job, but like suggested i'd rather do this practical. easier/faster and looks better.

also, out of Cam shots are no fun in SynthEyes.

probaly no more than 10 shots. I would do all practical if I have more than one puppeteer on each puppet, but I don't. So I think I will make the eyes just do the left and right movement, which will be fine for 90% of the shots. And then try and do eye replacement for the odd occasion that I might need the eyes to look up or down. Thanks everyone for your ideas on this :)

erikals
05-19-2017, 04:26 PM
yes, sound like tracking could be an idea for that, maybe check this video >
https://www.youtube.com/watch?v=vhkdqk_AVfM

i've only tested that technique a bit, so can't guarantee the results, but worth a look.

Surrealist.
05-22-2017, 06:40 PM
I am not sure I am following the logic of painting yourself in a corner at the planing stage. This is the time to be athinking about solutions to problems. I would strongly consider a rig to control the eyes. My2c.

gamedesign1
05-22-2017, 08:04 PM
I am not sure I am following the logic of painting yourself in a corner at the planing stage. This is the time to be athinking about solutions to problems. I would strongly consider a rig to control the eyes. My2c.

Yeah I have been thinking about it over past few days and I am going to add both axis control to the eyes so they can look up and down. And then if its not possible for one pupeteer to do both movements then they can just use the left and right. At least this way the option is there.

Surrealist.
05-23-2017, 12:04 PM
Cool.Best of luck with it. Sounds like a fun project.

gamedesign1
05-23-2017, 06:36 PM
Cool.Best of luck with it. Sounds like a fun project.

Thanks :)

Whelkn
05-24-2017, 08:20 PM
https://www.youtube.com/watch?v=JQAab-bUWi4

I did that a few years back just using fusion that is free now. once its all set up its easy to animate the eyes to do whatever you want.