PDA

View Full Version : using Syntheye's new Rolling Shutter capabilities in LW?



3dWannabe
08-22-2012, 09:41 AM
The developer of SynthEyes has added a method for dealing with rolling shutter to SynthEyes.

http://www.ssontech.com/content/RollingShutter.pdf

He basically solves while taking into account the rolling shutter (producing a far smaller error value), and can then produce an output so that a CGI program (like Lightwave?) could integrate CGI into the rolling shutter scene (using what is essentially a modified form of motion blur).

Would love to see this used in Lightwave (especially as I have a high rolling shutter RED Scarlet).

erikals
08-22-2012, 09:50 AM
on a side note, i know AE 6 removes rolling shutter easily.

3dWannabe
08-22-2012, 09:54 AM
on a side note, i know AE 6 removes rolling shutter easily.
You might want to read that pdf.

There isn't a program in existence, including the Foundry's plug-in, that will remove all rolling shutter artifacts from different objects moving at different speeds along with the camera movement.

The SynthEyes innovation involves getting better tracking with a rolling shutter scene, and then being able to modify a CGI scene composited into the rolling shutter scene so that it matches using a technique similar to motion blur.

erikals
08-22-2012, 10:05 AM
 
ah, cool...

(here is the AE6 video though > http://www.youtube.com/watch?v=-th1zJsmqVY)

but yep, that SynthEyes method sounds interesting...
"Common Misconceptions" in that .pdf enlightened the problem...

 

biliousfrog
08-22-2012, 10:38 AM
Although I welcome any methods that can deal with rolling shutter, I find it strange that anyone would prefer to add the hideous effects of using a CMOS sensor to CG footage rather than remove it from the video footage...no matter how good/bad AE or the Foundry's tools are. I guess it's going to become the new 'must have' effect to simulate bad cinematography. :D

Thomas Leitner
08-22-2012, 10:56 AM
We did this back in 2009 with LightWave 9.
We had to integrate a CG car into a shot filmed with a consumer HD cam. Since there was a huge movement the rolling shutter artifacts was huge too (the wheels of the CG car sticks to the ground but the top seems to wobble).
We solved the camera movement with syntheyes and rendered the rolling shutter with the Advanced Camera in LW. The downside was that the render times was much longer than the render times with the Perspective Camera. I donīt know if this issue is solved in newer versions of LW.

ciao
Thomas

erikals
08-22-2012, 10:58 AM
it all depends on the shot imo...

and real film is out of the question, unless you have $$$ x10...

Thomas Leitner
08-22-2012, 11:44 AM
The downside was that the render times was much longer than the render times with the Perspective Camera. I donīt know if this issue is solved in newer versions of LW.

I just did a quick test render with LW 11.1 (to be honest, it wasnīt really quick): unfortunately the time issue is still there.

ciao
Thomas

wellsichris
08-22-2012, 12:14 PM
Thomas, can you give us a quick example of what the setup is for rolling shutter?

thanks
Chris

Thomas Leitner
08-22-2012, 12:52 PM
Thomas, can you give us a quick example of what the setup is for rolling shutter?

thanks
Chris

This a screenshot of the camera settings:
Item is your camera.
Time Sweep depends on the solve of your camera matchmove (if the CG element sticks to the ground, I think "Range down image end now" is the right choice) and the time value is based on exposure time.

ciao
Thomas

wellsichris
08-22-2012, 02:45 PM
thanks, yeah that is slow, works, just really dang slow,

m.d.
10-07-2012, 11:00 AM
it all depends on the shot imo...

and real film is out of the question, unless you have $$$ x10...

Just for everybodys information...

Film also has a mild rolling shutter effect....usually equal to about 4ms....
It can't not have it....

the film is sliding through the plate vertically as the shutter is open....exactly what is happening during CMOS sampling

cmos cameras are getting close to that number, it is hard to find real specs on sensor readout rates but they are getting close...

and I wouldnt call the scarlet a high rolling shutter camera, it does have some...but compare that to a 7d or worse iphone to see real high rolling shutter

3dWannabe
10-07-2012, 12:29 PM
Just for everybodys information...

Film also has a mild rolling shutter effect....usually equal to about 4ms....
It can't not have it....

the film is sliding through the plate vertically as the shutter is open....exactly what is happening during CMOS sampling

cmos cameras are getting close to that number, it is hard to find real specs on sensor readout rates but they are getting close...

and I wouldnt call the scarlet a high rolling shutter camera, it does have some...but compare that to a 7d or worse iphone to see real high rolling shutter
I own a Scarlet.

It has 14ms rolling shutter, about equal to an inexpensive DSLR - a figure confirmed by the author of SynthEyes when the calculations were presented to him.

He stated "14msec is comparable to a budget CMOS camcorder doing 30i, and that can get ugly."

So the Scarlet with 14 ms is a LOT of rolling shutter, especially for use with CGI.

It's a bit of a sore spot with me, as RED specifically told me "I have confirmed with our engineers that the read/reset rate for both EPIC and Scarlet will be the same." in a February 2012 email after the Scarlet had started shipping but before I received mine.

jwiede
10-07-2012, 02:40 PM
thanks, yeah that is slow, works, just really dang slow,
You pay for all that configurability, it means the camera has to do more on-the-fly 3D calculations. Seems like an opportunity for someone to produce a LW camera specifically optimized around rolling shutter effect (either allow adjustment of ms, or provide a few set values if better for optimization), but without incurring the cost of the other configurability possible with the Advanced Camera. Maybe a new camera for the Liberty3D guys' UberCamera pkg?

jwiede
10-07-2012, 03:26 PM
Hmm, raises an interesting question: If producing motion vectors, does using the Advanced Camera include the rolling shutter distortion in the motion vectors produced? Or are the motion vectors produced still "idealized" (no rolling shutter distortion)? Eventually I can imagine wanting separate motion vector buffer and rolling shutter distortion vector buffer output, or at least ability for vector buffer to control which are included (combined if both).

m.d.
10-08-2012, 03:25 PM
I own a Scarlet.

It has 14ms rolling shutter, about equal to an inexpensive DSLR - a figure confirmed by the author of SynthEyes when the calculations were presented to him.

He stated "14msec is comparable to a budget CMOS camcorder doing 30i, and that can get ugly."

So the Scarlet with 14 ms is a LOT of rolling shutter, especially for use with CGI.

It's a bit of a sore spot with me, as RED specifically told me "I have confirmed with our engineers that the read/reset rate for both EPIC and Scarlet will be the same." in a February 2012 email after the Scarlet had started shipping but before I received mine.

I own an epic, so while not the same i know a bit of what you speak of

and although 14ms seems somewhat large...compare it to a 7d and its half or less
the foundries recommendation for 5d is .56, and Red one .32 (not ms just an arbitrary number)
while that does not directly compare to the scarlet...i would assume values closer to the red setting then the 5d....
if you go by these numbers the original RED has approx half the read reset time of the canons....so not quite in dslr range, and your running the same sensor just with more power


I could see why you would have a problem with this...especially if you spoke to engineering and they said it would be the same value

also as far as 14ms...you may want to run your own tests...with the scarlet I am pretty sure they used sensors that didnt quite pass test's to put them in epics. Much like most intel and AMD procs are the same die...they just run tests on batches and declock the ones that dont cut it for the top end samples....makes sense economically...point being your mileage may vary..as it is the sensor that sets the read reset times.

I'll have to do a test and see what the epic actually does....

3dWannabe
10-08-2012, 03:45 PM
I own an epic, so while not the same i know a bit of what you speak of

and although 14ms seems somewhat large...compare it to a 7d and its half or less
the foundries recommendation for 5d is .56, and Red one .32 (not ms just an arbitrary number)
while that does not directly compare to the scarlet...i would assume values closer to the red setting then the 5d....
if you go by these numbers the original RED has approx half the read reset time of the canons....so not quite in dslr range, and your running the same sensor just with more power


I could see why you would have a problem with this...especially if you spoke to engineering and they said it would be the same value

also as far as 14ms...you may want to run your own tests...with the scarlet I am pretty sure they used sensors that didnt quite pass test's to put them in epics. Much like most intel and AMD procs are the same die...they just run tests on batches and declock the ones that dont cut it for the top end samples....makes sense economically...point being your mileage may vary..as it is the sensor that sets the read reset times.

I'll have to do a test and see what the epic actually does....
The clocks and internal chips are slower in the Scarlet (as in the supporting circuitry is cheaper as it's spec'ed at a lower speed - just like the fastest i7's cost more). Some might be able to run faster, but they have the Scarlet clock set to a certain frequency, and it's not going to go higher, even if some of the chips could (unless some mods are made in China).

It would generate a massive customer relations problem if some Scarlets were faster than others.

I ran the calculations by the developer of SynthEyes, and he thought they were valid, as well as the method to obtain the values.

You can see the actual calculations and test shots shown starting at post #34 and #37 and #41 in the following thread.

http://www.reduser.net/forum/showthread.php?73308-Rolling-shutter-different-for-Scarlet

I'd be interested in what your Epic shows for the same calculations - as I've never heard of an 'official' read-reset time for the Epic.

m.d.
10-08-2012, 08:10 PM
I have heard the epic was at 9ms but they dialed it back to 11ms....for consistency....this is what I heard...and cannot confirm until I actually test.
Kinda what I was saying about clock speed...but you are right...they may have capped it at 14ms to be consistent among all the scarlets...but it is the sensor that governs the read reset....the other components play into the FPS
I definately would not question your calculations, as I don't own one....but as you know Russ is a perfectionist and the rolling shutter is not all that bad in practice....you have to do a fairly fast whip pan to make any problems in compositing....
The bigger issue is with a tracking solver....which he has the fix for in the latest build(syntheyes for those not in the know)

I have been experimenting with the advanced camera(which is how I found the thread) to deal with certain shots...which can be an issue...
If it is a big problem you can shoot and crop to 2 or 3k....the RS is the same but the effect will be half....as will the rez

Thomas Leitner
10-09-2012, 08:08 AM
the film is sliding through the plate vertically as the shutter is open...
No, when the shutter is open the film stands still.

ciao
Thomas

m.d.
10-09-2012, 10:10 AM
Your right of course....it is the shutter moving vertically through the plate....
Rolling shutter is still there...until you go global shutter which right now means CCD

Imageshoppe
10-13-2012, 06:43 PM
Your right of course....it is the shutter moving vertically through the plate....
Rolling shutter is still there...until you go global shutter which right now means CCD

I think this example animation I did for the CML forum back in '07 when the first RED's hit the street can clear up the confusion regarding film and skew...

http://ftp.datausa.com/imageshoppe/outgoing/SHUTTER_STUDIES/FILM_SHUTTER_SKEW_CLASSIC.mov

The bottom line is that any skew in a fast panning or tilting film frame is only at the beginning and end of the exposure cycle, as opposed to continually happening over the full exposure with rolling shutter CMOS. The center of the exposure time with film is completely rock solid and mimics the global shutter style. The out of focus nature of the shutter as it passes over the film plane will reduce the effect of skew even more than the animation example shows.

Regards,

Jim Arthurs