PDA

View Full Version : Temporal pixel shift rendering



vncnt
11-30-2019, 11:42 PM
Has anyone experimented with temporal pixel shift to cut render time?
For instance when double frame-rate is needed.

I was looking at the new Epson EH-LS500 UST projector that utilizes pixel shift and was wondering if this could be a useful additional features in LW2020+.

-FP-
12-01-2019, 11:32 AM
You can use Twixtor or a similar plugin in After Effects or any OFX host to stretch renders. About 200% is the most you can get away with before the artifacts become distracting. In other words, render 15 frames per second and get effective 30fps playback. It's not perfect but it will do in a crunch.

Another way to save render time is to forego motion blur in Lightwave and add it in post with the RSMB plugin or a suitable program's native tools. Lightwave's vector blur is not as good as the simulated blur provided by commonly available video plugins.

Dan Ritchie
12-01-2019, 01:10 PM
I'm pretty sure LW has a vector channel that could be used as a displacement map to shift pixels, if anyone wanted to write a plugin.
Howler does an exhaustive search and then does a displacement map. It would be much better if you knew the exact place where things are moving, which I believe the vector channel tells you.

erikals
12-01-2019, 02:39 PM
depends,

vector blur is good  (at times)
Ai Gigapixel upscalling is good  (at times)
Temporal denoise is good  (always, but just to a certain degree)
image denoise is good  (always, but just to a certain degree)
optical flow is good  (at times)

no secret sauce...  https://i.imgur.com/npb27rE.gif

vncnt
12-04-2019, 01:52 AM
I think pixel shifting should be the new interlacing because it creates much better results visually.
Without changing your render pipeline, it would let us to cut rendertime, or increase the frame-rate, or increase the resolution.

146342

At the moment there is no distribution pipeline or display standard for pixel shifted content but I'm shure we could use this in post production.

NewTek could add an additional option for Pixel Shifting to Interlaced and Progressive.
Maybe add a node to transcode render output into (standard) full resolution progressive?

I suspect it wouldn't be too complicated to add this feature to LW2020 but would it help you?

erikals
12-04-2019, 07:33 PM
edit; ok, saw a video.
(and image - https://www.flickr.com/photos/markgaler/25055206958/in/album-72157690734820075/)

sometimes, it seems like a temporal denoiser, just for resolution.
other times, it reminds me of a bicubic upscale, followed by sharpen.

from what i've seen, i'm not super-impressed.


i suggest also looking into Ai Gigapixels the video version.

vncnt
12-05-2019, 03:40 AM
They can't measure visual quality of pixel shifted video by comparising stills because the companion frame is missing.
With square (4x) pixel shifting they are missing even 3 frames.

Pixel shifting in post production will force you to increase frame-rate to get acceptable results:
2x more frames per second with diagonal pixel shifting (consumer projectors), and
4x more frames per second with square pixel shifting (as found in more expensive projectors).

With many professional projectors actually utilizing pixel shifting, why not adding support for a third video format to progressive and interlaced?
One that is suitable for high frame-rate.

With rendering for 8K 100/120 fps on the horizon, i think we're going to need pixel shifting because our CPU's can't cope and it's a lot better than those horrible interlace formats.

erikals
12-05-2019, 05:40 AM
ah, ok, get it, it's like interlaced just for pixels, sorta.

yeah, if they can find an efficient method, that would be very nice.

vncnt
12-05-2019, 08:49 AM
Another advantage for projected images is that pixel shift camouflages the LCD pixel grid so you end up with a much cleaner result.

For now it's too bad that pixel shifting is not a formal video standard that lives outside these projectors.
Even this 3k euro Epson does not accept such a format - for now it's all derived from the 4K signal - so this means there is no time difference between two "fields".

in the mean time we could search for applications in post-processing, with or without time differences between the 2 or 4 fields.
Interesting to see what field blending does to the result and its compatibility with progressive video.

Ryan Roye
12-05-2019, 03:53 PM
This is very commonly used VR applications, rendering two separate images at once is extremely resource intensive. Pixel shifting stops working well though when objects move too fast or the camera turns too quickly, you get smudging artifacts. For animated archviz, or slow panning shots it can save a ton of time though.

vncnt
12-06-2019, 11:08 PM
When comparing 25 progressive frames per second with 50 interlaced fields per second we're used to adjust panning speed and shutter time to soften these side effects.

Would that still be an issue with 100 fps or higher?
I suspect the pixel shifting would only be to our advantage in case of HFR.

These projectors use refresh rates >= 240 Hz, so HFR content would not be a problem in that part of the hardware.