PDA

View Full Version : Up-Rezzing Animation Footage



Megalodon2.0
05-08-2013, 02:51 PM
Currently I'm rendering animation in 32bit EXR's out of LW and bringing it into Fusion for compositing.

Does anyone know of a plugin or app that can up-rez footage - for example HD720p to HD1080p? It doesn't HAVE to be a plugin for Fusion, but is there anything "out there" than can help to resize images so that they don't look like you ran them through the blender - i.e. jagged edges. Ideally I'd like to uprez from 720p to 4k, but I doubt anything would go that high and still look decent.

Any suggestions - aside from "why don't you render at that resolution - would be appreciated.

Thanks!

nickdigital
05-08-2013, 03:30 PM
There's this but it's not really for uprezzing footage. More for anti-aliasing.
http://www.olm.co.jp/rd/technology/tools/?lang=en

This is decent but it's Photoshop.
http://www.ononesoftware.com/products/perfect-resize/

There's this but it's AfterEffects
http://www.topazlabs.com/enhance/

Megalodon2.0
05-08-2013, 09:27 PM
Enhance looks good. The other two probably not for what I need.

I've also looked at Black Magic Teranex and Resolve - definitely potential there.

Hopefully there are other plugins or apps that are good at this sort of thing.

raw-m
05-09-2013, 03:53 AM
Take a look at Red Giants plugins:

http://www.redgiant.com/products/all/magic-bullet-instant-hd/

Greenlaw
05-09-2013, 10:37 AM
A few years ago, I used Boris Uprez (http://www.borisfx.com/units/Image-Restoration.php) in Fusion and I thought it did a decent job at the time. This is a x32 plug-in for AE so you will need to run it in x32 Fusion. (Like LightWave, I still keep both x32 and x64 versions online in such cases. A little annoyed that Vegas Pro is only x64 now.) I'm not sure you can purchase the UpRez plug-in separately anymore though--looks like it only comes with a suite now.

BTW, I was looking for a plug-in that works with Fusion x64 or Vegas 12 Pro (x64) a few weeks ago myself, so if anybody has a suggestion for those programs, please post. I believe I can use one of the Re:VisionFX tools in Fusion for this (probably SmoothKit; I have the AE plug-in that works in x32 Fusion) but I'm really more interested in a Vegas compatible plug-in so I can process an entire project at once.

G.

Greenlaw
05-09-2013, 10:41 AM
Oh, I just realized that all the Boris stuff is available for Vegas Pro. The suite is more than I wish to spend right now so other suggestions are still welcome. :)

G.

Megalodon2.0
05-09-2013, 02:21 PM
Take a look at Red Giants plugins:

http://www.redgiant.com/products/all/magic-bullet-instant-hd/

Nice, but quite limited in the applications I use. Still, I will probably download the demo and see how well it works with AE CS4. I have a couple Red Giant plugins.

Thanks for this!


A few years ago, I used Boris Uprez (http://www.borisfx.com/units/Image-Restoration.php) in Fusion and I thought it did a decent job at the time. This is a x32 plug-in for AE so you will need to run it in x32 Fusion. (Like LightWave, I still keep both x32 and x64 versions online in such cases. A little annoyed that Vegas Pro is only x64 now.) I'm not sure you can purchase the UpRez plug-in separately anymore though--looks like it only comes with a suite now.

BTW, I was looking for a plug-in that works with Fusion x64 or Vegas 12 Pro (x64) a few weeks ago myself, so if anybody has a suggestion for those programs, please post. I believe I can use one of the Re:VisionFX tools in Fusion for this (probably SmoothKit; I have the AE plug-in that works in x32 Fusion) but I'm really more interested in a Vegas compatible plug-in so I can process an entire project at once.

G.

I actually did find this in my search - it's another one I will probably download the demo. I'm not sure if I'd rather work it in Fusion or at the end in Vegas. I guess it would depend upon which method would end up looking the best overall.

I have my two-part pilot episode rendered at 720p but was asked - if possible - to provide it in 4k. Yikes. I did uprez my 30 second promo directly in Vegas to 4k, but I'm sure that the quality is not what it could be. Then again, uprezzing ANYTHING from 720p to 4k is probably ludicrous. I certainly don't want to render everything at 4k - it would take me YEARS. I plan on rendering at 1080p and essentially doubling the resolution to 4k - hopefully THAT will be acceptable. Something tells me that if I want it to be REALLY as good as it can be, I'm going to have to go the hardware route - something like a Black Magic Teranex. Of course if we're making decent money... why not? :)

Thanks!

Greenlaw
05-09-2013, 02:59 PM
4k, yikes! Are they screening it for IMAX?

G.

Megalodon2.0
05-09-2013, 03:56 PM
All I was told was that Sony is interested in 4K content and EuTelSat and SES in Europe want 4K for content to run on their demo channels.

I just want to get my "stuff" out there to as many available outlets as possible. :)

kmacphail
05-10-2013, 05:57 AM
Last year at my old studio we were working on footage for a new ride at Universal, stereo 4K at 60fps. We ended up rendering dailies in stereo 2K at 30fps, then delivered stereo 2K at 60fps up-rezzed to 4K in Nuke. IIRC the up-rezzed 4K footage was indistinguishable from the same rendered at 4K when projected in the venue.

Best of luck with your project.

-Kevin

Megalodon2.0
05-10-2013, 04:16 PM
Last year at my old studio we were working on footage for a new ride at Universal, stereo 4K at 60fps. We ended up rendering dailies in stereo 2K at 30fps, then delivered stereo 2K at 60fps up-rezzed to 4K in Nuke. IIRC the up-rezzed 4K footage was indistinguishable from the same rendered at 4K when projected in the venue.

Best of luck with your project.

-Kevin

Interesting. I haven't tried to uprez in Fusion. I'll give that a shot and see how it looks.

Thanks for sharing your method!

Cryonic
05-16-2013, 10:40 PM
720p to 4k means an uprez of about 5x. That means each pixel in your renderings will become 25 or so in the final frames. Not sure how sharp the image will remain from that level up magnification.

Megalodon2.0
05-16-2013, 11:28 PM
720p to 4k means an uprez of about 5x. That means each pixel in your renderings will become 25 or so in the final frames. Not sure how sharp the image will remain from that level up magnification.

Yeah, it won't look that good at all. I'd love to render to 4k but the small renderfarm here wouldn't handle it well at all. Best I'll be able to do it 1080p and uprez to 4k.

m.d.
05-16-2013, 11:29 PM
that's a major uprez...
the blackmagic hardware wont do it any better then fusion...probably not as good....just in real time

if your going to do it....lanczos is the best algorithm to use as far as fusion is concerend
I wouldnt bother with any 3rd party stuff....all they can do is use one of the public algorithms, and quite often they are not as good as straight old lanczos

heres a little comparison from a while back
http://www.infognition.com/articles/video_resize_shootout.html

EDIT:
AE alsos uses lanczoz 3 if you have a CUDA card installed...otherwise it jumps back to a modified bicubic

Megalodon2.0
05-17-2013, 01:11 AM
that's a major uprez...
the blackmagic hardware wont do it any better then fusion...probably not as good....just in real time
Thanks for this info - I had a feeling this was for realtime, but I wasn't sure if it would still do a better job. Now I know.

if your going to do it....lanczos is the best algorithm to use as far as fusion is concerend
I wouldnt bother with any 3rd party stuff....all they can do is use one of the public algorithms, and quite often they are not as good as straight old lanczos
I'll give it a go in Fusion with lanczos. Fortunately they only needed it for the 30 second promo. I figured if I had to do it for the entire episode I would try to re-render at least in 1080p.

heres a little comparison from a while back
http://www.infognition.com/articles/video_resize_shootout.html

EDIT:
AE alsos uses lanczoz 3 if you have a CUDA card installed...otherwise it jumps back to a modified bicubic
Figures, don't have a CUDA card yet - was planning on getting one when I started playing with TFD.

And thanks for the link - excellent for comparisons. Of course I know that 720p to 4k is not going to look good. I don't see why it's important right now anyway since most don't have 4k TV's - at least AFAIK.

kmacphail
05-17-2013, 08:55 AM
Wouldn't a conversion of 720p to 4K be more of a 3X uprez, 1280720 to 38402160 for 4K Ultra HD, or an increase of 1 pixel to 9? Still likely a noticeable jump but not quite as drastic.

kadri
05-17-2013, 08:44 PM
I tested some times ago Genuine Fractals , Video Enhancer and the Warpresize plugin in Virtualdub.
Video Enhancer is not bad but what got me surprised was how similar Genuine Fractals and the Warpresize in Virtualdub was.
Probably too similar! It looked to me that they simply used the same way.

I did not tried the ones in the other packages but give Warpresize a try too.
It looks a little artificial but it depends on the material and the resize factor you will use of course.
not sure about 720 to 4K but from full HD to 4K it could work maybe.

m.d.
05-17-2013, 11:58 PM
just get the software CSI uses to uprez security cam footage....I think its called 'image enhance' or something

then just render 640x480 and uprez to 4k :)

interesting about warp resize...the only thing I dont like about virtualdub is its limited to 8 bit (AFAIK)....

this is a good description how it works....a smart guy could probably make a fusion macro to do this
http://www.virtualdub.org/blog/pivot/entry.php?id=79

Greenlaw
05-18-2013, 12:04 AM
heres a little comparison from a while back
http://www.infognition.com/articles/video_resize_shootout.html

Thanks for posting that. It's very useful information!

Wow, Super Resolution in Virtual Dub is pretty amazing. I wonder what it does between cuts though, since it extracts details from frames around the current frame.

G.

erikals
05-19-2013, 04:10 AM
just tried Infognition Video Enhancer (Super Resolution) and indeed, it's quite powerful...!
have a look at the examples attached below >

(also worth a read > http://forum.infognition.com/index.php?topic=174.0)

erikals
05-19-2013, 05:33 AM
at times lines can be too jagged, if so, i guess you can mask it / combine it with another uprez plugin or such...

m.d.
05-19-2013, 10:22 AM
Thanks for posting that. It's very useful information!

Wow, Super Resolution in Virtual Dub is pretty amazing. I wonder what it does between cuts though, since it extracts details from frames around the current frame.

G.

all of these up rez effects can be useful depending on the shot...but they are no magic bullet...
to apply it to an entire sequence would apply horrendous artifacts....

I am pretty sure a lot of shots would also completely fail on their own with these techniques

But, still handy to have in the toolbox

For all our regular uprez needs...Lanczos performs pretty well...if not the best.

m.d.
05-19-2013, 10:34 AM
just tried Infognition Video Enhancer (Super Resolution) and indeed, it's quite powerful...!
have a look at the examples attached below >

(also worth a read > http://forum.infognition.com/index.php?topic=174.0)

the guys at video enhancer try to sound like they have some proprietary tech......

this same up rez technique has been used in the foundry furnace plugins for years...
http://www.thefoundry.co.uk/products/furnace/features/

look under smart zoom

all of these resize techniques come from either research papers or....publicly available algorithms
stuff like magic bullet instant HD, in the previous incarnation was worse then plain old lanczos....yet they sold a lot of copies....now it is probably just lanczos with sharpen....something you can do easily in virtualdub or fusion (funny not so easy in AE or PS)

Greenlaw
05-19-2013, 12:19 PM
This is all excellent information. I've been wanting to up-rez 'Hello Frankenstein' (shot on an old DV tape camcorder) for some time and I may try a combination of techniques depending on a given shot. Same goes for my current demo reel--the more recent footage on the reel is HD but a lot of the older stuff was rendered in SD for pre-HD TV broadcast, so I decided to output everything in SD. When I have time, I may try up-rezzing the older stuff and output to HD (720p).

G.

m.d.
05-19-2013, 01:41 PM
You'll have to watch that one as well....

I have seen incredible softening just going from a PAL still to NTSC....nothing to do with the framerates....just because we were going from square pixel to a .9 aspect pixel

You will really have to watch image quality as you will have to de-interlace, change pixel aspect ratio, and depending if it was 24pfs apply the pulldown in the file. Not to mention if it was DV or Digital 8 or such it would be 4:1:1 video so you are uprezzing the blue and red channels that are already effectively 400% uprezzed

I would probably rescale first....leaving the pixel aspect as is, so you are rescaling from unmolested footage with no degredation applied from the pixel aspect correction, or the de-interlace.
Then I would fix the pixel aspect.....(720*2=1440, 1440/1280=1.125.....the .125 means pixels again have to be averaged and interpolated)

And again watch the de-interlace algorithm...a lot like to just repeat fields...which mean they just basically use half the resolution.
And even if you shot 24p...on a DV type camera it is 24p over 60i....so its interlaced

Make sure to use an uncompressed 444 codec...or something like cineform 444

its one of those things if your not careful will end up looking like mush....

Greenlaw
05-19-2013, 03:19 PM
Hi m.d.,

Thanks for these additional tips.


You'll have to watch that one as well....

I have seen incredible softening just going from a PAL still to NTSC....nothing to do with the framerates....just because we were going from square pixel to a .9 aspect pixel

You will really have to watch image quality as you will have to de-interlace, change pixel aspect ratio...
Neither will be a problem for 'Hello Frankenstein'--it has already been converted to 1:1 for Vimeo and YouTube, and the original footage was shot progressive. I had one of the few 'cheap' DV camcorders that could do that at the time. I think it was actually cheating this by merging and throwing out data from 60fps but it was good enough for keying and composting, well, at least appropriate enough for the budget and general 'cheesiness' of the production. 'Hello' is 'ancient history' for us anyway, so I'm not going to be too fussy about it.


Make sure to use an uncompressed 444 codec...or something like cineform 444
Good idea! I can finally make use of my NeoSceneHD license. :)

G.

erikals
05-19-2013, 07:26 PM
the guys at video enhancer try to sound like they have some proprietary tech......

this same up rez technique has been used in the foundry furnace plugins for years...
http://www.thefoundry.co.uk/products/furnace/features/

look under smart zoom

true, but that plugin is $5000... (!) not $40, like Infognition Video Enhancer...


all of these resize techniques come from either research papers or....publicly available algorithms
stuff like magic bullet instant HD, in the previous incarnation was worse then plain old lanczos....yet they sold a lot of copies....now it is probably just lanczos with sharpen....something you can do easily in virtualdub or fusion (funny not so easy in AE or PS)

yes, lanczos works pretty good as well, haven't compared it to Instant HD2 though...

indeed, none of them are a magic solution, but some of them are interesting and should be considered,
or even combined...

kadri
05-19-2013, 08:36 PM
Some of you guys might have used this in productions many times probably.
Some TV shows i saw had some bad looking comps .
Maybe because of up sizing (or other factors like too much blur etc.i do not know) of some elements.
Mostly i doubt that up sizing too much in itself will be useful , but together with other original size elements it should be Ok.
I used in this basic logo animation video , only half size Terragen 2 , background and cloud renders ,
with an original HD size Lightwave rendered parachuter and logo (the sound is not made by me in this video).
http://www.youtube.com/watch?v=NzGQy6abebo
The Youtube quality here is bad unfortunately. Look at the attached original image please .
If i had rendered the background and cloud in full HD in Terragen 2 my render time would be gone from 1,5 month or so to 4-6 months easily .
Render times in Lightwave were only 3 days or so in HD.
But of course this scene doesn't needed sharp backgrounds very much to begin with.

114427

m.d.
05-19-2013, 08:43 PM
wow looks good....
ya those kinda render times you really have no choice but to uprez

m.d.
05-19-2013, 08:45 PM
Hi m.d.,

Thanks for these additional tips.


Neither will be a problem for 'Hello Frankenstein'--it has already been converted to 1:1 for Vimeo and YouTube, and the original footage was shot progressive. I had one of the few 'cheap' DV camcorders that could do that at the time. I think it was actually cheating this by merging and throwing out data from 60fps but it was good enough for keying and composting, well, at least appropriate enough for the budget and general 'cheesiness' of the production. 'Hello' is 'ancient history' for us anyway, so I'm not going to be too fussy about it.


Good idea! I can finally make use of my NeoSceneHD license. :)

G.

LOL...i bought prospect 4k back when it was $2000....and then forked out another $500 for another license...

A year or so later it was $250....

Still great codec...I was editing 4k with it back in 2007 realtime on a quad core PC

m.d.
05-19-2013, 08:54 PM
true, but that plugin is $5000... (!) not $40, like Infognition Video Enhancer...



yes, lanczos works pretty good as well, haven't compared it to Instant HD2 though...

indeed, none of them are a magic solution, but some of them are interesting and should be considered,
or even combined...

the foundry plugins were always overpriced...but some of the tech seemed pretty magical back in the day...and there was a lot more then just the resize stuff

clearly they are way cheaper, and I may buy a license in the next few days to help uprez some sub SD stuff to 1080p for a client....
at least it is a little more then just a combination of different resize algorithms (which I think instantHD is) and something you cant do on your own with freely available tools

They are using this filter
http://www.compression.ru/video/super_resolution/super_resolution_en.html

Just wrapped a nice GUI around it

I wish someone would make this into an after effects plug....and give it higher then 8 bit support

kadri
05-19-2013, 08:58 PM
wow looks good....
ya those kinda render times you really have no choice but to uprez
Thanks! Yes unfortunately .
Since i read a post from Mr Rid (i think) about using half rez hypervoxel renders in scenes i wonder how much cheating is going on :)

erikals
05-20-2013, 12:14 PM
compared Lanczos3 (usually better than Lanczos8) to Infognition and the Infognition is indeed better, sometimes as much as 30%.

note, i highly recommend getting rid of all AA and flicker before you upscale...
two good post plugins are OLM plugins and Neat Video DeNoiser...

OLM PhotoShop
http://youtu.be/9f8OkFkxIHE

NeatVideo DeNoiser After Effects
www.youtube.com/watch?v=-zV7wHlPo38#t=1m50s