PDA

View Full Version : Request: Image Sequence Options



Chrusion
04-07-2007, 07:00 PM
We need SE to support ALL image sequences the same as SE treats TGA sequences, BUT also need the option to select just a single image from a sequence to use as a still as well (for instance as a hold frame).

To that end, there probably needs to be an option to disable thumbnail caching/generation so that folders with thousands of images do not "lock up" SE as it cycles thru those hundreds and hundreds of files.

Chrusion
04-08-2007, 04:15 PM
... and if the current "folder as a sub-project clip" method is retained for img seq handling (in addition to the sequence as a clip for all img formats request above), then PLEASE show us a PROGRESS BAR as those images are ingested, cuz right now, SE goes into "Not Responding" mode from several to tens of minutes when a folder of several thousand images is dragged to the storyboard/timeline.

What takes SE so long to do 5,000 imgs anway? Vegas imports the same seq. in less than 15 seconds. SE is currently going on 20 minutes, thus making it the world's SLOWEST editor.

Oops, there are 2 or 3 separate sequences in this folder. I guess SE is going to import them all, cuz there isn't a way to tell SE to just import one of the named seq's. That option is mandatory as well.

.

Chrusion
04-08-2007, 04:34 PM
... passed 30 minutes now, so I checked the folder in Explorer and it did have only two jpeg sequences, one 5,400 frames and the other half that.

But, curiously, there's a new hidden folder in there named "Newtek Info" and it looks like SE is making a thumbnail cache at the blazing speed of 1 (one) "img_seq_filename.ext.WindowsMediaDesc" file (1KB) per 1.3 seconds. At this rate its going to take SE TWO HOURS 52 MINUTES to ingest these two sequences! This just ain't right at all.

NEWTEK, Please hear our (my) pleading cry for mercy and spare us this horrific performance bottleneck!

.

John Perkins
04-08-2007, 08:40 PM
That's actually a bug report item, not a feature request.

You don't have to request that we fix them, just help us to duplicate the problem here and we will happily fix it.

I'm exporting a sequence right now to verify, but if I can't duplicate the slowdown for whatever reason, I'd like to get your contact info. Email me at [email protected]

Your suggestions for handling sequences are very good. I'll make sure that we think about those, but I can never say what features are accepted until just before release.

Thanks,

Chrusion
04-09-2007, 08:24 AM
John,

Just to clarify, it's the speed at which these info files are being created that's the known bug, correct?

I gave up after 3 hours at around filename_02000 or so... the speed of info files being created had decreased to one every 3 seconds.

As you'll see in the Gen Support Forum, I'm questioning the core handling of icons/thumbs by having SE do all the work scanning the entire folder and generate separate "info" files on a 1:1 basis before allowing you to do anything. No other app that I know off does this. In most apps, file icons/thumbs are handled directly by the OS. Let the OS use it's OWN icons and/or thumbs in file view panels, etc. Of course this totally negates the internal LOOK of SE, so I guess there has to be some better way to either disable thumbnail caching or speed it up 10,000% and NOT HOG disk sectors with an equal number of 1KB files. How about a single thumbnail.db file, instead?

Thanks for listening! Appreciate it very much.
.

cholo
04-09-2007, 01:24 PM
I have an idea. How about having SpeedEDIT recognize folders whose names start with "SEQ" as image sequences automatically and instead of allowing access inside the folder it just creates a thumbnail for the folder like any other video clip? Inside you could have sequences in any file format, even mixed.

Chrusion
04-11-2007, 12:28 PM
Excellent idea! And for thumbs within the clip itself for viewing on the timeline, just grab the image filenames at the required interval and make INTERNAL thumbs in RAM. No need to create/store an image to disc on a 1:1 basis. I'm sure this is what Vegas does. When a clip is expanded/zoomed into, just grab new images from the folder at a different interval, resample to thumbs, and stick them in memory. Wow! SUPER FAST!

Yes, this is the sort of RETHINK we need Newtek to not just consider, but DO!

.

Lightwolf
04-11-2007, 01:41 PM
How about SE just recognizing sequences by their frame numbers and displaying them as one clip, just like many compositors do (i.e. Fu5)?

Basically, treat them as a single video file.

Cheers,
Mike

cholo
04-11-2007, 03:41 PM
Lightwolf:

Ah... But what if you want to have a folder with numbered images and not interpret it as a sequence. I do this all the time for subtitling. with the SEQ folder name idea you can have it work any way you want. In fact, you could name it SEQXX, XX being the frame rate you want the clip to default to. For example... SEQ23976Spaceship, SEQ2997Backdrop, etc... Also, this way you don't have to fumble with interpretation dialog boxes, but just get down to editing. :) Aren't we supposed to be the world's fastest editor community?

Lightwolf
04-11-2007, 03:47 PM
Lightwolf:

Ah... But what if you want to have a folder with numbered images and not interpret it as a sequence.
Easy, make the darn thing user defineable on a sequence by sequence basis, right click, break concat (or whatever), done. After all, we do have a folder of hidden metadata that may as well store info like that.
And it would retain us the freedom of naming folders the way we want to.

Cheers,
Mike

SBowie
04-11-2007, 04:23 PM
I'm with Lightwolf on this one, but given the way SE works it might be necessary to have it automatically popup up a "Sequence or Stills?" requester on first access of the folder when a numeric sequence is initially detected.

nthused
04-11-2007, 05:52 PM
This is a BIG issue where I work. We write out PNG sequences for our animations as file size is an issue when dealing with the number of projects with animations that we do versus harddrive space available.

Please address this oversight in SpeedEDIT. Honestly everyone in the office was very excited about the product - until I brought up what they have to do to bring our sequences into the editor. It was a big "turn-off" for the folks already using Premiere.

Other than this one thing...I REALLY like working in SpeedEDIT.

cholo
04-11-2007, 06:03 PM
Lightwolf:

Sounds good enough for me. Now we need to figure out what happens when the metadata folder gets messed up or erased. The only challenge is having speededit not losing track of this information on a project basis so projects don't become corrupt when metadata folders get wasted. The last thing you want is speededit crashing when you try to open a project that had media dependant on metadata folders that don't exist anymore. ;)

wvp
04-11-2007, 07:48 PM
those that are going to NAB should make a point of talking to John and anyone else you can find about this - the more they hear about it the higher priority it will become.

radams
04-12-2007, 09:12 AM
Easy, make the darn thing user defineable on a sequence by sequence basis, right click, break concat (or whatever), done. After all, we do have a folder of hidden metadata that may as well store info like that.
And it would retain us the freedom of naming folders the way we want to.

Cheers,
Mike


I totally agree with you Mike !!!!

We also need to have several file options supported...as in jpeg, jpeg2000, dpx, png, hdr, openEXR, etc...etc...

Cheers,

Lightwolf
04-12-2007, 09:22 AM
Hiya'll...
Plenty to answer, please excuse me for not quoting:

@SBowie: What if there are multiple sequences in a folder? Be default they should be displayed as a clip (no need for thumbnails and probably covers 90% of the use cases) and broken up by user intervention.

@cholo: Nothing would happen if the meta data is lost afaik. The settings for clips used in a project are stored within the project. The worst thing that could happen is that a sequence you had broken into files up will display as a single sequence next time you navigate to the folder.

@radams: Would be nice to have, however, some would need extra conversion input by the user (I'd love to be able to import EXRs, set the HDR->LDR transfer settings and then have SE render that out in the BG to something that can be played back in RT - re-rendering if I change the settings of course).

Cheers,
Mike

Chrusion
04-12-2007, 11:01 AM
I strongly vote for NO metadata/thumbs/etc in hidden folders for image sequences! If metadata is absolutely required, then make it a single file made within the same folder as the seq.

How much more Efficient would metadata be if contained in a single file? Such would result in a HUGE savings of harddrive sectors and a big decrease in file access time, since only one file would need to be opened rather than hundreds or thousands.
.

Rich Deustachio
04-12-2007, 11:20 AM
How about if you want it to load as a sequence you just drag in the first image and if you don't want it to load as a sequence you hold ctrl or alt or some other similar key or key combo while dragging to the timeline.

Chrusion
04-12-2007, 11:21 AM
[5 minute rule]

I think we all realize that a 1KB file takes up 4 to 8 KB of hard drive space if using default allocation units when formatting hd's over 20 gigs. A folder full of 5,000 such files (5 megs of metadata) sucks up 15 to 35 MEGS of hd space. This is unnecessary.
.

Lightwolf
04-12-2007, 11:34 AM
I strongly vote for NO metadata/thumbs/etc in hidden folders for image sequences! If metadata is absolutely required, then make it a single file made within the same folder as the seq.

How much more Efficient would metadata be if contained in a single file? Such would result in a HUGE savings of harddrive sectors and a big decrease in file access time, since only one file would need to be opened rather than hundreds or thousands.
.
Not quite. Afaik NTFS stores tiny files within the directory structure, not as a separate block.
Also, we're talking about one byte (consolidate on/off) of information per sequence (which can be thousands of frames).

Cheers,
Mike

Lightwolf
04-12-2007, 11:36 AM
How about if you want it to load as a sequence you just drag in the first image and if you don't want it to load as a sequence you hold ctrl or alt or some other similar key or key combo while dragging to the timeline.
Actually, I like that idea.
However is there would be a consolidate option for sequences, the workflow might be a bit different:
Scrub to the frame you want in the picon view (which afaik is not possible now), press a key combo and drag into the project to just get that frame. The beauty is this could work with videos as well.

Cheers,
Mike

Lightwolf
04-12-2007, 11:43 AM
5 Minute rule as well. here.... *grrrrr*
Regarding small files:

"Location of small files — with the NTFS file system, the entire file is contained within the MFT record. The maximum file size that fits in the MFT record depends upon the cluster size and the number of attributes for the file."
From: http://www.microsoft.com/technet/archive/ntwrkstn/reskit/file_sys.mspx?mfr=true

Cheers,
Mike

SBowie
04-13-2007, 05:40 AM
Hiya'll..
@SBowie: What if there are multiple sequences in a folder? Be default they should be displayed as a clip (no need for thumbnails and probably covers 90% of the use cases) and broken up by user intervention.
Video clips normally show a thumbnail anim, and there are some convenient features in the Filebin in this connection. I'd hate to give those up, so this is a problem. As the thumbnails are created on first entry of a folder, you'd either have to automatically pop up something that asked "Clip, Stills, or None?" re: thumbs on sequence recognition, or permit/require the user to do the same thing. This could work for multiple sequences in a folder too.

I can't say I care for it either, but there only seem to be one alternative ... all sequentially numbered could be treated as clips, period (which would be just fine with me. I never use sequentially named stills anyway.)

Lightwolf
04-13-2007, 05:45 AM
Video clips normally show a thumbnail anim, and there are some convenient features in the Filebin in this connection. I'd hate to give those up, so this is a problem.
Sorry, I probably misphrased it. Af course you should get those for sequences as well. However, you'd still only see the first frame of a sequence once you enter the directory, with the other thumbs generated in the background. And there should be some indication of the fact that you're looking at a sequence of frames stacked up


As the thumbnails are created on first entry of a folder, you'd either have to automatically pop up something that asked "Clip, Stills, or None?" re: thumbs on sequence recognition, or permit/require the user to do the same thing. This could work for multiple sequences in a folder too.

Why do you need either? I don't see any reason for user intervention except to break up a sequence into individual stills - once per sequence. Wouldn't be that bad since this seems to be required only in 5% of cases or so (guessing here).


I can't say I care for it either, but there only seem to be one alternative ... all sequentially numbered could be treated as clips, period (which would be just fine with me. I never use sequentially named stills anyway.)
Yup, with the option to break them up.

Cheers,
Mike

SBowie
04-13-2007, 07:09 AM
And there should be some indication of the fact that you're looking at a sequence of frames stacked upGood idea...



Why do you need either? I don't see any reason for user intervention except to break up a sequence into individual stills - once per sequence.
Only needed if someone insists that sueqntially numbered stills must be accomodated ... I'd sooner they were not - treat all such as clips in the filebin, and maybe instead provide a toolshed option to 'convert sequence to stills of x length each" for those who really must go that route.

Lightwolf
04-13-2007, 07:12 AM
Only needed if someone insists that sueqntially numbered stills must be accomodated ... I'd sooner they were not - treat all such as clips in the filebin, and maybe instead provide a toolshed option to 'convert sequence to stills of x length each" for those who really must go that route.
I think we are on the same boat here, except that I can imagine such an option in the right click menu in the file bin. Also, I assume the toolshed only works for projects, not for external assets in the file bin/browser.

Cheers,
Mike

SBowie
04-13-2007, 04:14 PM
I think we are on the same boat here, except that I can imagine such an option in the right click menu in the file bin.I can see that something in that vein could work - it would have to force a rebuild of some or all of the info folder content, but seems possible.

Also, I assume the toolshed only works for projects, not for external assets in the file bin/browser.
Yep, but I'm thinking in minimalist terms, and if as a 'simple' solution all sequences were taken as a clip, a toolshed function for the rare times I wanted to treat a sequence as stills would satisfy any practical editing need I have ... more so, really, because (speaking just for me) I simply never need to do that. Anytime I use stills, they have unique names, not incrementally numbered ones (hence no issue.)

Lightwolf
04-13-2007, 04:23 PM
I can see that something in that vein could work - it would have to force a rebuild of some or all of the info folder content, but seems possible.
Probably. Then again, the metadata seems to be smart and only rebuilds stuff for changed files.


Yep, but I'm thinking in minimalist terms, and if as a 'simple' solution all sequences were taken as a clip, a toolshed function for the rare times I wanted to treat a sequence as stills would satisfy any practical editing need I have ...
I see what you mean. I suppose the "use case" for this issue would be subsequently names frames (such as subtitles) that are placed one at a time in the timeline, and in that case having them as single images in the folder would be a lot more useful.

Next: in/out/money shot options for all icons please ;) (Yes, I'm thinking of a split set-up, money shot on top, two smaller in/out icons at the bottom). And an option to scale them up in the file bin like the storyboard allows you to do...

Cheers,
Mike

Rich Deustachio
04-13-2007, 06:03 PM
I use sequentially numbered stills all the time for picture montages. I think many other VT users do as well. There has to be a way to tell VT what the intended use is for the stills so it can handle it accordingly.

Ahmed
04-14-2007, 07:12 AM
Look no further.. Mirage image loader is done right, and well conceived.. I like that..

Check it here
http://www.balfaqih.com//Temp_Images/Mirage_Squence_Loader.jpg

NewTek should implement this for version 1.x

Just my 2 cent.

SBowie
04-14-2007, 07:33 AM
Hmmmm ... without commenting on the pros and cons of Mirage's import panel, it's a very, very different thing than the existing VT/SE workflow, Ahmed. Hard to see how the two concepts could be married.

Ahmed
04-15-2007, 07:45 PM
Hmmmm ... without commenting on the pros and cons of Mirage's import panel, it's a very, very different thing than the existing VT/SE workflow, Ahmed. Hard to see how the two concepts could be married.

Steve,
I would have thought that the mere mention of Mirage/Aura, you would just agree regardless. After all, you and Mirage are a couple. :)


Anyway, I know that the work flow is not the same, but I like when i double click on an image seq be asked what do I want to do,

Should I load that frame only?
Should I load the entire seq?
Should I mark In or Out?

That to me is well thought of compared to what we have now...

Just my two cent (or 4 cent now)

SBowie
04-15-2007, 08:45 PM
Aah, I see - I think. (Maybe) you're suggesting something could pop up when the image sequence is added to the timeline/sb? But if so, this still leaves the issue of thumbnail generation for what can be thousands of files.

Chrusion
04-16-2007, 07:24 AM
The frozen molasses thumbnail/metadata file generation thing is a symptom of a bug affecting something else that has been fixed in the new upgrade, according to John P. in a private email.

Though it's fixed ( I hope, haven't d/l'd the upgrade yet), thousands of teeny files all over my HD just erks the H out of me. Microsoft, Adobe, and Sony don't do this, why does NT have to? Because that's the way VT Edit started out 10 years ago? Hey! it's 2007, NT HAS to rethink this metadata stuff. I've said it before, thumbnails can be generated on the fly as needed and kept in RAM, just like the way Vegas does. If thumbnails need to be stored, keep them in a single "database" file per folder like MS and Adobe do.

Regarding small files, Thanks for the MFT info. However, the MFT will out grow its 12.5% of reserved disk space (MFT Zone) and become fragmented into newly acquired HD space by the generation of tens of thousands of 1K thumb/metadata files for those of us using SE for animation. No matter how I look at it, it's still a nasty and unnecessary thing to molest my HD this way when far more efficient methods are readily available today.
.

Lightwolf
04-16-2007, 07:27 AM
No matter how I look at it, it's still a nasty and unnecessary thing to molest my HD this way when far more efficient methods are readily available today.
.
No doubt. Especially since there are a lot more ways to improve image sequence handling than the ones we've discussed here:
* a single metadata file per sequence
* automatic linking of single frame sequences to pre-rendered video files (i.e. if a frame in the sequence changes, the video gets re-rendered using that new frame) - this would make importing sequences from a network feasible as well.

Cheers,
Mike