PDA

View Full Version : The New Tek AVI file curse.



SteveRother
07-31-2007, 06:44 PM
Seem that it is entirely possible to shut down the Tricaster while the archive button is still going. (Not good programing) If you do shut down the program the remaining Temp .avi file is encoded with the NewTek25 proprietary codec.

Final Cut Pro, Abobe Premier Pro, Sony Vegas and several other I have tried can not open this file to add the missing ending to the file. They all report that it is not an.avi file. Now I have 42 gigs of .avi file with invaluable data on it but I can not access it.

The request is two fold:
1. Please give us a patch for the program so that it can not shut down until the recording has been stopped.

2. Help us to recover some of these files with a simple program or set of instructions to regain some of our irreplaceable productions that have been lost to this problem.

Thanks

SteveRother

UnCommonGrafx
07-31-2007, 07:18 PM
If the TriCaster can still read the file, head over to the download area to get the Codec Pack for those other machines: this may help you get over the hump.

G'Luck!

SteveRother
07-31-2007, 08:12 PM
If the TriCaster can still read the file, head over to the download area to get the Codec Pack for those other machines: this may help you get over the hump.

G'Luck!

Thanks, NewTek25 is installed on all of the machines.
The problem is two fold:

1. the file wan never ended. (broken)
2. the file is in a proprietary codec "NewTek25"

If there were only one of these there would be no problem. Both of them together seem to be fatal.

Thanks
Steve

D3Cast
08-01-2007, 06:27 AM
The problem is two fold:

1. the file wan never ended. (broken)
Paul Lara agreed, a while back, that a software enhancement to automatically 'finish' a currently recording AVI on the TC when you exited TriCaster normally would be a dandy idea. Don't know where this sits...

-steve

SBowie
08-01-2007, 07:30 AM
Stop recording on Exit is a good idea. It leaves unsolved the problem of how to recover from an unexpected crash ('stuff happens') ...

An included tool that could at least make a valiant attempt to recover unnaturally truncated capture files would still be a very nice thing to provide. 'Home team advantage' going to the guys who designed the codec would seem to make success more likely. I smell a feature request.

D3Cast
08-01-2007, 07:36 AM
'Home team advantage' going to the guys who designed the codec would seem to make success more likely.:agree:

I've learned way more than I've wanted to about AVI files over the past few years, trying to fix some of my most valuable broken TC AVIs.

-s

SteveRother
08-02-2007, 10:26 AM
I have had a partial success that I posted here. (http://www.newtek.com/forums/showthread.php?t=71962)

Still not there, a little help would go a long way here. New Tek, you out there?

I guess this could also be some Karmic thing that only happens to guys named Steve...

SteveRother

Paul Lara
08-05-2007, 10:52 AM
Yes, NewTek is here. Let me talk to the TriCaster coders and see if there is some method they know of recovering after an improper end of file.

SteveRother
08-07-2007, 02:24 PM
Thanks for your help I am sure there are many 40 to 60 gig files out there that people just wont give up on.

Thanks

SteveRother

Paul Lara
08-08-2007, 09:25 PM
Well, don't thank me just yet. There seems to be no easy fix or utility that can actually repair these files. There are utilities that would let you play these files, but not repair and re-validate them. :(

D3Cast
08-09-2007, 07:58 AM
Yeah, I mean, I definitely give the NewTek folk the benefit of the doubt that if fixing these files were 'quick and easy', it'd be done by now. So that leaves 'time-consuming and hard', which is still within NewTek's ability. :)

Even something that patched these files enough so that we could load them into the TriCaster 'Edit Media' tab and then write out standard DV-AVI files with 'Prepare for VCR' would be cause for a national holiday for some of us...

Trying to fall somewhere between 'broken record' and 'squeaky wheel' here... thanks for all the assistance, Paul.

-steve

Cineman
08-09-2007, 03:52 PM
Well Paul,

In the interim, and in the spirit of "A stitch in time saves bunches." or better, "An ounce of prevention is worth a pound of cure." why not take steps to at least try and prevent the: "Whoops, I forgot to stop recording, before I shut down." phenomenon?

When I click the up right "X" now, the first print still says: "What would you like to do?" But on that same line, centered under "Shut Down TriCaster", it is followed by: "Do you need to STOP 'Record Output' First?" Then spaces and "If so: Click 'Cancel' and stop it."

With Ctrl+Alt+E, are the same words, except that the new first sentence is centered under: "Exit TriCaster".

I don't think that this is beyond the things that us end users might do as this closely parallels what we do for Limiting User Access. It is even the same two files that need to be, I guess, replaced. (I have modified "SAFEInialization" to match these same). The replacement procedure is almost identical to Limiting User Access.

Would you like to make these available for download at NewTek?

Nes Gurley

Paul Lara
08-09-2007, 06:50 PM
Would you like to make these available for download at NewTek?

Thank you Nes. Engineers are working on a software update that will remedy this and a few other things as well.

Cineman
08-10-2007, 08:17 AM
Well, you had only talked of something to repair the corrupted recordings up to now. But, yes, an automatic stop of "Record Output" on TriCaster Shut Down, would be much better.

Alan69
08-21-2007, 11:38 AM
I have had a partial success that I posted here. (http://www.newtek.com/forums/showthread.php?t=71962)

Still not there, a little help would go a long way here.


I'm working with totally different AVI files, 8 FPS Xvid codec Camstudio files, but the basic theory should be the same. If they go over 2GB, Camstudio won't touch them again at the end, and finish the file, or if I've crowded the disc it stops with a smaller unfinished file. Finally had one I really wanted to keep, so I got off my *** and figured it out. Should have done it 2 or 3 years ago really.


Fix it manually. Header size or some other aspect may be a little different, but the general idea should be the same. AVI has a header at the start with framerate, size, aspect, etc etc, that isn't written until the file is done. At the end of the file is the index, with keyframe info etc. With a stopped file, you don't have the index written at the end, and there is no header info since it wasn't retouched after knowing the file size etc.

First, get some programs. Hexapad is key, not the perfect editor but free, and will let you insert info into a file to fix the header.
http://www.softpedia.com/get/Programming/File-Editors/Hexapad.shtml

Then
http://www.videohelp.com/tools/sections/video-editors-wmv-avi
From here, get VirtualdubMod or VirtualdubMpeg2. Vdub can reindex the file, fixing the rest of it. This will take a long time on such a long file, takes a good time just with the 600 mb file I was working on. Could not even handle such a large file, but somewhat open development and a lot of people use it, so likely it will.


With the same settings, make two short files for testing. One, a minute or two of video, and close the file normally. The second, a minute or two of video then close the program to make a stopped file.


Now look at the stopped file. Vdub under tools menu has it's own hex editor, load it in there. (Has to not already be open in Vdub main program to load into the editor.) First part of the file will likely be blank, for me it was 0 to 7ff hex. 800 hex was where the frames started. Your blank header may not be all 00 or similar though, some programs may put a header, but with just blank file info. Either way you need to determine where the start of actual file frame data is, I'm not sure if the header is 2K (0-7ff hex) always or variable size.

Now load your good AVI, and under edit, extract 0-800 (doesn't save the last byte) to a file, avistart.avi. That's your header. In hexapad, open the short broken file. Put your cursor on byte 0. Scroll down to 7ff, note it is the end of the first column of that row. Hold shift and click the byte at 7ff, this will select the header. Then do Edit, delete. Make sure your cursor is now at the start of the file, and Edit, insert from file, and pick the avistart.avi file. This should insert your AVI header into the broken file. Then save the file with a new name..

This still won't play, there's no index on the broken file. But it won't say 'not an avi file' since it has a header now.

Open Vdub. Under Video menu, select 'direct stream copy', so you don't forget later. This makes the copy fast, no advance editing, just copy the file stream data. Now do File, open. Click the check box for 'ask for extended options' in the lower left before selecting the file with new header. An extra dialog will open, click the upper left 're-derive keyframes' option. This should take a very long time on a 60 GB file. Once it's done, you should be able to seek like normal. Then File, save as AVI and give a new file name and you'll make a new file with right info. Direct stream copy means writing the file should go pretty fast, you're not doing heavy processing.


Again, do the short test files to get the idea and make sure it works first, and make sure Vdub can handle your video codec etc. Also work on a backup of your 60 GB file, so if you mess something up you can start over and try different programs if needed. Basically once you've done it a time or two it's easy, Vdub does most of the work of actually fixing it. You're really only doing a little work, cutting out the old and splicing in a new header, then letting Vdub rebuild the keyframe index. I'd really copy out an avistart.avi header from a longer file for general patching use though. Vdub doesn't seem to be bothered by a short file length in the header, but can't hurt, and some other program might not like it if you need something else. And of course you'll need some elbow room on the drive, since you'll be making 2 or 3 extra files in the process, I wouldn't delete anything until you're sure the end result works well..

Now of course I'm wishing I hadn't deleted so many of my 2GB failed files. Nothing too important, but now it's easy to get them back to working.


Hope this helps. For certain it's working for me on smaller files, only real questions would be the larger file sizes and if Vdub can handle working with the Newtek codec.

Alan

D3Cast
08-21-2007, 11:50 AM
...only real questions would be the larger file sizes and if Vdub can handle working with the Newtek codec.AlanThat's the rub -- I have done approximately the same steps as above, with different but similar hex editors and AVI fixers, and the NT25 codec hasn't been recognized. Still, worth another swing at it at some point -- thanks for all the detailed documentation!

-steve

Alan69
08-21-2007, 03:09 PM
That's the rub -- I have done approximately the same steps as above, with different but similar hex editors and AVI fixers, and the NT25 codec hasn't been recognized. Still, worth another swing at it at some point -- thanks for all the detailed documentation!

-steve


Yes, will need the NT25 codec installed and working of course, if Windows Media Player can play a good file then Vdub should be able to cope with it. But, had thought about something else last night, and forgot it in my post. If it's a 2 pass codec, then that file can be in an intermediate form from the first pass and won't necessarily parse with trying to play with the codec, since it needs another encode pass. Might still be able to do similar with an altered method, even with Vdub, but I don't use 2 pass much so haven't gone through that way often enough to get detailed about it.

So there are some other possible gotchas, but definitely worth another try in Vdub, at least with small test files to see if it works in a general sense. Most programs will still choke on an AVI with no index and with a forged header and all of the discrepancies, or screw up something trying to write them back out from that wrong info in the header. Vdub is one of the few things that handles a lot of problems at the same time correctly, even though it's free it's better than most shareware and commercial programs.

Alan

animlab
03-08-2008, 09:41 AM
Want to get your cached video back after a TriCaster crash or forgot to turn off the record button before shut down TriCaster?
FixNTavi can help to scan and fix the unclosed cached avi files of TriCaster.
Drop me an email for details.

Chen Pan
AnimLab
[email protected]

-EsHrA-
03-08-2008, 01:46 PM
huh? hmm. cool job.

so an outsider fixed it??? ..



mlon

animlab
03-09-2008, 08:43 AM
:D It is magic...

Acturally it was not too hard but need time to study on both avi file sturcture and NewTek NT25/SHQ2 files. I have many customers here worried about they might lost their recorded live video with Tricaster. I believed most TriCaster users have experiences of TriCaster crashes or accidentally shut down when they are doing live recording.

NewTek R&D guys need to focus at major features of VT/TriCaster. I think they don't have time in solving the minor features. If possible, I will do it for my local customers as an after service (free or fee service). Some of VT[2] users might remembered that I wrote a VTR control program for VTEdit print to tape before NewTek did it. I have open it free for about 20 international users. I have finish about 10 different addon projects form VT[2] to VT[4] locally at Taiwan. If I think any of my projects might be useful for the VT/TriCaster communities, I will post them out.

UnCommonGrafx
03-09-2008, 09:32 AM
You really ought to, Chen: this crowd is rather low on coders of goodness.

:D It is magic...

...
I have open it free for about 20 international users. I have finish about 10 different addon projects form VT[2] to VT[4] locally at Taiwan. If I think any of my projects might be useful for the VT/TriCaster communities, I will post them out.

-EsHrA-
03-09-2008, 09:36 AM
:) crashes or shutdowns doesnt sound like minor features/bugs to me!

well done, animlab!


mlon

ted
03-09-2008, 10:33 PM
eshra, you seem to want to blame NewTek for these issues in more then one post.
All the posts on all the threads I've read have been due to power outages or users accidently shutting down the software while recording. "User error".
I don't see how this is NewTeks fault.

A warning box would be a great addition after seeing how many peole have done this.
So yes, it's nice "an outsider" came to the rescue of the users.

-EsHrA-
03-10-2008, 04:02 AM
ted - correct, that was my observation.
then read the first post of this thread.
how is (not good programming) not nt's fault?..
the next thing i read is that there isnt a simple avi fix solution and again the next that someone outside fixed it.

im glad though that now users can get help with their gigs of avis.



mlon

Jim_C
03-10-2008, 08:53 AM
I say if a user is so clueless as to what his system is doing that he powers it down while it is in the middle of recording something he deserves whatever consequences he gets.

Do people REALLY do this that often with the Tricaster? Just be recording along and BAM! shut down the entire system without even paying attention?


I realize much of this thread covers crashes and lock ups, but are we really trying to blame Newtek because someone shut down a computer while it was recording and that action corrupted a file?


Shame on you Newtek...! How dare you not give users more common sense!

PIZAZZ
03-10-2008, 09:06 AM
I say if a user is so clueless as to what his system is doing that he powers it down while it is in the middle of recording something he deserves whatever consequences he gets.

Do people REALLY do this that often with the Tricaster? Just be recording along and BAM! shut down the entire system without even paying attention?


I realize much of this thread covers crashes and lock ups, but are we really trying to blame Newtek because someone shut down a computer while it was recording and that action corrupted a file?


Shame on you Newtek...! How dare you not give users more common sense!

Easy Jim... :)


Call me when you have a moment. 409-860-9283

UnCommonGrafx
03-10-2008, 09:20 AM
I dunno...

This is something that through use could just be labelled a design issue/flaw, not a bug. When many users hit the same problem, as NewTek has said, Engineering gets called in to make that problem less easy to hit.

Yeah, the user beware but there's lots the engineering side can do to alleviate such issues.