PDA

View Full Version : Major bug with Surface Nodes -- "IFF file chunk"



Captain Obvious
12-03-2008, 04:05 AM
1. Add an object, any object, to a scene. The number of surfaces does not matter.

2. Open the Node Editor for a surface and add a Mixer node. Now press a, ctrl-c, ctrl-v over and over again to select all the nodes, copy & paste them. Do that until you have 256 Mixer nodes.

3. Save your object. Works fine.

4. Bring the total number of Mixer nodes up to 285, and save again. Still working.

5. Add ONE MORE Mixer, to 286, and try saving. Now it displays an error message:

IFF file writing failed.
An IFF file chunk exceeded 65535 bytes.
If using Node Editor, shorter the names of your nodes.
and then another error message, saying object saving failed. The object file is now corrupted!


The number of nodes required for this error varies from node to node. In my tests, it takes 286 Mixer nodes to produce the error. Using the IFW2 nodes, it takes a mere 33. If you connect them to eachother, the number required to produce the error seems to go down. The length of the node names has zero impact.

Node Editors in other parts of Lightwave do not seem to have this problem. I was able to add 512 IFW2 nodes to a Displacement node tree and the LWS saved fine anyway.

The error occurs in both 9.3.1 AND 9.5. I've only tried it on 64-bit Windows, because it's the only thing I have access to.

This is a really, REALLY bad bug. Not only does it make it difficult to construct complex node trees without using Pomfried's Get/Store Material node to break up trees between different surfaces, but it CORRUPTS YOUR FILES! Can someone please try to verify this?

Sensei
12-03-2008, 04:50 AM
Node Editors in other parts of Lightwave do not seem to have this problem. I was able to add 512 IFW2 nodes to a Displacement node tree and the LWS saved fine anyway.

LWS is text file, not binary IFF, so it can't have this bug..
IFF sub-chunk length is limited to 64 kb since ever. That was designed in 1985 by Electronics Arts in the first graphics software Deluxe Paint, and later implemented by Commodore in AmigaOS v2.0 in shared library iffparse.library.

Lightwolf
12-03-2008, 06:39 AM
LWS is text file, not binary IFF, so it can't have this bug..
LWO is binary and IFF though, and Cptn. did specify surfaces.

But you're right, the problem is that it's a 16-bit sub-chunk and not a proper 32-bit chunk - which might not have been possible to use in that context.

Cheers,
Mike

Captain Obvious
12-03-2008, 10:09 AM
After reading up on the LWO format, I guess that kind of makes sense, in a certain way. But the PNTS chunk can obviously be bigger than 64kB, so why impose a limit on the chunk containing the nodes? Or is it just that sub-chunks are limited?

Even if there is a limit and it's difficult to get rid of without breaking the LWO format, the fact still remains that this bug corrupts your objects! If you load it in Modeler and keep the partially loaded, you can restore the mesh, but the surfacing gets nuked.

A bug that can corrupt your objects is a pretty bad bug, if you ask me.

Lightwolf
12-03-2008, 01:06 PM
Or is it just that sub-chunks are limited?

Yes.

But I agree, if it corrupts a mesh it's bad. Did you report it?

Cheers,
Mike

Sensei
12-03-2008, 01:11 PM
After reading up on the LWO format, I guess that kind of makes sense, in a certain way. But the PNTS chunk can obviously be bigger than 64kB, so why impose a limit on the chunk containing the nodes? Or is it just that sub-chunks are limited?

PNTS is high level chuck, which are using unsigned long 32 bit integer for size. sub-chunks which are used in surface attributes are 16 bit unsigned integer.



A bug that can corrupt your objects is a pretty bad bug, if you ask me.

You should report it to Fogbugz https://secure.newtek.com/FogBugz/default.asp
Obviously objects made with fixed version won't be fully loadable in older/currently available LW versions anymore..

Lightwolf
12-03-2008, 01:23 PM
Obviously objects made with fixed version won't be fully loadable in older/currently available LW versions anymore..
Just use a different chunk name. I suppose it needs to be placed differently within the file as well and that could require a few changes to LW itself.
Since SURF only allows for sub-chunks, there'd need to be another Chunk to hold the nodal data outside of SURF.

Cheers,
Mike

Captain Obvious
12-03-2008, 01:28 PM
But I agree, if it corrupts a mesh it's bad. Did you report it?
No, not yet. I didn't know about the Fogbugs thing until Sensei linked to it. I'll sort that out, though.




Obviously objects made with fixed version won't be fully loadable in older/currently available LW versions anymore..
That's fine, though. If the old version is broken, breaking backwards compatibility to fix it seems perfectly acceptable.

Pavlov
04-29-2009, 05:08 PM
I get same error if i apply a Fryrender shader which contains a proprietary noise function.
Same error, and model is corrupted.
Captain, did you report that ?

Paolo

Iaian7
01-12-2015, 10:40 PM
I'm running into this issue with LW 11.6.3 (downloaded yesterday to make sure I had the latest)...

"IFF file writing failed.

An IFF file chunk of 65580 bytes exceeds 65535 bytes. If using Node Editor, shorten the names of your nodes."

Which is pretty terrible, seeing as I'm only about 75% done with this surface, and I've renamed almost none of the nodes...they're all short. When I tried to save the object the first time, Layout crashed and corrupted the model. Now I'm trying to save out the individual parts, but I'm unable to save out a .srf file.

Any news on if Newtek will be fixing this? Pretty catastrophic...

spherical
01-13-2015, 12:29 AM
Ok, I'll ask.. have you read and understand the thread, as posted previously? Yes, I suppose that some kind of ground-up rewrite using an entirely different method is possible (anything is possible if you strive hard enough) but it is a known limitation. Of course, it would be great if models didn't become corrupted when the limitation is exceeded, but it seems that control of that is out of scope when a basic (lower level) threshold has been exceeded.

MSherak
01-13-2015, 02:45 AM
Make sure you turn on the Save Options so Modeler and Layout create backups of your models and scenes so you don't lose everything if this happens. Has saved me tons of time when something goes wrong beyond your control.

-M

Sensei
01-13-2015, 04:24 AM
I'm running into this issue with LW 11.6.3 (downloaded yesterday to make sure I had the latest)...

"IFF file writing failed.

An IFF file chunk of 65580 bytes exceeds 65535 bytes. If using Node Editor, shorten the names of your nodes."

Which is pretty terrible, seeing as I'm only about 75% done with this surface, and I've renamed almost none of the nodes...they're all short. When I tried to save the object the first time, Layout crashed and corrupted the model. Now I'm trying to save out the individual parts, but I'm unable to save out a .srf file.

Any news on if Newtek will be fixing this? Pretty catastrophic...

Get TrueArt's Global Materials http://globalmaterials.trueart.eu
and copy and paste your node tree to Global Node Editor.

It saves surfacing data to LWS scene file, instead of LWO object file! So you don't have to duplicate object to have different surfacing data. It is also automatically solution to IFF Sub-Chunk limitation and error.

It's not fixable issue. IFF Sub-Chunk has 2 bytes length since ever (AmigaOS times). 2^16=65536 max length per sub chunk.
"Fixing it" would require making non-compatible changes to IFF LWO structure, and LWO written in future wouldn't be readable by older LW versions.

People that never experienced this bug can see it on my video tutorial
https://www.youtube.com/watch?v=gzwtr9MNcFc

pinkmouse
01-13-2015, 06:12 AM
..It's not fixable issue. IFF Sub-Chunk has 2 bytes length since ever (AmigaOS times). 2^16=65536 max length per sub chunk.
"Fixing it" would require making non-compatible changes to IFF LWO structure, and LWO written in future wouldn't be readable by older LW versions.

And that's a problem because?

Sensei
01-13-2015, 06:44 AM
And that's a problem because?

Is this serious question?!

Imagine we have plugin that works only in v11 and not working in v12. Or some function is broken in LW and works fine in previous release.
We will load LWO in v11, where changes are not made, and save LWO after using plugin/tool.
Nodes compatible with v12 will be gone from file.

pinkmouse
01-13-2015, 06:55 AM
Yes. At some point, LW3DG needs to abandon all this limiting backwards compatibility stuff if it's going to develop sufficiently for future use, and frankly, the sooner that happens the better.

Sensei
01-13-2015, 07:03 AM
Yes. At some point, LW3DG needs to abandon all this limiting backwards compatibility stuff if it's going to develop sufficiently for future use, and frankly, the sooner that happens the better.

You don't want 3rd party plugins to work with any new LightWave version?
or you don't use, or didn't buy any 3rd party plugin?
Then demand on keeping old LW installed, and using it for older stuff, will be much higher.
And this way you will be losing data that are compatible only with the latest version of LW, while using older LW.

pinkmouse
01-13-2015, 07:24 AM
So you want LW to remain crippled with stupid limitations forever just so someone can use a ten year old plugin in a five year old package? Somehow, I don't see that as a particularly successful business model.

Stuff has to change if it is to stay valid, backwards compatibility is a good thing in moderation, but sometimes, like Apple did with OSX, you just need to start again from scratch.

Sensei
01-13-2015, 07:32 AM
Suppose so backward compatibility is broken in v12.
If it was done once, it might be done unlimited times.
Who will guarantee you that in v12.1 it won't be done again?
and v12.2?
and v12.3
and v12.5.. ?
etc. etc.

People would be forced to buy upgrades. From buying software and using it would move to buying license for a year, and using it for a year, and then software stops working..
Great business of model for company, not good for its customers.

It must be the case for max. Older version of max can't load/understand newer version of history/modifier.

pinkmouse
01-13-2015, 07:43 AM
Nobody would be forced to buy anything. If LW9.6 with Worley's plugins works fine for you, then keep using it. I'm still using Photoshop CS4 because it works for what I need. If I need new features, then I'll look at upgrading or changing. But if stuff needs changing to cope with new features, then it should be changed, crippling a new feature so it can be backwards compatible is just foolish.

Sensei
01-13-2015, 08:13 AM
Nobody would be forced to buy anything.

Only if you're working alone.
While working with other people, you will be forced.
They will be sending you objects and scenes you can't even load..


If LW9.6 with Worley's plugins works fine for you, then keep using it.

Such person is completely not interested in future of LW..


I'm still using Photoshop CS4 because it works for what I need.

Do you experienced PSD file that you can't load?

MSherak
01-13-2015, 10:51 AM
Nobody would be forced to buy anything. If LW9.6 with Worley's plugins works fine for you, then keep using it. I'm still using Photoshop CS4 because it works for what I need. If I need new features, then I'll look at upgrading or changing. But if stuff needs changing to cope with new features, then it should be changed, crippling a new feature so it can be backwards compatible is just foolish.

Basically the file format just needs to be expanded which means all older files will work and newer ones will only work with that version and above. I bet they are already looking into this..

-M

spherical
01-13-2015, 04:03 PM
Nobody would be forced to buy anything.

If the current system cannot be increased, so as to be backward compatible, many of us would be forced into buying many hundreds of dollars of plugins that had to be updated in order to accommodate the new system. Just because you're used to non-compatibility because of Apple (I have many Macs and really don't appreciate the short obsolescence that they pull on users all the time), doesn't mean that everyone else has to fall in line. I look upon backward compatibility as a Good Thing, if a company can pull it off. If moving forward can be done without sacrificing existing users, so much the better. Says a lot about the company. It all comes down to one's culture, I guess. Can't please all of the people all of the time. Some, never.

There are some ways available to have the cake and eat it, too. We already have differences in scenes in newer versions. But, we also have the ability to save in previous formats from a newer version. Not complete backward compatibility, which would be transparent, but still workable in some situations; with the obvious limitations.

pinkmouse
01-14-2015, 02:32 AM
I really don't get the plugin argument. Some plugins have always had problems with a new version of LW, they get updated or you have to use them in a legacy version. Always happened, always will...

What does this have to do with the option to save objects in an updated file format that offers new features and bug-fixes?

jwiede
01-14-2015, 08:25 PM
Basically the file format just needs to be expanded which means all older files will work and newer ones will only work with that version and above. I bet they are already looking into this...

Given how old this thread is, and how few related changes have occurred since then, you might not want to wager too much on that bet. ;D

Some number of older plugins have broken with every single version-level release, and have since at least LW9. Arguing "plugin compatibility" as a priority now is effectively closing the barn door after all the horses have run away. Further, as Pinkmouse points out, the compat. issue still doesn't explain (let alone justify) leaving a known, not-infrequently-triggered "user data corruption/loss"-type bug unaddressed/unresolved in the app for years.

Sensei
01-15-2015, 06:39 AM
Some number of older plugins have broken with every single version-level release, and have since at least LW9.

Because of senseless extending LWNodalAccess structure...
I told them to include size field in such structures, so code would immediately know which fields are valid and which not, and adopt.
This would prevent having to constantly recompile plugins using LWNodalAccess structure (like FPrime,Kray,TrueGroup).

Microsoft have size field in such structures since designing them in '80 years!