PDA

View Full Version : The most Gianormous image ever



Beamtracer
11-21-2003, 02:49 AM
I've got the biggest image that ever existed! It's around a gigabyte in file size (a tiff with LZW compression). If it was a bitmap it would be even bigger. I'm thinking of entering this in the Guinness Book of Records!

So what can I do with this monstrosity? It's a satellite image of the Pacific Ocean, and you can zoom right in on those tropical island paradises with mega detail.

But shock horror, Lightwave won't load it up. It's too damn big!

People say that Lightwave has no limits on the size of images that can be loaded as textures. I'm beginning to think otherwise.

I can load images like this into Photoshop. In fact I can load this huge one plus a couple of other 200MB images at the same time in Photoshop.

I'm finding that Lightwave doesn't load images (as texture maps) that are much over 100MB in size. Is this the cutoff for Lightwave? Does it really have an image limit?

Apologies for anyone who has read earlier posts in other sections that didn't get large numbers of responses.


My system:

Apple G5 (64-bit hardware, king of RAM!)
Twin IBM 970 2GHz PowerPC (G5) processors
2.5 Gigs of RAM
OS X operating system (10.2.7)
Lightwave 7.5c

Note: Lightwave 7.5c is still a 32-bit app. at the time I post this, even though the hardware is 64-bit. (Maybe future revisions will be 64-bit, allowing more RAM). I believe that Lightwave 7.5c on both Mac OS X and Windows is limited to 2gigs of RAM, although on Mac OS X the other apps and the OS can sit in other RAM up to 8 gigs.

takkun
11-21-2003, 03:41 AM
Where did you get the image from and what's it's dimensions? Just curious.

js33
11-21-2003, 04:04 AM
It's probably one of the NASA blue marble images. They have an image of the earth that is 21,000 x 10,500 or something like that.
Yes it is a hugh image. :D

So it won't load Beam. I have one that is 8,000 x 4000 but its only about a hundred megs I think.

Have you tried breaking it into 4 smaller images and mapping them onto 4 parts of a sphere.

Cheers,
JS

Lightwolf
11-21-2003, 04:28 AM
Hi Beam,
damn, you're on a Mac. I have a plugin for exactly that kind of work, but there is only a PC Version (check my sig).
I've used images that were gigabytes in size with this.
Cheers,
Mike

Beamtracer
11-21-2003, 04:41 AM
Hi JS... yeah, NASA images. One is 43200 x 21600.

Hi Lightwolf. I had looked through some of your earlier posts on this subject. I understand that your plug-in is for image compression, am I wrong?

While using compressed image formats might save download times, it makes no difference to Lightwave's RAM usage. A compressed JPEG uses the same RAM footprint as an uncompressed bitmap.

I figured that because I can load these images (and more) in other applications that it must be Lightwave that is imposing the RAM limit. Splitting the image up doesn't seem to help either, as LW seems to hit a total RAM limit between 100-200MB.

After searching through old forum threads I was surprised not to be able to find any reference to the maximum size image that Lightwave can use as a texture. There doesn't seem to be anything in the manual, either.

With gigabyte sized images floating around, nobody can say that 64-bit computing systems won't be needed.

Lightwolf
11-21-2003, 04:52 AM
Hi Beam,
you're wrong.
It does use compressed images as an input, but, when rendering, only loads the parts of an image that are actually visible, only in the resolution that is actually needed.
I created some terrain animations with satellite textures that were GBs in size, but only consumed around 200MB of memory per frame (Basically, think LoD for textures).
Cheers,
Mike - and, before you ask, a Mac version won't be available until I finished my re-write of the PC version, sorry.

omeone
11-21-2003, 05:35 AM
Hi Lightwolf, I had a glance at your Infinimap plugin before but never realised the functionality until that post, the "LOD for images" sounds great. So I am quite interested now :)

I have georeferenced tif's for my city, could you tell me / do you know if I need to get ER mapper to convert them to ECW?

js33
11-21-2003, 05:44 AM
Originally posted by Beamtracer
Hi JS... yeah, NASA images. One is 43200 x 21600.
With gigabyte sized images floating around, nobody can say that 64-bit computing systems won't be needed.

Yeah that's right I remember now. 43.2k x 21.6k. It's so big you have to get permission to download it. :D

I think maybe I did download that one but never used it for anything.

Yes you are correct that 64bit everything is or will be needed soon but on the other hand most people aren't using images anywhere near that size for regular work.

Cheers,
JS

Lightwolf
11-21-2003, 05:58 AM
Originally posted by omeone
I have georeferenced tif's for my city, could you tell me / do you know if I need to get ER mapper to convert them to ECW?
First of all, sorry for hijacking the thread...
You can get a free compressor from www.ermapper.com, but it only compresses a max of 500MB of input RGB. You'll need ERMapper for anything above that.
Cheers,
Mike

omeone
11-21-2003, 06:04 AM
'Yeah, sorry beam for the hi-jack,
I have been reading the site all morning, but what I really want to know is how to convert my tif's so that they will retain their georeferenced data, but not to worry, I'll find out from Er Mapper or Bentley.

Lamont
11-21-2003, 10:42 AM
Originally posted by Beamtracer
I've got the biggest image that ever existed! It's around a gigabyte in file size (a tiff with LZW compression). If it was a bitmap it would be even bigger. I'm thinking of entering this in the Guinness Book of Records! I'm sorry dude.. I have 2 deca-meter (every pixel is equal to 20cm) data of the entire United States. It's bigger than that. It's so big that it's been cut up into 4gb chunks and stuffed on an external firewire HD.

It was made on some Linux machine by our Geo guy.... Only he can open it... no one here has the machines to deal with it.

I also have 5dm data of San Clemente island. Primo stuff... it's about 4 1.8gb files.

js33
11-21-2003, 10:56 AM
Originally posted by Lamont
It was made on some Linux machine by our Geo guy.... Only he can open it... no one here has the machines to deal with it.


Well it's not much use if only one guy on a Linux machine can open it. What do you use an image that big for? Are you making an interactive trip across the country? Or maybe a 15 minute flyover at Mach 16. :D

Cheers,
JS

Lamont
11-21-2003, 11:31 AM
We used to do stuff directly for the Military.

The GIS guy down samples them to like 4096x4096, and we run them through un-holy machines... with un-holy videocards you'd only dream about ;).

RUdiger
11-21-2003, 12:32 PM
A 43200x21600 rgb image will require
43200*21600*3*4 bytes of RAM, not including any extra for mipmaps or antialiasing.

If my calculator tells the Truth, this means over 10678.71 megabytes, or ~10.4 gigabytes of RAM.

Also, an LZW tiff is a bitmap.

Earl
11-21-2003, 12:52 PM
Hi Beam,

RUdiger is correct. LightWave doesn't just load one single instance of the image at one resolution. It requires several versions (this is discussed in the LW manual). So an image that would normally take up a gig or two in PhotoShop will require a lot more in LightWave. It's not that LightWave handles images inferiorly, it's just that LightWave needs to do more with them than PhotoShop.

Beamtracer
11-21-2003, 04:48 PM
Originally posted by Earl
LightWave doesn't just load one single instance of the image at one resolution. It requires several versions

So would that mean that the only image-size limitation in Lightwave is the RAM that the application is able to address? There are no other image limits?

It's interesting that there's other people dealing with very huge images. We're really getting into the territory of 64-bit computers here (64-bit machines can address more RAM).

Hopefully Lightwave 8 will be able to address more RAM for those with new generation machines.

WizCraker
11-22-2003, 01:13 AM
Beam,

I dont think it is just LW as Maya does not load the image as well. At least the 21,600 does not as that makes me believe anything bigger would not load either. funny thing is LW crashes where Maya pops up a error message.

SamuraiSlayer
11-22-2003, 10:55 AM
Beam: why don't you just reduce it down to the resolution that your scene will be exported as? And, what res will your scene be exported as?

SamuraiSlayer
11-22-2003, 10:56 AM
oh nevermind that last post

Adrian Lopez
11-22-2003, 11:15 AM
Originally posted by js33
Yeah that's right I remember now. 43.2k x 21.6k. It's so big you have to get permission to download it. :DDo you need permission because of the amount of bandwidth it consumes, or because of the amount of detail it reveals?

WizCraker
11-22-2003, 02:47 PM
They dont put anything over 20 or so meg on the website it is always better to transfer large files [200+ MB] over FTP than over HTTP protocols.

Beamtracer
11-22-2003, 05:31 PM
If your client wants you to create a zoom from the wide texture image into an extreme close up it can create a few technical problems. Especially when your image is 43200 pixels across.

But, work-arounds must always be found. I'm using a lower-res version of the image for the world, and using a portion of the high-res image for the section I zoom in on.

It's also a bit tricky to make one portion a different image when using spherical or cylindrical mapping, but it can be done.

Lightwave's maximum 2 gig RAM footprint is an issue. Running on a 64-bit Apple G5, Lightwave can take up a full 2 gigs of RAM, and the OS can sit in a separate 500MB of RAM. As far as I know, Lightwave 7.5c on any platform can't use more RAM than this.

I'd really like to see Newtek take advantage of the arriving 64-bit hardware as far as RAM goes. The Apple G5 (IBM-970) and the AMD Opteron (x86-64) are the two viable platforms to watch here.

I think there's an immediate need for at least 4 gigs of RAM with Lightwave, and probably the full +4gig RAM capacity of the new 64-bit computers.

Elmar Moelzer
11-22-2003, 05:31 PM
Oh yeah, I got that one (the 1 KM- res- earth- map).
You will usually only see a small portion of it in an animation and when you are far away enough, to see more, you wont need that high a resolution.
So I usually cut out those parts that I will zoom in on and add them as another imagemap on top of a downsampled version for the entire globe (you can do more steps too). Works pretty well.
Please also notice that textures have to be resampled for OpenGl as well (multiples of 2). So if you display them in OpenGL, you will need even more memory (even though the max OpenGL- texture- res is only 1024x1024 and this wont be that much, but still...).
CU
Elmar

Bytehawk
11-22-2003, 06:14 PM
jezus,

I still have a few 4 gig harddisks in use here.

How time moves fast

Beamtracer
11-23-2003, 12:35 PM
When computer capacity increases (in this case memory), people will soon find ways to use up that capacity.

A few years ago a one gig hard drive was considered huge. Now one gig of RAM is considered small.

Wasn't there once a quote from Bill Gates, who said that computers would never need more than a megabyte of RAM? Ha! Maybe that's why he's not ready for 64-bit computer systems that enable more than 4 gigs of RAM.

There are now many digital still cameras on the market with >10 megapixel image sensors. A year from now there'll be 20 megapixel cameras. Apply a few images from these cameras as textures in Lightwave and you'll soon exceed LW's RAM limits.

As I said, when computer capacity increases, people will find ways to use it!

Rei
11-23-2003, 02:30 PM
not to turn this into a pissing contest...

i have a 100mb harddrive, sitting right next to a 80Gig HD... now thats improvment!
I also run a 1Gig drive as a master for my main machine... fun fun directX issues made me do that!

Bytehawk
11-23-2003, 05:05 PM
you wait till I get my new machine !!!!

:D

wapangy
11-23-2003, 11:41 PM
When I tried to use some of those huge NASA images, I could only get them to open in LightWave and FilmGimp (or whatever its called now). Photoshop could not, strange that its different for you.

Beamtracer
11-24-2003, 12:13 PM
wapangy, if you upgrade to the latest Photoshop it can take much bigger images. Macs also take more RAM.

meshmaster
11-24-2003, 12:48 PM
to those nasa ones... I think I've seen it before, but just want to make sure I'm on the same boat...

yuchenpan
11-24-2003, 06:59 PM
is there a link for that NASA image u guys talking about>??

wapangy
11-24-2003, 07:07 PM
Originally posted by Beamtracer
Macs also take more RAM.

I had 2 gig, with photoshop 7

Beamtracer
11-24-2003, 07:38 PM
Originally posted by wapangy
I had 2 gig, with photoshop 7

Photoshop CS takes larger images than Photoshop 7 does.

The Mac G5 allows more RAM, even for 32-bit apps, because the OS can reside in different RAM. A Mac G5 computer can be decked out with 8 gigs of RAM. Is that enough RAM for you?

There are many satellite and space images at JPL and NASA websites.
http://www.jpl.nasa.gov/earth/
http://visibleearth.nasa.gov/
However the extremely large images I was referring to require written permission from NASA, due to bandwidth overload. I think that applies to anything above about 100MB in size.

wapangy
11-24-2003, 09:22 PM
Originally posted by Beamtracer


The Mac G5 allows more RAM, even for 32-bit apps, because the OS can reside in different RAM. A Mac G5 computer can be decked out with 8 gigs of RAM. Is that enough RAM for you?

However the extremely large images I was referring to require written permission from NASA, due to bandwidth overload. I think that applies to anything above about 100MB in size.

(you don't have to sell me a G5, I have one :) , and I also had to get permission from NASA for the files I used.)

Hervé
11-26-2003, 12:29 AM
well on PC, I am loading max 3000 x 3000 pix tifs, more than that lw crashes, yes just lw has problem with large files, and i have 2 gb ram and 100 gb space disk, so really lw needs a fix here..... (and in memory managment anyway)

Beamtracer
11-26-2003, 01:39 PM
Originally posted by Hervé
so really lw needs a fix here..... (and in memory managment anyway) The fix is to migrate to the new 64-bit platforms... Apple G5 and AMD Opteron (when MS brings out their OS).

Lamont
11-26-2003, 01:57 PM
Originally posted by Hervé
well on PC, I am loading max 3000 x 3000 pix tifs, more than that lw crashes, yes just lw has problem with large files, and i have 2 gb ram and 100 gb space disk, so really lw needs a fix here..... (and in memory managment anyway) Really? I have 512mb ram and 40gb of space and I opened two images; 2048x2048 and 4096x4096. At home I have 768mb ram and I open a 8192x8192 just fine.

What kind of ram are you using?

Lamont
11-26-2003, 02:20 PM
Ok, here's a single 8192x8192... but of course my GF3 doesn't have the power to display it. But I can load this on the GForceFX machine just fine and see it in the viewport (only 2, the rest can not be textured), it get's respectable FPS.

Normally I work on large terrains, so this is twice the normal size I deal with.

When I took this, I had 2 instances of 3DS Max open and Photoshop 7 was open to paste the screen grab.

Dual 900 P3 Xeon 512mb Ram GF3 128 WinXP.

Although for rendering, this could be an issue, but I just crank up the segments...:rolleyes:

CiaranM
11-26-2003, 08:50 PM
Originally posted by Hervé
well on PC, I am loading max 3000 x 3000 pix tifs, more than that lw crashes, yes just lw has problem with large files, and i have 2 gb ram and 100 gb space disk, so really lw needs a fix here..... (and in memory managment anyway)


mmm, that's odd. I'm using 5000x5000 textures in LW at the moment, with just 1GB RAM. Have you tried different file formats other than tifs?

Hervé
11-26-2003, 11:36 PM
well I always had this problem, I am using registred (ECC)ram from Kingston technologies, I have 2x500 and 1x1024 ram bars... also I cannot render big files... max is around 2000x1000 pix.... (but that's another problem (related to Shadow designer, it takes huge amount of memory)

well dunno ... I have a excellent machine here, supposed to be the best machine for LW, big studios are using it.... (BOXX dual AMD 1900-Quadro 750XGL)... well for now I dont need big files, so.... I think the problem comes more with the number of images LW has to manage, rather than than just loading one big file, in the last project I had about 350 image maps, some small, some large, different formats.... I use JPG, TIF (no comp), and PNG....

BTW, Happy Digital is making a jpg enhancer that look rather interesting......

Hervé
I called NT about it, coz'once I had to render a pretty big file, and that was not possible, and NT had no idea why, so at that time I gave up...

Lamont
11-26-2003, 11:43 PM
One thing I tend to notice is that all the 3D apps I use, I tend to use OpenGL friendly texture sizes...

Herve, if you can, try the same scene with a gForceFX 5900 Ultra. I noticed that although the Quadro card is more beefy, the GF series tends to give me more stability in OGL apps.

Ahhh.. yeah... and the all or nothing textures in the viewport is kinda lame. That there could be your problem, not the size of the images...

Beamtracer
11-26-2003, 11:47 PM
Hervé, maybe you should go back to using a Mac like you had before! ;) Then you could exceed 2 gigs of RAM, which seems to be the limit of most Windows machines.

Using compressed images like JPEG will not help your RAM situation. JPEGs will reduce your image quality, but still use up just as much RAM as a TIF or other uncompressed image.

Hervé
11-27-2003, 12:23 AM
yes I know about the tif, jpg, they all take up the same amount of ram.... well going back to mac is way to expensive, and I am using apps that are just not available for macs, and probably never (but I never say never)....

as for open GL, I never use it , to slow in layout, and my scene are just to large for it, and I really dont need it, for me the best is wireframe.... in fact I alwys use wireframe in layout, in modeler I use all the shading options though....

BTW I'd love a shading option to be a mix of shaded wireframe and textured....

also to close the point on images, whether I use a 3000x2000 image (render) or 1000x500, it does not make any difference on my printer (I love my Epson printer -also scanner)), and I never send images for real printing anyway....

Hervé
11-27-2003, 02:31 AM
well Gentleman, since I re-installed LW, it allows now to load a 25,000 x 5,000 pix tif file, now i'd like to know something... this image in photoshop takes 366 mb, but when opened in lw, it says (-154,4 MB).... bizarre no?

let's try how big i can go.....

Hervé
11-27-2003, 02:39 AM
30000x5000..... works !

photoshop does not let me make huge files...

Lightwolf
11-27-2003, 03:01 AM
Hervé,
have you actually tried not only to load it, but to apply it to an object and render it?
I think the real culprit is the filtering that LW does when using an image as a texture.
Cheers,
Mike

Hervé
11-27-2003, 03:28 AM
no havent tried, iam still in discovery edition...

help nt.....