PDA

View Full Version : Which compiler is used by the devs to build Lightwave?



Rayek
04-28-2012, 01:37 PM
This is *not* a post about Blender, but about the impact a compiler can have on the speed of rendering, and how this might reduce render times in Lightwave (for Windows).

Cycles is the new render engine under heavy development for Blender.

In the last couple of weeks a discovery was made: on Windows, Cycles ran much slower than on Linux. A car benchmark scene that is used on the blenderArtists forum by people to compare render speeds between CPUs and GPUs gave me these figures:

4 min 59secs CPU Windows 7 64bit
3 min 20secs CPU Ubuntu 32bit (live distro)

Quite a difference. The discrepancy was finally explained due to the inefficient(?) machine code generated by MS Visual C++.

So a couple of MinGW 32bit/64bit builds have appeared for Windows, and this is the result:

2 min 52secs CPU Windows 7 64bit

A huge improvement of 42%(!) on Windows caused by merely using a different compiler. That is just a ridiculous difference in render time.

Now, my question: this is just a guess, but I suspect the Lightwave developers are probably also using Visual C++ to build Lightwave? Obviously, both Blender and Lightwave are written in and for a very different environment, but still - my interest is piqued.

Are the LW devs using Visual C++, and if so, could the switch to MinGW cause a similar boost in render speed?

evenflcw
04-28-2012, 03:59 PM
Good question. Otherwise I think the Intel compiler is often touted as the one that generates the fastest executable code. One hopes (and assumes) NT have at one time or other reviewed a couple of compilers. But it would be nice and interesting to hear from the horse's mouth. They have partnered with Intel on trade shows - that would imply that they are using something of theirs (other than OS) - probably parts for the video editing systems, but could it also be the compiler!?

roboman
04-28-2012, 04:20 PM
Interesting. I would have assumed that the time critical core parts of the render engine would have been Assembly language, in either Lightwave or Blender. Maybe that's just old school thinking. I know that use to be standard, but then drives didn't use to be measured in tb and cpu's weren't rated in ghz back then. Might be unreasonable with the size and functionality of modern programs.

juanjgon
04-28-2012, 04:44 PM
Best code optimization for windows and perhaps for any plataform is made by Intel compiler, AFAIK. LW bin folder has the tbb.dll who is the Intel Threading Building Blocks library. Perhaps some critical code of LW, or all, is compiled with Intel compiler.

-Juanjo

Lightwolf
04-28-2012, 04:52 PM
I just checked, and it seems that the official Blender binaries don't use the current MSVC compiler, which does produce better code especially when it comes to 64-bit.
However, it still doesn't have some of the functionality that the latest GCC 4.7 has (which was used for the test build), that's slated for the MSVC version to be released this year (in open beta now).
However, afaik LW does indeed use the Intel compiler which certainly produces extremely good code.

@roboman: Some core routines may still be written in assembler, especially to make use of newer CPU functionality. However, that's usually just on the level of single functions that get replaced.
One problem is maintainability across 32-/64-bit (which usually means writing different code) as well as different compilers on various platforms. And it may just be obsolete for the next generation of CPUs and compilers.

Cheers,
Mike

Lightwolf
04-28-2012, 04:53 PM
LW bin folder has the tbb.dll who is the Intel Threading Building Blocks library.
TBB isn't an indication of the compiler that's being used. Nowadays there's an open source version that runs on a variety of platforms and compilers.

Cheers,
Mike

warmiak
04-28-2012, 05:31 PM
I doubt it.

I have been compiling various code bases with VC++ and GCC for the last 10 years and I have never seen GCC producing executables outperforming VC++ output.
In fact, the only Win32 compiler that outperforms VC++ comes from Intel.

Lightwolf
04-28-2012, 05:38 PM
I doubt it.

I have been compiling various code bases with VC++ and GCC for the last 10 years and I have never seen GCC producing executables outperforming VC++ output.
Here you go then: http://www.graphicall.org/918

One hunch would be auto-vectorisation - but then again that's old news for GCC. On the other hand, the link time optimiser is relatively new.
I also suspect that the MSVC build doesn't use POGO (PGO) - but that's quite tricky to do for a big and complex application as well.

I can only compared gcc 4.0/4.2 to MSVC 2010 anyhow, since those are the main ones I use.

Edit: http://www.g-truc.net/post-0372.html hints at visual studio up to 2008 producing fairly iffy SSE code in comparison. That may well be a root cause as well, since 2008 is the windows compiler used for the official binaries.

Cheers,
Mike

Sensei
04-28-2012, 06:55 PM
In fact, the only Win32 compiler that outperforms VC++ comes from Intel.

When I was testing Intel compiler vs VC++ in mine renderer (compatible to LW) Intel was always slower. But that was 4 years ago or so.
In VC++ the most important thing to enable is using Floating Point Model: Fast. Default is Precise, which is generating all floating operations as functions, to which code is jumping (!).. So, default settings are several times slower. Ray-tracing renderer is very vulnerable for this settings.
Enabling SSE1/2 doesn't have such impact as FP Model- SSE1 was/is 8% faster than without, and SSE2 was/is 8% faster than SSE1 (so 17% faster than no SSE).

Hand made SSE2/3/4 would change results though. But that would require complete rewrite of renderer..

Rayek
04-28-2012, 07:00 PM
Wasn't the Intel compiler biased in favour of their own Intel CPUs, against AMD, producing less than ideal code for AMD CPUs? I read somewhere that Intel was forced to mention this in their documentation.

Sensei
04-28-2012, 07:02 PM
I was making test on Intel 2 Core..

Lightwolf
04-29-2012, 12:32 AM
Wasn't the Intel compiler biased in favour of their own Intel CPUs, against AMD, producing less than ideal code for AMD CPUs?
It did... but it the resulting binary could be patched to treat AMD CPUs the same way. The development team at Eyeon did that for example (they may still do).

Cheers,
Mike

Titus
04-30-2012, 12:57 PM
Now, my question: this is just a guess, but I suspect the Lightwave developers are probably also using Visual C++ to build Lightwave? Obviously, both Blender and Lightwave are written in and for a very different environment, but still - my interest is piqued.

While I don't know the answer., something I've noticed when LW crashed at render time, there was an Intel compiler message. But will wait for an answer.

And yes, I can confirm Blender internal renders run faster in Linux than in Windows.

jeric_synergy
04-30-2012, 01:17 PM
And yes, I can confirm Blender internal renders run faster in Linux than in Windows.
Anything like the 42% cited above???

Sensei
04-30-2012, 01:21 PM
That might depend on how Blender is updating renders (if it's rendering with previewing turned on).

Rayek
04-30-2012, 01:36 PM
Anything like the 42% cited above???

No, that is not the case. Up to about 1 1/2 week ago the new Cycles renderer did run *much* slower, though, in Windows. With the newer builds I actually get slightly faster render times in Windows. But with all these different optimized builds, it is hard to make a direct comparison at the moment.

It is interesting to see how the switch to a different compiler can make such a huge impact on performance. Being proprietary, biased against AMD, and costly, it only makes sense for a commercial product to make use of it - open source would never touch it.

However, looking at some of the figures Intel is boasting about, a similar speed boost is mentioned by Intel when comparing their compiler with the baseline: visual studio. So, I gather the WinMG and Intel compiler would be on par in terms of execution speed in regards to their compiled executable.

And, looking at this comparison:
http://www.nersc.gov/users/computational-systems/hopper/performance-and-optimization/compiler-comparisons/
(check out bottom of the page)

I seem to be correct in my thinking. The GNU and Intel compilers create the most optimized code.

Titus
04-30-2012, 01:51 PM
Anything like the 42% cited above???

Yes, or even more on certain scenes. We're in the rendering phase of an ani mated shortfilm and we started with windows boxes; a mix of XP, vista, win 7, 32 and 64 bit. A Blender (internal not cycles) render result is the same regardless the hardware-OS, but stability. So I've decided to install Ubuntu on the problematic machines first, then in the rest. A couple of problematic renders went from 40~ to 15 minutes in side by side comparisons.

Titus
04-30-2012, 02:06 PM
I'm checking right now a scene and after leaving a sequence rendering over night, the Blender sequence (100 frames) ended an hour before than the windows box.

dsol
04-30-2012, 02:12 PM
I wonder if Mac Lightwave takes advantage of the newer C/C++ support for LLVM in the latest versions of Xcode. It is amazing how much of a difference the compiler you use makes, though perhaps not if you think about what it's doing!

Rayek
04-30-2012, 03:52 PM
Yes, or even more on certain scenes. We're in the rendering phase of an ani mated shortfilm and we started with windows boxes; a mix of XP, vista, win 7, 32 and 64 bit. A Blender (internal not cycles) render result is the same regardless the hardware-OS, but stability. So I've decided to install Ubuntu on the problematic machines first, then in the rest. A couple of problematic renders went from 40~ to 15 minutes in side by side comparisons.

Titus, I take it this is the older, non-MinGW compiled Windows version you are comparing to Linux? How do those scenes fare on a current MinGW build?

On my machine the huge difference between Windows and Linux disappeared - actually, the Windows version is a tad faster now (but again, comparing apples with pears).

Rayek
04-30-2012, 03:54 PM
Btw, I wonder if any of the Lightwave devs could chime in, and enlighten us a bit regarding compiling Lightwave? And has anyone made comparisons between the Mac and Windows versions on the same hardware?

Lightwolf
04-30-2012, 04:54 PM
I wonder if Mac Lightwave takes advantage of the newer C/C++ support for LLVM in the latest versions of Xcode. It is amazing how much of a difference the compiler you use makes, though perhaps not if you think about what it's doing!
If you look at the benchmark page I posted then you'll see that, for the code tested (a raytracer) there is little difference in performance. And the advantage goes either way depending on the specific version and CPU extensions used.
Having said that, LLVM is very interesting as an integrated compiler for various purposes.

Cheers,
Mike

Titus
05-01-2012, 10:27 AM
Titus, I take it this is the older, non-MinGW compiled Windows version you are comparing to Linux? How do those scenes fare on a current MinGW build?


I don't know about this MinGW build. We haven't changed version for this production, we're still running 2.49a. After this project ends we'll move to 2.5x branch.

Rayek
05-01-2012, 01:13 PM
Ah, in that case you will see a huge speedup in render times - 2.49 is *much* slower than 2.63, with Blender internal.

The official 2.63 build takes 6.48 seconds to render the scene at http://www.eofw.org/bench/
The MinGW bat3a 2.63 build renders the test scene in 6.2 seconds. The official 2.49b build takes 24 seconds (same settings). And an optimized build of 2.49 build I used in the past takes 14.28 seconds.

Talking about render time comparisons, did anyone here compare render times with identical scenes between LW 9.6 and LW 11? I still have not decided to upgrade, and if render times have drastically decreased, that would entice me more than all the new bells and whistles.