Benchmarks
From Sunflow Wiki
Contents |
Java Virtual Machine (JVM): Client vs. Server
Sunflow renders faster if you use a server JVM. If you load Sunflow and see a message stating that "API error : You do not appear to be running Sun's server JVM Performance may suffer. This means that you are running Sunflow using a client java virtual machine (though I've found Fedora 8's IcedTea Java will display this even though it is running a server JVM). You can instead use the server JVM by installed the Java JDK and pointing to it when loading Sunflow and adding the -server switch as in the below example:
java -Xmx2G -server -jar sunflow.jar
So what's the difference between server and client? Here's the excerpt from the Java VM FAQs:
"These two systems are different binaries. They are essentially two different compilers (JITs) interfacing to the same runtime system. The client system is optimal for applications which need fast startup times or small footprints, the server system is optimal for applications where the overall performance is most important. In general the client system is better suited for interactive applications such as GUIs. Some of the other differences include the compilation policy, heap defaults, and inlining policy."
Benchmarks
Author mikews99:
I've been investigating various renderers and found out about SunFlow. Curious as to its performance, especially since it was a java app, I ran some benchmarks on several different platforms to get an idea of its efficiency. After reading this article, I thought I would compare various OS'es JVM implementation on a single computer in hopes that I'd find the best setup for rendering.
The machine I selected was one (that I had) that I could get the OSes I tried installed properly with minimal variations:
AMD Athlon 64 3800+ (2.4GHz) single core socket 939, nVidia 6150 chipset, using the onboard video 2GB DDR400 memory Hitachi 80GB IDE hard drive on its own channel Yamaha IDE CD-RW on its own channel
OS's Tested
| OS | Notes |
|---|---|
| Ubuntu Linux 7.10 | standard installation |
| Windows XP SP2 | with current patches |
| Windows Vista | no notes |
| Windows 2000 SP4 | hacked for AMD installation |
| Fedora 8 | no notes |
| Ubuntu x64 Linux 7.10 | standard installation |
| Windows Vista 64 | no notes |
JVM's Tested
| VM | Notes |
|---|---|
| IBM JDK Java 6 server for Linux 32-bit | beta |
| Sun JDK Java 6 server for Linux 32-bit | installed from Ubuntu apt repository |
| OS X Java 32-bit for 10.4.10 (Tiger) | Apple developer preview 6 |
| Sun JDK Java 6 server for Windows 32-bit | 1.6.0_03 |
| Sun JDK Java 6 server for Linux 32-bit | RPM for Fedora |
| JDK Java 1.7.0 for Fedora 8 | IcedTea |
| Sun JDK Java 6 server for Linux 64-bit | installed from Ubuntu apt repository |
| Sun JDK Java 6 server for Windows 64-bit | 1.6.0_03 |
The Windows installations were stripped of any unnecessary services and all background processes were killed. The *nix installs had all unnecessary processes killed.
Results
I used the Gumbo_and_teapot.sc example file as a test case. Since I found that the video output affected the times, I minimized the image window when testing. To even out the results, I ran the test four times, but only counted the last three runs so as to eliminate any effects of disk caching. These are the results averaged over three runs:
| OS | VM | Time |
|---|---|---|
| Ubuntu Linux | IBM Java | 5m:36s |
| Ubuntu Linux | Sun Java | 4m:20s |
| OS X | Apple Java | 4m:06s |
| Windows XP | Sun Java | 3m:54s |
| Windows Vista | Sun Java | 3m:50s |
| Windows 2000 | Sun Java | 3m:48s |
| Fedora 8 | Sun Java | 4m:26s |
| Fedora 8 | IcedTea Java | 4m:44s |
| Ubuntu x64 Linux | Sun Java | 4m:04s |
| Windows Vista 64 | Sun Java | 3m:49s |
Conclusions
It was quite surprising to see that Sun Java on Windows XP bested the *nix implementations by a such a large margin. I was even more surprised to see the IBM version fair so poorly. Granted it was the current beta version, so perhaps it hasn't been optimized, but for it to trail by a minute was still quite a lot. Ubuntu had a big improvement going from 32 to 64 bit. Windows Vista 64 bit showed no significant improvement.
Since this is only one file, it's imprudent to draw an absolute conclusion. Large, complex scenes may very well give different results. Scenes that require more memory than the 32 bit OSes can support will only be able to render on 64 bit based OSes. None the less, one can at least take this data to make a more informed choice as to OS and renderer.
As requested I used the Apple Java developer preview 6 on OS X which helped improve it 15 seconds (along with the image window being closed). Also, closing the image window on Ubuntu gave an 11 second improvement. Since the Ubuntu 32 and 64 bit tests were so far apart, I tried installing Fedora Core 8 32 bit and running the tests on both the preinstalled JVM and the Sun JVM to see if there was a problem with the Ubuntu packages. The Fedora times were 4:44 for the included JVM and 4:26 for the RPM downloaded from Sun.
I really can't think of a good reason why the Linux 32 bit implementations should be so much slower than either the 64 bit version or the 32 bit Windows versions. Either Sun didn't bother turning on optimizations for Linux, or they used the gcc compiler, which just doesn't cut it (they should have used the Intel compiler).
