I’ve been improving our font rendering again over the last one or two weeks, which in turn brought me to testing its performance towards the end. In a weird turn of events though, I found myself going around the world of operating systems and compilers trying to finally find an acceptable alternative to MinGW’s slow compile times.

It is no news to me that MinGW compile times were longer, even Lorenz was cursing over it many months ago, but I got used to it and never looked back. It didn’t help that I was enjoying Eclipse and the only other alternative was Microsoft Visual Studio’s C++ compiler.

But it was finally time to get some numbers out and perhaps switch my main development environment from Windows to Linux – I’ve always felt not having all GNU utilities, ssh and other native Unix tools readily available was an impairment.

MinGW 4.5.0 – Windows 7 64-bit – i5 750 @ 2,67 GHz, 4 GB DDR3-1066

My first commandment was to time my every day Windows + Eclipse + MinGW build times. After scratching my head for a few minutes how to time it I saw CMake has an -E option which “provides a list of commands that can be used on all systems”, including time, how nice of them. I changed Eclipse’s build command to “cmake -E time make -j4″ and off I went. We’re really only interested in the relative differences between the compilers, especially since it’s a private code base.

  • Debug (no changes):
    Engine: 1 s
    Valerie: 4 s

  • Release (no changes):
    Engine: 1 s
    Valerie: 3 s

  • Debug (engine touch Clock.cpp):
    Engine: 19 s
    Valerie: 25 s

  • Release (engine touch Clock.cpp):
    Engine: 4 s
    Valerie: 9 s

  • Debug (everything):
    Engine: 117 s
    Valerie: 140 s

  • Release (everything):
    Engine: 126 s
    Valerie: 150 s

* I used Valerie for the name of our new game project as to not reveal too much just yet (it has no relevance to it).
** To get the total wait time sum both Engine and Valerie.

Alright this looked pretty bad, especially little changes on debug, but I already knew linking times there were over the roof on debug.

After reading about sub-second incremental compile times and grinning about it for a while, it was time to get some comparison timings. Being my lazy self, instead of testing it on Linux or god forbid MSVC, I grabbed my MacBook Pro and -j4 here I went.

GCC 4.2.1 – Mac OS X 10.6.7 – i5 (520M) @ 2.4 GHz, 4 GB DDR3-1066

  • Debug (no changes):
    Engine: 0 s
    Valerie: 1 s

  • Release (no changes):
    Engine: 1 s
    Valerie: 1 s

  • Debug (engine touch Clock.cpp):
    Engine: 2 s
    Valerie: 2 s

  • Release (engine touch Clock.cpp):
    Engine: 1 s
    Valerie: 2 s

  • Debug (everything):
    Engine: 139 s
    Valerie: 166 s

  • Release (everything):
    Engine: 180 s
    Valerie: 188 s

In the email to Lorenz, which had these results, I included “I think I might switch to either Linux or OS X.”. Obviously this meant just for development purposes, as everyone clearly still needs Windows for gaming. You have to understand this was quite a bit weaker machine and yet incremental compile times were a lot faster – I could actually compile on debug!

Switching to my Mac running OS X was a pretty meh solution. I’d have to hook it up to another monitor and plug in my too-expensive keyboard and sharp mouse. Doing these switches, not using the MacBook’s cool trackpad and generally using a slower machine didn’t make a lot of sense. Linux? That’d be rad, it would give me the push to drop the last few proprietary tools, but I’m not big on dual booting. Getting comfortable with the workflow and having to manage two systems on one machine irks me, moreso with questionable file system support for the other OS. I would optimally also have to get another hard drive.

Checking up on Ubuntu I see it has a “run it alongside windows” solution. Interesting, it’s not a VM, it installs like an application, creating an image file on your drive and modifying your boot loader to boot into it. Sounds cool, I don’t need to mess with anything and I can easily eradicate it.

To keep the next several hours short, the experience was not amazing. Unity’s impressions went from cool to “I want my taskbar on while my Firefox is fullscreen”, “alt-tabbing reacts too slow”, “dual screen actions are a bit weird”. Anyhow, I managed to grab Eclipse CDT (even if it acted weird), all the project dependencies and compile the boss.

GCC 4.5.2 – Ubuntu 11.04 64-bit – i5 750 @ 2,67 GHz, 4 GB DDR3-1066

  • Debug (no changes):
    Engine: 0 s
    Valerie: 0 s

  • Release (no changes):
    Engine: 0 s
    Valerie: 0 s

  • Debug (engine touch Clock.cpp):
    Engine: 1 s
    Valerie: 2 s

  • Release (engine touch Clock.cpp):
    Engine: 1 s
    Valerie: 1 s

  • Debug (everything):
    Engine: 51 s
    Valerie: 59 s

  • Release (everything):
    Engine: 55 s
    Valerie: 57 s

Now that looks nice. Finally a proper comparison. I mostly care about little incremental builds (engine touch Clock.cpp) on debug (so I can actually use the debugger if needed) where it’s a whooping ~14,67 times faster (44 s to 3 s), but even on release it’s ~6,5 times faster (13 s to 2 s). Just checking the files seems insanely slower on Windows (5 s to < 1 s). Finally, recompiling everything is ~2,34 times faster (4,28 min vs ~2 min).

That’s quite a compelling argument to switch, but my experience with Ubuntu and the hassles of dual boot didn’t convince me (I also like the Windows Eclipse fonts more). I think I also got somewhat used to it. I compile on release and when it takes a bit longer to compile I clean up some code or think about the next change. As long as I work on engine code it’s kinda okay, but when I do game GUI work it becomes almost unbearable.

Another interesting note on this test was that I started feeling the difference between verbose and non-verbose mode. Even though the reported times were the same, with verbose the printing took longer than the actual build and one could really feel it. Unlike as on Windows where everything takes forever anyway.

My next thought was using the Visual Studio C++ compiler within Eclipse with the help of CMake’s NMake makefile generator. Lorenz uses that for his thesis. Compilation gets faster, but you lose the debugger. I’m not sure how much faster since I didn’t feel like recompiling or downloading all the library dependencies and fixing any other compilation issues. Perhaps I’ll try it another day if I’m desperate enough. And no, I’m not switching to Visual Studio.

There is another beacon of hope however. Apple uses clang/llvm in their new Xcode 4, and there seems to be an Eclipse CDT plugin in development for llvm, but sadly it doesn’t support building C++ projects on Windows just yet. Godspeed boys.

Related Posts: