CMake Performance with Open Babel

Recently, Luca made a post comparing the speed of CMake and autotools in which some timings were posted. I have to say that I am not sure I agreed with the conclusion and have had a very different experience with the projects I am involved in. As with anything your mileage may vary, and I have not looked at Wesnoth.

I think it is questionable at best to include the time it takes to build CMake, but not autotools. Seems like this is a one time cost and the build time is not that high for either. All tests were performed on my quad core Gentoo box at work. Each step is for the first cold run as would normally be the case when compiling Open Babel from source. The make step used `time make -j5` and I have listed the real time in each case.

The timings are shown in the table below. They seem somewhat similar to the experiences of the QGIS developers who made this move quite some time ago. All times shown are in seconds and are the real time reported by the time command.

  Autotools CMake
Configure time 20.39 4.79
Compile time 116.45 102.23
Install time 28.39 4.19
Total time 165.23 111.21

For those interested, on this system the total CMake compilation/installation time (cmake-gui disabled) was 1 minute and 54 seconds. The compilation/installation time for automake, autoconf, libtool, m4 was 2 minute and 14 seconds. I am not sure how relevant either of those times are, other than to show neither of them take that long to compile and install. Gentoo users/developers may or may not have CMake installed, most other developers will install the binary packages for either one and are likely to be much more interested in how well it integrates with their development environment, compile and install times.

As a developer I prefer CMake, and have been using/maintaining the CMake based build system for Open Babel for over a year now. It was originally contributed by the KDE Windows porting team, but I found that I spent less time waiting for it to do things when I was working on code too. Add to that the extras CMake comes with, such as CTest, CDash and CPack I think it makes a very attractive option for many projects. I am also hoping that it will allow Open Babel to drop maintaining a totally separate build system for MSVC.

I am sure the Open Babel autotools build system could be optimized (I never tried), but when you add in the additional benefits mentioned above, support for multiple targets such as makefiles, XCode, MSVC, Eclipse etc, one shared language/syntax for all build files and an increasingly polished competitor to autotools, I honestly think it is a sensible choice for projects to move to CMake. There are a few less well known features such as Fortran module dependency parsing that I think are fairly unique and valuable, in scientific coding at least (and I have used the Fortran module dependency parsing at least once and was pleasantly surprised).

Full disclosure: I recently accepted a job offer with Kitware, and will start in the Fall assuming all the visa paperwork falls into place. The opinions expressed here are my own. I think it is great to discuss issues like this objectively, and hope to be a part of making CMake a better build system. As with most software - there are areas that need improving.

Share Comments
comments powered by Disqus