Yes you are correct on all your observations. We already use ninja and ccache wherever we can. In fact we have an issue about the whole end-to-end build performance where we track all effort on this throughout the project (if you're interested: https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/issues/1649)
There are indeed quick and easy wins to be had with those tools (and we are working on it). But my original question is not about that. It's about speeding up the configure step with makefile generator. I still don't think it's normal that it does hundreds of thousands of I/O on files that are a few bytes or even empty. However it's possible that it's because we do something incorrect in our CMakeLists.txt and not CMake's fault. It's true that a few minutes of configure is not much when doing a full build, but consider incremental builds where all you do is change one cxx file (and cmake is triggered because you changed git branch or something). Then the generate step is a significant part of the time you wait. Victor Poughon De : Eric Noulard <eric.noul...@gmail.com> Envoyé : mercredi 14 novembre 2018 12:34 À : Poughon Victor <victor.poug...@cnes.fr> Cc : CMake Mailinglist <cmake@cmake.org> Objet : Re: [CMake] Tips on reducing configure time Le mer. 14 nov. 2018 à 12:06, Poughon Victor <victor.poug...@cnes.fr<mailto:victor.poug...@cnes.fr>> a écrit : Thanks for the replies You can see configure times on our dashboard (switch to advanced view): https://dash.orfeo-toolbox.org/index.php?project=OTB It ranges from 1-3min. There's even one windows machine that's at 4m30s. Somes windows system are known to be slow on many files accesses. However on this particular machine (https://dash.orfeo-toolbox.org/buildSummary.php?buildid=342008 I guess) you already seem to be using NInja + MSVC. so many small files access to ?makefiles? should not be the culprit. That said whatever the system your build and test time seems to dominate the overall time (in the CI). I have had great speed-up experience using CMake+ninja+ccache for our build in CI. We did go from 1 hour down to less than 10min. We only have various linux build (no Windows) it is really a big time savior combo. We even share the compil' cache between the CI and the developers. So everybody is feeding compile cache and takes benefits from others including CI. You seem to be using that as well here: https://dash.orfeo-toolbox.org/buildSummary.php?buildid=341993 and you get less than 5min build time which corroborate my suggestion. The same machine build with clang: https://dash.orfeo-toolbox.org/buildSummary.php?buildid=342033 is lasting 30min+ and is not using ccache. Thanks for the tip about Ninja generator, it's indeed much faster. But switching the entire dev team + CI servers from make to ninja is not really an option for us right now because of a bug in ninja that affects our SuperBuild configuration (see here if you're interested in the details: https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/issues/1625) I see. Not using ninja in the Superbuild case does not mean you cannot use it in other cases, right? May be the one-size fits all is a little too much to ask at this point? My opinion though. Did you report the issue upstream (ninja or cmake)? I was impressed in the past by the amount of tests orfeo is doing, I'm still are !! Are you running those tests in parallel (with ctest --parallel) ? It can be a time savior as well is the running machine has a fair amount of cores and memory. -- Eric
-- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
