kwk wrote: I want to add something as a side note because it is PGO related. Last year I've experiment with a PGO enabled LLVM toolchain on Fedora ([1], [2]).
## Summary of my experiment For training data I've used a set of packages that we build on Fedora and I've smuggled in my PGO toolchain. The RPM packaging was also tweaked to pick up the profile data and create a sub packaging automatically just like we do for debug information. This part was rather tricky and RPM related but it worked and we ended up having a profiledata-sub-package for any given project in Fedora that uses LLVM. Like I've said I've only used it for a handful of project as demonstration. ## Encountered problem But back to the point I wanted to make: **disk space for profile data** can quickly increase. I've [compiled chromium](https://kwk.github.io/pgo-experiment/#find_and_merge_profiles) in my tests and our builders exploded because of diskspace just for profile data: ``` LLVM Profile Error: Failed to write file "/builddir/build/BUILD/raw-pgo-profdata //chromium.llvm.1970228969820616430_0.24617.profraw": No space left on device ``` ## Question Is it possible to have CMake collect and merge the profile data in the background? I've used tools like [`inotifywait`](https://src.fedoraproject.org/fork/kkleine/rpms/llvm/blob/pgo/f/pgo-background-merge.sh#_128) in my PoC to look for `close_write` events to any file under a build directory that matches `.*\.profraw$`. When an entry was found, I've stored it into a file to be consumed and all profiles in it merged, once there are 10 files. This continuous merge helped me reduce the size of total profile data from ~1,4GB (unmerged) to 1,6MB (merged). This was for the `retsnoop` project. Chromium simply took to long to build for my experiment. I understand this is a special use case but one should not underestimate the size of profile data, that's all I'm saying. [1]: https://kwk.github.io/pgo-experiment/ [2]: https://developers.redhat.com/articles/2023/11/07/how-i-experimented-pgo-enabled-llvm-fedora https://github.com/llvm/llvm-project/pull/77347 _______________________________________________ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
