Template Instantiation Profiler with Callgrind Output (KCacheGrind)
Hi boost developers and users, I just wanted to announce a recent development in my little pet-project, which is Templight: a template instantiation profiling tool based on Clang. I think that many of you would be interested in experimenting with it because many boost libraries are great examples of template-heavy libraries and many maintainers, developers and users might be interested in gathering data and diagnosing the real compilation costs (in time and memory) of various components of Boost. Templight is a Clang-based profiler that can act as a drop-in replacement (e.g., in cmake) for the clang compiler, with the addition of generating a complete trace of the template instantiation history with associated time and memory (optional) costs. This allows you to see not only how deeply recursive certain template instantiations are, but also identify which instantiations the compiler spends the most time or memory on. It is basically to meta-programming what a run-time profiler is to ordinary code. The templight profiler itself has reached a fairly stable state. It works at least under Linux and Windows (through "templight-cl.exe", which is a drop-in replacement for "clang-cl.exe", which is compatible with MSVC "cl.exe"), but probably works wherever clang works. Which means that you are welcome to try it. I have also used it successfully to build a few cmake-based projects (by setting CC and CXX environment variables to point to templight). Templight is currently also used in the back-end of the "metashell" project as well (an interactive shell interpreter and debugger for C++ template meta-programs). You can build the templight profiler against the clang source (with a patch + some added code), as it instructs on the github readme: https://github.com/mikael-s-persson/templight But most recently, I have created a separate set of tools to convert templight trace files to other formats. The most awesome of these outputs is the "callgrind" output, which produces a "meta-"call-graph, that is similar to (and in the same format as) call-graphs generated by run-time profiling tools like callgrind, but the profiling data is, of course, related to the compilation costs (time / memory). That conversion tool is available here: https://github.com/mikael-s-persson/templight-tools With that, you can open the converted templight trace files in the visualization tool KCacheGrind, which is great (works both on unix-like systems and Windows, thanks to Qt5). Here is a screenshot of that for one of Boost.Container's example programs: https://raw.githubusercontent.com/mikael-s-persson/templight-tools/master/im... I really hope that some of you will give this a try. I'm really hoping that it will be useful to you, and that you can provide feedback and feature requests. I couldn't think of any better target audience for this tool. Cheers, Mikael. -- Sven Mikael Persson, M.Sc.(Tech.) PhD Candidate and Vanier CGS Scholar, Department of Mechanical Engineering, McGill University,
On 1/25/2015 8:33 PM, Mikael Persson wrote:
Hi boost developers and users,
I just wanted to announce a recent development in my little pet-project, which is Templight: a template instantiation profiling tool based on Clang. I think that many of you would be interested in experimenting with it because many boost libraries are great examples of template-heavy libraries and many maintainers, developers and users might be interested in gathering data and diagnosing the real compilation costs (in time and memory) of various components of Boost.
Templight is a Clang-based profiler that can act as a drop-in replacement (e.g., in cmake) for the clang compiler, with the addition of generating a complete trace of the template instantiation history with associated time and memory (optional) costs. This allows you to see not only how deeply recursive certain template instantiations are, but also identify which instantiations the compiler spends the most time or memory on. It is basically to meta-programming what a run-time profiler is to ordinary code.
The templight profiler itself has reached a fairly stable state. It works at least under Linux and Windows (through "templight-cl.exe", which is a drop-in replacement for "clang-cl.exe", which is compatible with MSVC "cl.exe"), but probably works wherever clang works. Which means that you are welcome to try it. I have also used it successfully to build a few cmake-based projects (by setting CC and CXX environment variables to point to templight). Templight is currently also used in the back-end of the "metashell" project as well (an interactive shell interpreter and debugger for C++ template meta-programs). You can build the templight profiler against the clang source (with a patch + some added code), as it instructs on the github readme:
What is to keep clang from changing the files which templight patches or the llvm/tools/clang/tools/CMakeLists.txt which I am instructed to manually change. Is there a valid place to further discuss templight if I have problems ?
But most recently, I have created a separate set of tools to convert templight trace files to other formats. The most awesome of these outputs is the "callgrind" output, which produces a "meta-"call-graph, that is similar to (and in the same format as) call-graphs generated by run-time profiling tools like callgrind, but the profiling data is, of course, related to the compilation costs (time / memory). That conversion tool is available here:
https://github.com/mikael-s-persson/templight-tools
With that, you can open the converted templight trace files in the visualization tool KCacheGrind, which is great (works both on unix-like systems and Windows, thanks to Qt5). Here is a screenshot of that for one of Boost.Container's example programs:
https://raw.githubusercontent.com/mikael-s-persson/templight-tools/master/im...
I really hope that some of you will give this a try. I'm really hoping that it will be useful to you, and that you can provide feedback and feature requests. I couldn't think of any better target audience for this tool.
What is to keep clang from changing the files which templight patches or
Hi Edward, the llvm/tools/clang/tools/CMakeLists.txt which I am instructed to manually change. For now, this hasn't been a problem. I haven't had to make any significant changes to the patch in at least 6 months. I am in the process of submitting that code (the patch) to clang such that this step will be removed in the future for templight, meaning that it will build against the unpatched clang sources, once that patch has gone through clang's review process. For the moment, if, some time in the future, you have any issues with applying the patch with a newer version, then just notify me and I'll make the necessary updates to it.
Is there a valid place to further discuss templight if I have problems ?
You can discuss them on the github pages, just start an "issue" if you have anything to discuss. For templight itself: https://github.com/mikael-s-persson/templight <https://github.com/mikael-s-persson/templight-tools> For templight-tools: https://github.com/mikael-s-persson/templight-tools For the patch being submitted to clang: http://reviews.llvm.org/D5767 Or else, you can just email me. Thanks, Mikael.
On 1/28/2015 1:40 PM, Mikael Persson wrote:
Hi Edward,
What is to keep clang from changing the files which templight patches or the llvm/tools/clang/tools/CMakeLists.txt which I am instructed to manually change.
For now, this hasn't been a problem. I haven't had to make any significant changes to the patch in at least 6 months. I am in the process of submitting that code (the patch) to clang such that this step will be removed in the future for templight, meaning that it will build against the unpatched clang sources, once that patch has gone through clang's review process. For the moment, if, some time in the future, you have any issues with applying the patch with a newer version, then just notify me and I'll make the necessary updates to it.
Thanks ! The ideal is to get Templight into clang directly so that building clang just builds Templight also. It is great that the way clang is designed it can provide advanced tools for programmers to use, and I applaud that effort by clang developers such as yourself.
Is there a valid place to further discuss templight if I have problems ?
You can discuss them on the github pages, just start an "issue" if you have anything to discuss.
For templight itself: https://github.com/mikael-s-persson/templight <https://github.com/mikael-s-persson/templight-tools>
For templight-tools: https://github.com/mikael-s-persson/templight-tools
For the patch being submitted to clang: http://reviews.llvm.org/D5767
Or else, you can just email me.
I will use the "issue" feature of Github if I run into anything I do not understand when trying to use Templight.
participants (2)
-
Edward Diener -
Mikael Persson