dcreager.net

Filtering some functions in a flamegraph

I'm doing some performance profiling on red-knot, using cargo-flamegraph to generate some flamegraphs. We're using salsa and rayon heavily, and those introduce a lot of cruft into the stack traces.

red-knot

Luckily you can use a post-process step to filter those out!

(I'm also breaking this apart into two steps — that lets you run the command you want to profile once, and iterate on different flamegraph options while processing the same file of raw profiling data.)

$ perf record -F 1024 --call-graph dwarf,16384 -g -o perf.data [COMMAND TO PROFILE]
$ flamegraph --open --inverted --perfdata perf.data \
  --post-process "awk -v RS=';' -e '/\\n/ || !(/FUNCTION1/ || /FUNCTION2/ || /ETC/) { printf \"%s%s\",sep,\$0; sep=\";\" }'"

The parts to substitute are the COMMAND TO PROFILE in the first step, and the list of regular expressions (FUNCTION1 etc) in the second.

How does it work

The perf data that goes into the post-process script has one stack trace on each line, with the individual functions separated by ;. The ‘RS=;’ bit causes awk to use semicolon as the input record separator, so the awk program will be run against each semicolon-separated chunk of text. The regular expression filter throws away any records that match any of the function names that we don't want to include.

Importantly, that means that the newlines are no longer treated specially! The last field of one line will be joined with the first field of the next as a single “record” for this awk program. The first /\\n/ clause in our awk program ensures that we don't throw away any newlines, even if they would otherwise be thrown out by our function filter.

With this filter in place, all that's left is to output each matching record. Normally, you could use the default ‘{print}’, though you'd want to set ORS to make sure that awk puts the semicolons back in place in between each record.

But unfortunately that doesn't work because it treats ORS as a terminator, not a separator, so you end up with a trailing semicolon at the end of the post-processed data. The printf and sep bits are a fix to this problem, as described in this Stack Overflow answer:

Make the record seperator in AWK not apply after the last record? [Stack Overflow]