Sysprof 3.28 Feature Planning

[Christian Hergert <christian hergert me>]

Hi Folks,

TL,DR: I want to add features to Sysprof for GNOME 3.28.

Overview
--------

 * New memory visualizer to track system/process memory usage at a very
   high-level, similar to our CPU visualizer.
 * Introduce "Instruments" (better naming solicited) which are
   a "high-level" task to perform. They would register SpSource-based
   data collectors with the SpProfiler as necessary.
 * Introduce memory allocation profiling to track what call-stacks
   are allocating/freeing.

Other Ideas
-----------

 * __cyg_proflie_func_enter() and __cyg_profile_func_exit() based
   tracing profile, which requires the inferior is compiled with
   -finstrument-functions.
 * gprof-style profiling and source code annotations
 * Better symbol directory loading for analyzing captures on
   workstations other than the capture device. (Embedded for example).
 * Network transfer data source


Memory Visualizer
-----------------

This is fairly straight-forward and we already have some contributions
to get it started. Simply track /proc/meminfo and /proc/$pid/statm and
register counters in the capture file.

We also need a new SpMemoryVisualizerRow to add to the visualizer list.

The only real open question here is how we should handle monitoring
multiple processes (presumably one-row per inferior or a single row
for system-wide profiling).


Introduce Instruments (or Tasks?)
---------------------------------

We run in two phases, basically. The data collection phase, and the
analysis phase. It would be nice to have a simpler way to bridge
together what belongs in each of those phases.

For example, a "leak detector" instrument would need to add a given
SpSource as well as a visualization component in the analysis phase.

When opening previously recorded captures, we'll need to discover if an
instrument was used. If so, they add their necessary visualizers.


Memory Allocation Profiling
---------------------------

To me, the advantage of Sysprof has always been the 3-section callgraph.
It's very easy to navigate and find what functions are taking time and
how you got there. I'd like to be able to extend this to mallocs.

These days, I think we have pretty good leak detectors. You have
valgrind for slow but good heap detection, or ASAN for something faster
but requiring instrumentation.

What I'm missing from my workflow with Sysprof/Builder is a quick way
to locate where allocations are and how much of the total they are.
I want to quickly determine where to spend my time with memory
allocation optimizations.

In particular, I think it would be handy to see:

 - Allocations in self (and cumulative descendants)
 - Frees in self (and cumulative descendants)
 - Temporary allocations

I have a quick prorotype I put together today. It's based on what
our sister program memprof did at the turn of the century. It is an
LD_PRELOAD with symbol overrides for malloc and friends. it use
libunwind's unw_backtrace() instead of backtrace() because it's much
faster, and seems to be generally useable from a speed perspective.

However, it generates many MB of capture data per second, so we need
to come up with an alternate strategy for how to get the data into the
capture files. We might be able to get clever with hashing the backtrace
to reduce the sample size.