Module rustc_data_structures::profiling
source · Expand description
Rust Compiler Self-Profiling
This module implements the basic framework for the compiler’s self-
profiling support. It provides the SelfProfiler
type which enables
recording “events”. An event is something that starts and ends at a given
point in time and has an ID and a kind attached to it. This allows for
tracing the compiler’s activity.
Internally this module uses the custom tailored measureme crate for
efficiently recording events to disk in a compact format that can be
post-processed and analyzed by the suite of tools in the measureme
project. The highest priority for the tracing framework is on incurring as
little overhead as possible.
Event Overview
Events have a few properties:
- The
event_kind
designates the broad category of an event (e.g. does it correspond to the execution of a query provider or to loading something from the incr. comp. on-disk cache, etc). - The
event_id
designates the query invocation or function call it corresponds to, possibly including the query key or function arguments. - Each event stores the ID of the thread it was recorded on.
- The timestamp stores beginning and end of the event, or the single point in time it occurred at for “instant” events.
Event Filtering
Event generation can be filtered by event kind. Recording all possible
events generates a lot of data, much of which is not needed for most kinds
of analysis. So, in order to keep overhead as low as possible for a given
use case, the SelfProfiler
will only record the kinds of events that
pass the filter specified as a command line argument to the compiler.
event_id
Assignment
As far as measureme
is concerned, event_id
s are just strings. However,
it would incur too much overhead to generate and persist each event_id
string at the point where the event is recorded. In order to make this more
efficient measureme
has two features:
-
Strings can share their content, so that re-occurring parts don’t have to be copied over and over again. One allocates a string in
measureme
and gets back aStringId
. ThisStringId
is then used to refer to that string.measureme
strings are actually DAGs of string components so that arbitrary sharing of substrings can be done efficiently. This is useful becauseevent_id
s contain lots of redundant text like query names or def-path components. -
StringId
s can be “virtual” which means that the client picks a numeric ID according to some application-specific scheme and can later make that ID be mapped to an actual string. This is used to cheaply generateevent_id
s while the events actually occur, causing little timing distortion, and then later map thoseStringId
s, in bulk, to actualevent_id
strings. This way the largest part of the tracing overhead is localized to one contiguous chunk of time.
How are these event_id
s generated in the compiler? For things that occur
infrequently (e.g. “generic activities”), we just allocate the string the
first time it is used and then keep the StringId
in a hash table. This
is implemented in SelfProfiler::get_or_alloc_cached_string()
.
For queries it gets more interesting: First we need a unique numeric ID for
each query invocation (the QueryInvocationId
). This ID is used as the
virtual StringId
we use as event_id
for a given event. This ID has to
be available both when the query is executed and later, together with the
query key, when we allocate the actual event_id
strings in bulk.
We could make the compiler generate and keep track of such an ID for each
query invocation but luckily we already have something that fits all the
the requirements: the query’s DepNodeIndex
. So we use the numeric value
of the DepNodeIndex
as event_id
when recording the event and then,
just before the query context is dropped, we walk the entire query cache
(which stores the DepNodeIndex
along with the query key for each
invocation) and allocate the corresponding strings together with a mapping
for DepNodeIndex as StringId
.
Structs
- A helper for recording costly arguments to self-profiling events. Used with
SelfProfilerRef::generic_activity_with_arg_recorder
. - An
EventId
is aStringId
with the additional guarantee that the corresponding string conforms to the event_id grammar. - Something that uniquely identifies a query invocation.
- A reference to the SelfProfiler. It can be cloned and sent across thread boundaries at will.
Enums
- Which format to use for
-Z time-passes