RFC-0021: Trace Processor report Subcommand

[github-actions[bot]]

📄 RFC Doc: 0021-trace-processor-report-subcommand.md

---

Trace Processor report Subcommand

Authors: @lalitm

Status: Draft

Note: This document is a thought experiment exploring a possible future
direction for trace_processor_shell. It is NOT a proposal for immediate
implementation. The goal is to capture the design space and solicit feedback.

Motivation

Perfetto traces are rich, multi-dimensional datasets. Today, extracting a
useful summary requires either:

1. Loading the trace in the UI and clicking around.
2. Writing ad-hoc SQL queries against trace_processor.
3. Authoring TraceSummarySpec textprotos for the summarize subcommand.

None of these serve the "I just collected a trace, what's in it?" use case
well. Users coming from perf report expect to point a tool at a data file
and immediately see an opinionated, useful summary: no query authoring, no
spec files, no UI.

This gap is especially felt by:

• CLI power users who want quick triage without leaving the terminal.
• AI tools that need structured trace summaries to reason about
  performance.
• CI pipelines that want a human-readable (or machine-parseable) trace
  summary as a build artifact, and to compare traces across builds.

Decision

Pending

Design

Relationship to summarize

The two subcommands split by who decides what is computed:

• summarize is for summaries the user asks for themselves: they
  author TraceSummarySpec protos defining exactly what to compute.
• report is opinionated summaries we give the user: built-in views
  over the common trace dimensions, with no authoring required.

Under the hood both are PerfettoSQL. summarize converts query results to
protos as directed by the user's spec, with typed schemas and semantics
such as higher/lower-is-better; report is a presentation layer over the
PerfettoSQL standard library, where each view is computed by curated
stdlib queries and rendered from a hand-written proto schema
(report.proto, one message per view). Two things follow:

• Typed output from a fixed proto. summarize derives its output
  schema from the user's spec because the computation is arbitrary;
  report's views are a closed, compile-time set, so a hand-written proto
  gives stable text/JSON output directly.
• Report improvements land in the stdlib. Every gap a report exposes is
  fixed in a module all SQL users benefit from, and the queries shown in
  the SQL drill-down hints are the same ones the tool runs.

CLI surface

tp report [<noun> [<view>]] [FLAGS] (<trace_file> | --remote <addr>)

The first positional is always a noun (what data you are looking at); the
second is always a view (how to see it), from a closed per-noun set with a
default:

Noun	Views (default first)
(none)	overview
processes	table
tracks	table
slices	table, histogram, timeline, inspect
stack-samples	functions, top-down, bottom-up
heap-profile	functions, top-down, bottom-up
heap-dump	types
scheduling	summary, timeline, inspect
counters	list, timeline, histogram, inspect

tp report trace.pftrace                          # overview
tp report stack-samples top-down trace.pftrace
tp report slices histogram --name doFrame --remote calm-blue-otter
tp report scheduling inspect --thread RenderThread --ts 2.734s trace.pftrace
tp report slices --baseline base.pftrace target.pftrace   # diff two traces

With no noun, the overview covers every noun that has data in the trace,
skipping the rest. processes, tracks, and counters list exist for
structural discovery: they enumerate the names that the scoping flags and
the other views take as input, so a user can orient in an unfamiliar trace
before drilling down. Output shape is statically known per (noun, view).

The nouns are deliberately generic trace primitives, applicable to any
trace regardless of origin. Domain-specific reporting (e.g. Android frame
timeline, ANRs, battery) is out of scope for now; it may be added in the
future.

Output format flags

--format text|json     Output format (default: text).

• text: human-readable tables, similar to perf report --stdio.
• json: one stable schema per (noun, view), generated from
  report.proto, with the scope context embedded and richer stats than
  text (e.g. p50/p95/p99 where text shows min/avg/max). Text output is
  explicitly not a stable interface; JSON is.

--show-sql             Print the stdlib queries the view executed.

--show-sql is what makes the SQL escape hatch concrete: the printed
queries are the ones the view ran, and a starting point for going beyond
what report offers.

Scoping flags

These filter the report to a subset of the trace data:

--pid <pid>            Scope to a specific process ID.
--process <name>       Scope to a process by name.
--tid <tid>            Scope to a specific thread ID.
--thread <name>        Scope to a thread by name.
--track <name>         Scope to a track by name.
--cpu <cpu>            Scope to a specific CPU.
--time <start>,<end>   Scope to a time range. Accepts raw nanoseconds or
                       human-friendly format (e.g. 2.7s,3.1s).
--ts <ts>              Scope to a point in time (same formats as --time).
                       Used by the inspect views.

Scoping flags are translated into structured filters on the underlying
stdlib queries. Every invocation carries its full scope in flags; there is
no sticky state, so commands are reproducible, CI-safe, and safe for
concurrent clients of one warm session.

Filtering and aggregation control

--name <glob>          Row filter, same meaning in every view. In
                       bottom-up, rows are roots, so the callers of
                       function X are: stack-samples bottom-up --name X.
                       Single-series views (histogram, timeline, inspect
                       on slices/counters) require the glob to match
                       exactly one distinct name; if it matches several,
                       the command fails and lists them. Many instances
                       of one name are fine (see inspect).
--min-duration <dur>   Hide rows below a duration threshold.
--min-count <N>        Hide rows with fewer instances.
--sort <key>           Sort key, enumerated per view (e.g. dur|count|max).
--top <N>              Entries per table section (default: 10).
--max-lines <N>        Node budget for tree views (default: 100).
--buckets <N>          Bucket count for timeline/histogram views.
--group-by <key>       Grouping key where a view supports one
                       (scheduling summary: thread (default) | process).

Warm sessions

--remote <addr>        Run against a warm session (see RFC 0031);
                       mutually exclusive with a trace path.

RFC 0031's --remote subcommand list grows report. The interactive
drill-down loop (overview -> view -> inspect) is a rapid sequence of
invocations against one trace, the pattern warm sessions exist to serve.

Overview output

When invoked without a noun, the overview produces a one-line trace context
followed by per-noun highlights, ending with drill-down hints. The overview
accepts the scoping flags and --top (applied per section); --name and
--sort are rejected, since the sections have disjoint name spaces and
sort keys.

Example (--format text):

[trace.pftrace | full trace | 12.3s | Android 14 | Pixel 7 Pro | 12 processes]

Slices (12.3M total):
  Name                          Count     Total dur   % of trace   Max dur
  Choreographer#doFrame         83.2k     4.1s        33.2%        128ms @ 2.734s (RenderThread)
  DrawFrame                     83.1k     3.5s        28.4%        96ms @ 5.101s (RenderThread)
  measure                       41.6k     890ms       7.2%         42ms @ 8.913s (main)
  ...
  * 210 more slice names below --top 10; rerun with --top 50

Stack Samples (3.2k total):
  Function                                  Self%     Total%    Samples
  art::Thread::RunRootClock                 18.2%     42.1%     583
  __epoll_pwait                             12.1%     12.1%     387
  ...

Scheduling:
  Thread                        CPU time   Runnable   Sleeping   % of trace
  RenderThread                  3.2s       120ms      8.9s       26.0%
  mali-cmar-backe               1.8s       45ms       10.4s      14.6%
  ...

Heap Profile: (not present in trace)
Heap Dump: (not present in trace)

Next: tp report processes trace.pftrace
      tp report slices trace.pftrace --top 50
      tp report slices inspect --name "Choreographer#doFrame" --ts 2.734s --thread RenderThread trace.pftrace

View semantics

Tree views

The tree views (top-down, bottom-up) are never dumped in full. Nodes
are included best-first under the --max-lines budget, scored by total%
with a depth decay (internal, not user-configurable). Single-child chains
are collapsed to the same indent, and pruned subtrees are summarized:

└─ ... (5 more children: combined 12.3%, max 4.1%)

functions and the tree views show both self and total, with self
displayed more prominently.

timeline and histogram

timeline buckets the trace (or the --time range) and renders one line
per bucket: an ASCII bar, the value, and the dominant entries in that
bucket. scheduling timeline needs no selection (CPU load per bucket,
plus the threads that dominate each bucket); slices timeline and
counters timeline take a --name and show
count and total duration, or average value, per bucket.

histogram renders the distribution of a single series: durations of a
slice name, or values of a counter. The output labels its scale (linear or
logarithmic) so bucket widths are not misread; JSON carries percentile
stats alongside the buckets.

inspect

inspect drills from a noun's aggregate into instances. The noun's normal
filters select the entity; scoping flags narrow which instances match. The
output shape is fixed per noun: aggregate stats over the matched instances
plus the instances themselves, rendered in full detail when exactly one
matches. --name must resolve to one distinct name (usually copy-pasted
from report output); many instances of that name are not an error: the
output is a wider instance table plus a hint on how to narrow.

• slices inspect --name <n>: instance count, duration percentiles, top
  instances by duration (each carrying its ts/thread for narrowing). A
  unique match shows dur, self dur, args, parent chain, and children.
• counters inspect --name <n>: value at --ts, plus min/avg/max and
  delta over the scoped range.
• scheduling inspect --thread <t> --ts <ts>: state, CPU, end state,
  waker thread, and wakeup latency: "why was this thread not running at T".

Adding an inspect view to a noun must pass a two-part test: (1) instances
are addressable by name or coordinates, never tool-emitted ids; (2) it
answers an instance-level question the aggregate views do not.

Coverage gap: stack-samples, heap-profile, and heap-dump have no
inspect view yet. A per-source function inspect (callers/callees of one
function) is tractable under this grammar, since the noun fixes the data
source, but is left as future work; heap-graph objects remain id-only.
Until then, the tree views plus --name and the SQL hints cover these.

Diffing

Comparing two traces of the same workload (before/after a change, across
builds in CI) is an explicit goal. Any table view, and the overview,
accepts:

--baseline <trace|addr>   Compare against a baseline trace. Accepts a path
                          or a warm-session address (as with --remote).

Diff is a presentation-layer join: the view's queries run independently on
each trace, and the two typed results are joined on the view's aggregation
key (slice name, function, thread, process, track, or counter name) in the
shell. Neither engine ever sees both traces; multi-trace queries are out
of scope, both because they are very hard to reason about and because
sessions are single-trace (RFC 0031). Because the two runs are
independent, target and baseline may each be a path or a warm-session
address, in any combination.

Semantics:

• Value columns become Baseline / Target / Delta with signed formatting.
  Default sort is the duration delta; --sort selects others (e.g. count
  delta). Rows present on only one side are marked added/removed and rank
  by absolute delta.
• Deltas are raw values. The context header shows both trace durations and
  a footnote warns when they differ materially; report does not attempt
  to normalize for workload size and makes no claim of statistical
  significance.
• The join is by exact name. Canonicalizing names that embed dynamic
  values (tids, addresses) is an open question.
• Scoping flags apply to both traces identically.

[target.pftrace 10.1s | baseline base.pftrace 12.3s | full trace]

Slices (vs base.pftrace):
  Name                     Baseline dur   Target dur   Delta
  Choreographer#doFrame    4.1s           5.0s         +900ms
  inflate                  95ms           2ms          -93ms
  BitmapDecode             (absent)       310ms        +310ms
  * baseline is 22% longer than target; deltas are not normalized

tp report --baseline base.pftrace target.pftrace diffs the overview: a
per-noun digest of the biggest movements, the intended CI build artifact.
Tree and single-series views do not accept --baseline initially;
structural tree diffs need their own design (see open questions). A flag
is easier to miss than a subcommand, so --baseline is included in the
report help text's example block (as above).

Per-view column definitions

Slices table

Aggregation key: slice name.

Column	Description
Name	Slice name
Count	Number of instances
Total dur	Sum of wall durations across all instances
% of trace	Total duration as percentage of trace duration
Max dur	Maximum single-instance duration, with the instance's
	ts/thread (outlier detection, feeds the inspect hint)

The noun covers sync and async slice tracks; instants (zero duration)
contribute to Count only. Durations sum per instance without overlap
deduplication, so % of trace can exceed 100 on async-heavy traces.

stack-samples functions

Column	Description
Function	Function name (symbol)
Self%	Samples where this function is at the top of the stack
Total%	Samples where this function appears anywhere in stack
Samples	Absolute sample count

heap-profile functions

Column	Description
Function	Allocation site / function
Self bytes	Bytes allocated directly by this function
Total bytes	Bytes allocated by this function and its callees
Count	Number of allocations
Avg size	Average allocation size

heap-dump types

Column	Description
Type	Type or allocator
Retained size	Total retained memory
Live objects	Count of live objects

Retained size needs a dominator-tree computation and the Total%/Total
bytes columns need callstack ancestry walks; these are the most expensive
views to compute, and a further reason report benefits from warm
sessions.

scheduling summary

Column	Description
Name	Thread (or process, with --group-by process)
CPU time	Total time spent running on a CPU
Runnable	Total time in runnable state (waiting for CPU)
Sleeping	Total time sleeping
% of trace	CPU time as percentage of trace duration

processes / tracks / counters list

Discovery tables: Process, PID, CPU time, % of trace, top threads; Track,
Type, Event count, Process/Thread; Counter name, Track/Process, Sample
count, Min, Avg, Max.

Output contract

Every view emits:

1. A context header echoing the effective scope, derived purely from
   the invocation (see the overview example).
2. Truncation footnotes: what was hidden and the flag to reveal it.
3. Next-command hints: complete, copy-pasteable drill-down invocations
   (including --remote and scope flags), including tp query commands
   for raw data. The intended progression is: overview -> aggregate view
   -> inspect -> SQL.
4. Terminal-aware tables: columns are width-fitted when stdout is a
   TTY; piped or redirected output is untruncated.
5. Errors fail fast: an unknown noun/view combination, a missing
   required flag (e.g. --ts for scheduling inspect), or a malformed
   value exits non-zero with a message listing the valid options; with
   --format json the error is a structured object.

Testing

Text output is locked by trace processor diff tests: one golden per
(noun, view) over reference traces, so any output change is explicit in
review and documented behavior cannot drift from shipped behavior. JSON is
validated against report.proto by construction.

Alternatives considered

Build on the summarize machinery

Pro:

• Reuses an existing declarative engine; specs shareable with summarize.

Con:

• Tree views have no representation in the flat-row summary output.
• Scoping flags rewrite the computation per invocation, so the
  "pre-authored spec" is really a template rewritten per invocation.
• Typed output does not require the engine: a fixed proto provides it.

Alternative grammars

Two other grammars were drafted: a --view flag on a noun positional, and
view-first positionals with data-selection flags (top-down --heap-profile). Both rejected: noun-then-view matches how users approach
a trace (pick the data, then the presentation) and how comparable
heterogeneous profiling tools have converged, keeps each view a closed
per-noun set, and has in-repo precedent (tp server http).

Sticky server-side session state

Holding analysis state (a selected thread, zoom/filter stacks) in a
long-lived server makes interactive drill-down terse, but single
invocations become non-reproducible and concurrent clients unsafe. Warm
sessions (RFC 0031) already give the real benefit, parse-once performance,
so report keeps all state in flags.

Id-based inspect addressing

inspect slice:187432 style handles: rejected. Ids are only knowable if
the tool emitted them and are unstable across trace loads and versions;
names and coordinates (thread, ts) are legible from any output. Id-precision
lookups remain available via the SQL bridge.

A --focus <function> stack filter

A pprof-style focus transform (keep stacks containing X, re-root per view):
rejected as a second drill-down mechanism with subtle semantics (recursion
double-counting, per-view re-rooting). bottom-up --name covers the
callers workflow; re-rooting is left as an open question.

A separate diff subcommand

tp report diff <base> <target> was considered. Rejected: it would need to
replicate every noun/view under itself, whereas --baseline composes with
the existing views and the overview for free, and inherits scoping flags.

Open questions

• Domain-specific reports: Android frame timeline, ANRs, battery and
  similar have stdlib modules but are out of scope for now; they may be
  added in the future.
• Inspect for the callstack/heap nouns: per-source function inspect
  (callers/callees) and heap-graph object drill-down (see coverage gap).
• Tree diffs: delta-annotated call trees under --baseline; the
  rendering and matching rules need their own design.
• Per-trace time windows in diffs: --time applies to both traces;
  comparing different windows (e.g. startup vs startup at different
  offsets) may need a --baseline-time.
• Diff join canonicalization: names embedding dynamic values (tids,
  addresses) break the exact-name join into spurious added/removed pairs.
• Diff noise: accepting multiple baseline traces (min/median/max) to
  separate real movement from run-to-run variance.
• --root <function> on tree views: pure re-rooting for "what does X
  call", if hop-wise navigation proves insufficient.
• A heap-dump retainers view: retaining paths for a type.
• Cross-sectional view: "what was happening at time T" across all
  nouns (the CLI analog of a vertical line in the UI).
• JSON schema discovery: how consumers find the per-view schemas
  (help section vs a dedicated subcommand).

---

💬 Discussion Guidelines:

• This discussion is automatically synced with the RFC document
• Please provide constructive feedback and suggestions

[github-actions[bot]]

📝 RFC Document Updated

View changes: Commit History