tools/perf: M2-first CPU profiling rig + runner --perf-jitdump

Step 0 of the `neither` cherry-pick funnel: a repeatable, attributable, wallclock-anchored CPU bottleneck baseline for our own ARM64 port, since the RK3562-era numbers are stale and the target has shifted to Snapdragon 865. Built and validated on M2 Max / Asahi first; the same scripts run on SD865 with a new devices/<label>.env and no code change. Test-harness change (no production code path, no shipped env gate): - pcsx2-eerunner --perf-jitdump: emit a Linux perf jitdump under EmuFolders::Cache so `perf inject --jit` resolves EE_/VU0_/VU1_/IOP_/VIF_ block symbols. The enable is driven through the EmuCore/Profiler EnablePerfDump config bool (ApplySettings -> LoadSettings re-applies Perf::SetJitDumpEnabled every apply, so a manual enable would be reset). SetJitDumpDir is set before the first block compiles. Also honors an explicit `--renderer null` under --perf-jitdump for the CPU-only diagnostic. - pcsx2-vurunner --perf-jitdump: same flag for VU-only captures (manual enable; the runner doesn't go through ApplySettings). Tooling (pure stdlib + bash): - tools/perf/bucket_perf.py: parse a `perf report --stdio` dump into a PCSX2 subsystem ranking. Buckets tuned against real M2 R&C UYA / Katamari captures: JIT-by-prefix (EE/VU0/VU1/IOP/VIF) + VU-glue (mVUlookupProg / dispatch envelope) + native VIF + GS (incl. GIF decode + XXH3) + a GPU-driver bucket that quarantines the host Asahi/Vulkan/DRM stack (host-specific, NOT an SD865 proxy) + JIT-other (unsymbolized continuation blocks) + startup/io + a visible `unattributed`. - tools/perf/profile_run.sh: one-command wrapper (precondition gate -> perf record -> inject --jit -> single report dump -> bucket -> median wallclock + median per-bucket share -> summary.md). Device/scene parameterized. - tools/perf/devices/m2max-asahi.env: the working M2 incantation (P-core PMU apple_avalanche_pmu/cycles/, -F 999). - tools/perf/scenes/*.env: R&C UYA + Katamari cinematic/gameplay scenes (assets are copyrighted, not checked in; paths reflect this dev box). Validated on M2: go/no-go #1 (perf+jitdump resolves 277 JIT symbols) and go/no-go #2 (per-bucket shares stable across 3 runs) both green. First finding: on M2 the `vk` profile is dominated by the Asahi GPU stack (GS thread ~57% of samples) — the CPU shape comes from `--renderer null`, where VU (bodies+glue) is the largest emulation cost. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-24 16:50:16 -07:00 · 2026-06-24 12:40:31 -07:00
parent f016d603d2
commit a187fdc0b6
12 changed files with 592 additions and 1 deletions
@@ -43,6 +43,7 @@
 #include "common/FileSystem.h"
 #include "common/MemorySettingsInterface.h"
 #include "common/Path.h"
+#include "common/Perf.h"
 #include "common/ProgressCallback.h"
 #include "common/StringUtil.h"

@@ -100,6 +101,7 @@ static bool s_no_console = false;
 static bool s_contmem_vu0_interp = false; // --vu0-interp modifier for --contmem
 static GSRendererType s_renderer = GSRendererType::Null; // --renderer (Null default; vk for Intel/headless)
 static std::string s_memdump_prefix; // --memdump <prefix>: write <prefix>.{interp,jit}.bin at the last frame
+static bool s_perf_jitdump = false; // --perf-jitdump: emit Linux perf jitdump for `perf inject --jit` (profiling)

 bool EERunner::InitializeConfig()
 {
@@ -497,6 +499,9 @@ static void PrintCommandLineHelp(const char* progname)
 	std::fprintf(stderr, "  --savestate <file>: Savestate to load after Initialize (required).\n");
 	std::fprintf(stderr, "  --frames N: Number of frames to run (default 300).\n");
 	std::fprintf(stderr, "  --iso <file>: Game ISO/disc to mount (required so the savestate has its disc).\n");
+	std::fprintf(stderr, "  --perf-jitdump: Emit a Linux perf jitdump (under EmuFolders::Cache) so `perf inject --jit`\n");
+	std::fprintf(stderr, "               resolves EE_/VU0_/VU1_/IOP_/VIF_ JIT block symbols. Profiling only; with --liverun it\n");
+	std::fprintf(stderr, "               also honors an explicit --renderer null. Requires a USE_PERF_JITDUMP build.\n");
 	std::fprintf(stderr, "  -help: Displays this information and exits.\n");
 	std::fprintf(stderr, "  -version: Displays version information and exits.\n");
 	std::fprintf(stderr, "\n");
@@ -570,6 +575,15 @@ bool EERunner::ParseCommandLineArgs(int argc, char* argv[], VMBootParameters& pa
 				s_mode = RunMode::LiveRun;
 				continue;
 			}
+			else if (CHECK_ARG("--perf-jitdump"))
+			{
+				// Emit a Linux perf jitdump so `perf inject --jit` can resolve EE_/VU1_/...
+				// JIT block symbols. Profiling only; honors explicit --renderer null (the
+				// liverun null->VK force below is skipped when this is set). Requires a
+				// USE_PERF_JITDUMP build (no-op otherwise).
+				s_perf_jitdump = true;
+				continue;
+			}
 			else if (CHECK_ARG("--disasm"))
 			{
 				// --disasm: load the savestate, then disassemble EE code in
@@ -700,7 +714,11 @@ void EERunner::SettingsOverride()
 	// like Null) and MTVU. Null GS is meaningless for it, so force VK if unset.
 	const bool live = (s_mode == RunMode::LiveRun);
 	GSRendererType rend = s_renderer;
-	if (live && rend == GSRendererType::Null)
+	// Liverun normally needs a real GS (Null drops GIF/PATH3), so Null is forced to VK.
+	// When profiling (--perf-jitdump), honor an explicit --renderer null so the
+	// "scalar EE/IOP minus GS-feeding" diagnostic baseline is reachable. vk stays the
+	// representative whole-system profile; null is the secondary diagnostic.
+	if (live && rend == GSRendererType::Null && !s_perf_jitdump)
 		rend = GSRendererType::VK;
 	s_settings_interface.SetIntValue("EmuCore/GS", "Renderer", static_cast<int>(rend));

@@ -778,6 +796,11 @@ void EERunner::SettingsOverride()
 	s_settings_interface.SetStringValue("SPU2/Output", "Backend", "Null");
 	s_settings_interface.SetStringValue("SPU2/Output", "SyncMode", "Disabled");

+	// Profiling: drive the perf jitdump enable through the normal config path so
+	// ApplySettings/LoadSettings (which re-applies Perf::SetJitDumpEnabled from this
+	// bool) keeps it on for the whole run instead of resetting it to the default.
+	s_settings_interface.SetBoolValue("EmuCore/Profiler", "EnablePerfDump", s_perf_jitdump);
+
 	// No frameskip.
 	s_settings_interface.SetBoolValue("EmuCore/GS", "FrameSkipEnable", false);
 	s_settings_interface.SetIntValue("EmuCore/GS", "FramesToDraw", 1);
@@ -2889,6 +2912,17 @@ static void CPUThreadMain(VMBootParameters* params, std::atomic<int>* ret)

 	if (VMManager::Internal::CPUThreadInitialize())
 	{
+		// Profiling: set the jitdump output dir before any JIT block compiles (the
+		// first compile happens during the first FrameAdvance, well after this). Dir =
+		// EmuFolders::Cache so the 100s-of-MB dump avoids /tmp/tmpfs, matching the
+		// production rationale in common/Perf.cpp. The ENABLE flag is driven through the
+		// normal settings path instead (EmuCore/Profiler EnablePerfDump, set in the
+		// harness config) — ApplySettings() below calls LoadSettings() which re-applies
+		// Perf::SetJitDumpEnabled(EnablePerfDump), so a manual enable here would just get
+		// reset to the config default (false). No-op on non-jitdump builds.
+		if (s_perf_jitdump)
+			Perf::SetJitDumpDir(EmuFolders::Cache);
+
 		// apply new settings (e.g. pick up renderer change)
 		VMManager::ApplySettings();

@@ -33,6 +33,7 @@

 #include "DebugTools/Debug.h"
 #include "common/FPControl.h"
+#include "common/Perf.h"
 #include "common/PmuCounters.h"

 #include <algorithm>
@@ -63,6 +64,7 @@ struct Options
 	bool bench_no_reprime = false;
 	bool print_bases = false;
 	bool no_progcache = false;  // determinism gate: force program cache + recording off
+	bool perf_jitdump = false;  // emit Linux perf jitdump for `perf inject --jit` (profiling)
 	u32 dump_count = 64;
 	u32 cycle_override = 0;  // 0 = use captured budget
 	int vu_clamp_mode = -1;  // -1 = leave EmuConfig default (mode 1); 0..3 = force VU clamp mode
@@ -173,6 +175,10 @@ bool ParseArgs(int argc, char** argv, Options& opts)
 		{
 			opts.no_progcache = true;
 		}
+		else if (a == "--perf-jitdump")
+		{
+			opts.perf_jitdump = true;
+		}
 		else if (a == "--cache-dir")
 		{
 			if (i + 1 >= argc)
@@ -1188,6 +1194,14 @@ int main(int argc, char** argv)
 		return 1;
 	}

+	// Profiling (--perf-jitdump): enable the perf jitdump writer as early as possible
+	// — before ANY VU block compiles — so `perf inject --jit` resolves VU0_/VU1_
+	// symbols. Dir defaults to /tmp (EmuFolders::Cache isn't populated this early in
+	// the harness; fine for the tiny per-program dumps vurunner emits). No-op on
+	// non-USE_PERF_JITDUMP builds.
+	if (opts.perf_jitdump)
+		Perf::SetJitDumpEnabled(true);
+
 #if defined(_M_ARM64) || defined(__aarch64__)
 	if (opts.no_progcache)
 		mVUPersist::SetProcessDisable(true);
@@ -0,0 +1,2 @@
+__pycache__/
+*.pyc
@@ -0,0 +1,49 @@
+# tools/perf — PCSX2 ARM64 CPU profiling rig
+
+Step 0 of the `neither` cherry-pick funnel (`/home/bmd/pcsx2/neither/CLAUDE.md`): get a
+**current, repeatable, attributable** bottleneck baseline for our own port. The old
+RK3562 numbers are stale and from the wrong device; we re-profile on **M2 Max / Asahi**
+first, then SD865.
+
+## Pieces
+
+| File | Role |
+|---|---|
+| `bucket_perf.py` | parse a `perf report --stdio` dump → subsystem ranking (EE-JIT/VU0/VU1/IOP/VIF-JIT + GS/SPU2/vtlb/dispatcher/sync/kernel) + thread-comm axis; `--json` for aggregation. Pure stdlib. |
+| `profile_run.sh` | one-command wrapper: precondition gate → `perf record`/`inject --jit`/`report` → bucket → median wallclock + median shares → `summary.md`. Device-parameterized. |
+| `devices/<label>.env` | per-device `BIN_DIR`/`FREQ_DEFAULT`/`CYCLES_EVENT` (`m2max-asahi.env` shipped). |
+| `scenes/<id>.env` | per-scene `ISO`/`SAVESTATE`/`FRAMES`/`RENDERERS`/`LABEL` (4 scenes; you fill ISO+savestate). |
+
+## Drivers
+
+- **eerunner** (whole-system): `pcsx2-eerunner --liverun` runs EE-JIT + MTVU + IOP + VIF + real GS headless for a fixed frame count from a savestate. perf-recorded, JIT-symbolized via the new `--perf-jitdump` flag, bucketed. **`vk` is the representative profile; `null` is a secondary "scalar EE/IOP minus GS-feeding" diagnostic** (Null drops GIF/PATH3). Audio is excluded (SPU2 forced Null) — noted in every summary.
+- **gsrunner** (GS-only cross-check): `pcsx2-gsrunner -perf` replays a `.gs` dump deterministically and prints `@HWSTAT@` frame time + CPU/GS/GPU thread %. No perf/bucketing.
+- VU isolation: `pcsx2-vurunner --bench <cap.vucap>` (deterministic PMU cycles) — run directly; cross-checks the `VU1-JIT` bucket.
+
+## One-time setup (M2)
+
+1. Build: `cmake --preset clang-perf && cmake --build build-perf --target pcsx2-eerunner pcsx2-gsrunner pcsx2-vurunner`
+2. **Allow perf sampling:** `sudo sysctl kernel.perf_event_paranoid=1` (currently 2; the wrapper aborts with this hint otherwise).
+3. Provide assets (copyrighted, not checked in): the 2 ISOs + 4 savestates referenced in `scenes/*.env`. Capture each savestate in pcsx2-qt paused exactly at the scene start.
+
+## Run
+
+```bash
+# whole-system, both renderers, 3 runs, median
+tools/perf/profile_run.sh --device m2max-asahi --scene uya-gameplay --renderer both --runs 3
+# → ~/pcsx2-profiles/m2max-asahi/uya-gameplay/{vk,null}/summary.md
+
+# GS-only cross-check
+tools/perf/profile_run.sh --device m2max-asahi --driver gsrunner --gs-dump test-dumps/<dump>.gs
+```
+
+## Go/no-go checklist
+
+- **#1 (perf works on M2):** a tiny liverun under `perf record -k mono` + `perf inject --jit` + `perf report` must show real `EE_*`/`VU1_*` symbols, **not** raw `[JIT]` addresses. Resolve the Apple-PMU unknowns here (cycles event binding across the two PMUs `apple_avalanche_pmu`/`apple_blizzard_pmu`; max `-F`; `-k mono` honored). Record the working `CYCLES_EVENT` in `devices/m2max-asahi.env`.
+- **#2 (share stability):** top-bucket median-share MAD across 3 runs is small (a few %) → MTVU nondeterminism doesn't break the ranking.
+- **#3 (before SD865):** all 4 scenes yield a stable, attributable ranking (`unattributed` < ~5%) with a median wallclock. Then add `devices/sd865.env` + re-captured savestates; no code change.
+
+## Notes
+
+- The new `--perf-jitdump` flag lives in `pcsx2-eerunner/Main.cpp` (test harness only; no production gate). jitdump path: `~/.config/PCSX2/cache/pcsx2-perf-<pid>/jit-<pid>.dump`.
+- Deferred: audio profiling (opt-in real-SPU2 knob), a dedicated EE numeric microbench.
@@ -0,0 +1,229 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-3.0+
+#
+# bucket_perf.py — turn a `perf report --stdio` dump into a PCSX2 subsystem
+# bottleneck ranking. Part of the M2-first profiling rig (plan: Step 0 of the
+# neither cherry-pick funnel). Pure stdlib; runs identically on M2 Asahi and SD865.
+#
+# Input is the ALREADY-DUMPED text report (the wrapper runs `perf report` once and
+# redirects to a file — see tools/perf/profile_run.sh — because a raw 500-700 MB
+# perf.data is expensive to traverse). This script never invokes perf and never
+# holds the .data; it only parses text.
+#
+# Bucketing has two axes:
+#   1. SUBSYSTEM — JIT blocks by their Perf::Group symbol prefix
+#      (EE_/VU0_/VU1_/IOP_/VIF_, from common/Perf.cpp) + native code by symbol regex.
+#   2. THREAD comm (CPU=EE thread, MTVU, GS, Audio) — a secondary breakdown so the
+#      EE-vs-MTVU split is visible and bucket attribution can be sanity-checked
+#      (e.g. VU1-JIT should land mostly on MTVU under async MTGS).
+#
+# Anything unmatched falls into the visible `unattributed` bucket — never silently
+# dropped, so a large value flags a missing rule rather than a clean-looking lie.
+#
+# Usage:
+#   perf report -i perf.jit.data --stdio --percent-limit 0 -g none > report.txt
+#   bucket_perf.py report.txt            # human table + @BUCKET@ grep lines
+#   bucket_perf.py --json report.txt     # machine JSON (the wrapper medians these)
+#   cat report.txt | bucket_perf.py      # stdin also works
+
+import argparse
+import json
+import re
+import sys
+
+# --- Subsystem bucketing -----------------------------------------------------
+# JIT symbol prefixes emitted by Perf::Group (common/Perf.cpp). Checked first.
+JIT_PREFIX = re.compile(r"^(EE|VU0|VU1|IOP|VIF)_")
+JIT_BUCKET = {"EE": "EE-JIT", "VU0": "VU0-JIT", "VU1": "VU1-JIT",
+              "IOP": "IOP-JIT", "VIF": "VIF"}
+
+# --- GPU-driver quarantine (checked BEFORE kernel) ---------------------------
+# The host GPU stack (userspace Vulkan driver + DRM kernel + GPU memory manager).
+# On the M2/Asahi box this dominates the GS thread (libvulkan_asahi + [asahi] Rust
+# driver + drm_mm allocator: ~38% kernel + ~2% userspace at go/no-go #1). It is
+# entirely HOST-SPECIFIC — it tells you nothing about SD865 (Adreno) CPU cost, so it
+# gets its own visible bucket instead of polluting kernel/other and unattributed.
+# Matches both the userspace driver dso AND the explicit DRM/GPU-MM kernel symbols;
+# generic kernel routines (memset/mutex/malloc) stay in kernel/other on purpose
+# (can't honestly attribute a generic memcpy to the GPU by symbol alone — the
+# --renderer null run is the clean way to drop the whole GS/GPU path).
+GPU_DRIVER_DSO = re.compile(r"asahi|libvulkan|libVkLayer|_mesa|libdrm|radeonsi|"
+                            r"libGLX|libEGL|nvidia|anv_|\btu\b|panfrost|libmali")
+GPU_DRIVER_SYM = re.compile(r"^drm_|drm_mm|^add_hole|^rm_hole|^hk_|_mesa_|^vk_|"
+                            r"gpu_|RunVertex|RunFragment|FwCtlChannel|HeapAllocator")
+
+# Native code, ordered — first match wins. Tuned against real demangled symbol names
+# from R&C UYA captures (2026-06-24). Many hot natives are under the `isa_native::`
+# MultiISA-dispatch namespace, so anchors match substrings, not leading symbols.
+# Kept broad on purpose — re-tighten if a bucket starts swallowing unrelated symbols.
+NATIVE_RULES = [
+    # One-time costs (savestate/ISO decompress, shader compile, gamedb parse). Kept
+    # separate so the steady-state emulation ranking isn't inflated by startup. Note:
+    # over a bounded liverun this is a fixed cost — its share shrinks as FRAMES grows.
+    ("startup/io",      re.compile(r"shaderc|glslang|spirv|SPIRV|libzstd|ZSTD|HUF_|"
+                                   r"c4::yml|ryml|ParseEngine|GameDatabase|"
+                                   r"Decompress|inflate|lzma|LZ4")),
+    ("IPU/video",       re.compile(r"yuv2rgb|_DCT|IDCT|idct|IPU|ipu|getBits|[Mm]dec")),
+    # VU dispatch envelope (program lookup/search, dispatcher entry/exit, sync-ahead,
+    # block clear/compile) — distinct from the jitted VU bodies (VU0_/VU1_). This is
+    # the "VU cache duplication / lookup" cost the stale RK3562 notes flagged.
+    ("VU-glue",         re.compile(r"mVUlookupProg|mVUsearchProg|mVUcompile|mVUdispatch|"
+                                   r"VU0StartFunc|VU1StartFunc|vu0SyncRunAhead|"
+                                   r"vu1SyncRunAhead|recMicroVU|BaseVUmicroCPU|"
+                                   r"mVUexecute|mVUreset|mVUcleanUp|microVU.*[Dd]ispatch|"
+                                   r"vu0ExecMicro|vu1ExecMicro|vuExecMicro")),
+    # VIF (unpack dynarec front-end + native transfer/interrupt). Jitted VIF_ blocks
+    # land in the VIF bucket via JIT_PREFIX; these are the native halves.
+    ("VIF",             re.compile(r"vifTransfer|VIF0transfer|VIF1transfer|vif0Interrupt|"
+                                   r"vif1Interrupt|dVifUnpack|vifUnpack|VifUnpack|Vif_|"
+                                   r"VIFunpack|vifCode|dVifsetVUptr|vifExecQueue|"
+                                   r"_VIF[01]chain|VIF[01]chain")),
+    ("GS",              re.compile(r"GSXXH|XXH_INLINE|XXH3|GSLocalMemory|GSRenderer|GSState|"
+                                   r"GSDevice|GSDraw|GSRasteriz|GSVertex|GSTextureCache|"
+                                   r"GSClut|GSGet|GSLookup|GSVector|GSBlock|GSClip|::GS|"
+                                   r"GS[A-Z][a-z]|Gif_Unit|Gif_|GIFTag|GIFPath|GIFPackedReg")),
+    ("SPU2/audio",      re.compile(r"[Ss][Pp][Uu]2|SndOut|[Ss]oundtouch|cubeb|TimeStretch|"
+                                   r"ReverbDo|V_Volume|V_Core|VolumeSlide|V_ADSR|ADSR")),
+    ("vtlb/mem",        re.compile(r"vtlb|[Mm]em[RW]rite|[Mm]em[Rr]ead|GetMemPtr|iopMem|"
+                                   r"eeMem|recMemory|RecMemcheck|GoemonUnloadTlb")),
+    ("EE/IOP-glue",     re.compile(r"cpuEventTest|iopEventTest|CPU_INT|recClear|"
+                                   r"Arm64BaseBlocks|ExecuteBlock|"
+                                   r"psxBranchTest|intcInterrupt|dmacInterrupt|hwIntc|"
+                                   r"hwDmac|cpuException|psxException|eeloadHook|_cpuTest|"
+                                   r"psxRcnt|psxCounter|rcntUpdate|EEcnt|hwRead|hwWrite|"
+                                   r"dmaExec|dmacWrite|dmacRead|dmaGetAddr|eeHw|DMAVerbose|"
+                                   r"_dmaGIF|_dmaVIF|sif[01]|EEsif")),
+    ("dispatcher/glue", re.compile(r"Dispatcher|recExecute|recRecompile|iopRecRecompile|"
+                                   r"JITCompile|recompileNextInstruction|recCall|dyna_|"
+                                   r"sync_cache_range|__clear_cache|FlushInstructionCache")),
+    ("memops",          re.compile(r"__memcpy|__memset|__memmove|__pi_mem|memcpy_|memset_|"
+                                   r"memcpy@|memset@|memmove|crc32|"
+                                   r"_int_malloc|_int_free|\bmalloc\b|\bfree\b|cfree|"
+                                   r"malloc_consolidate|operator new|operator delete")),
+    ("sync/mtgs/mtvu",  re.compile(r"pthread_mutex|pthread_cond|futex|__lll_|"
+                                   r"condition_variable|Semaphore|WaitForBits|"
+                                   r"std::.*mutex|Threading::|sem_post|sem_wait|"
+                                   r"spin_on_owner|raw_spin|MTGS|MTVU|VU_Thread|"
+                                   r"ThreadEntryPoint|Get_MTVUChanges|mtvu|GIFPath_|"
+                                   r"ExecuteRingBuffer|ExecuteGSPacket")),
+]
+
+# Buckets we always print even at 0% (so the ranking shape is stable run-to-run).
+ALL_BUCKETS = ["EE-JIT", "VU0-JIT", "VU1-JIT", "IOP-JIT", "VIF",
+               "VU-glue", "GS", "IPU/video", "SPU2/audio", "vtlb/mem", "EE/IOP-glue",
+               "dispatcher/glue", "memops", "sync/mtgs/mtvu", "startup/io",
+               "JIT-other", "GPU-driver", "kernel/other", "unattributed"]
+
+# A leading percent column, e.g. "    41.23%". `perf report -g none` emits one
+# Overhead column; if a Children column sneaks in there are two — we take the LAST
+# leading percent as self%.
+PCT = re.compile(r"^\s*((?:\d+\.\d+%\s+)+)(.*)$")
+# The symbol-type marker splits "comm dso" from "symbol": [.] user, [k] kernel, etc.
+SYMMARK = re.compile(r"\s\[[.kguHh]\]\s")
+
+
+def classify(dso, symbol, is_kernel):
+    """Return the subsystem bucket for one report row. Order: JIT prefix → GPU-driver
+    (host-specific, before kernel so the GPU stack's kernel symbols are quarantined) →
+    generic kernel → native rules → unattributed (raw 0x addrs fall through)."""
+    m = JIT_PREFIX.match(symbol)
+    if m:
+        return JIT_BUCKET[m.group(1)]
+    # Unsymbolized JIT continuation blocks: perf inject names only the program-entry
+    # block (per a41d849f4), so sub-blocks show up as `[JIT] tid N  0x...` raw addrs.
+    # Label them JIT-other rather than letting them sink into unattributed — they ARE
+    # guest JIT execution, just unnamed (which JIT engine is unknowable from the addr).
+    if dso.startswith("[JIT]"):
+        return "JIT-other"
+    if GPU_DRIVER_DSO.search(dso) or GPU_DRIVER_SYM.search(symbol):
+        return "GPU-driver"
+    if is_kernel or dso == "[kernel.kallsyms]":
+        return "kernel/other"
+    for name, rx in NATIVE_RULES:
+        if rx.search(symbol):
+            return name
+    return "unattributed"
+
+
+def parse(lines):
+    """Parse a perf report --stdio dump -> (bucket->pct, comm->pct, total_pct)."""
+    buckets = {b: 0.0 for b in ALL_BUCKETS}
+    comms = {}
+    total = 0.0
+    for line in lines:
+        if not line.strip() or line.lstrip().startswith("#"):
+            continue
+        m = PCT.match(line)
+        if not m:
+            continue
+        # last leading percent = self%
+        pct = float(m.group(1).split("%")[-2].split()[-1])
+        rest = m.group(2)
+        sm = SYMMARK.search(rest)
+        if sm:
+            left = rest[:sm.start()]
+            symbol = rest[sm.end():].strip()
+            mark = rest[sm.start():sm.end()].strip()
+        else:
+            # no marker (rare) — treat whole remainder as "comm ... symbol"
+            left, symbol, mark = rest, rest.split()[-1] if rest.split() else "", ""
+        # Columns are separated by 2+ spaces; comm itself may contain a single
+        # space (e.g. "CPU Thread"), so split on runs of >=2 spaces, not any space.
+        parts = re.split(r"\s{2,}", left.strip())
+        comm = parts[0] if parts else "?"
+        dso = parts[-1] if len(parts) > 1 else ""
+        is_kernel = (mark == "[k]") or (dso == "[kernel.kallsyms]")
+        bucket = classify(dso, symbol, is_kernel)
+        buckets[bucket] += pct
+        comms[comm] = comms.get(comm, 0.0) + pct
+        total += pct
+    return buckets, comms, total
+
+
+def main():
+    ap = argparse.ArgumentParser(description="Bucket a perf report into PCSX2 subsystems.")
+    ap.add_argument("report", nargs="?", help="perf report --stdio dump (default stdin)")
+    ap.add_argument("--json", action="store_true", help="emit JSON instead of a table")
+    args = ap.parse_args()
+
+    src = open(args.report) if args.report else sys.stdin
+    with src:
+        buckets, comms, total = parse(src)
+
+    # Normalize to the parsed total so shares sum to 100% regardless of perf quirks
+    # (e.g. a multi-PMU recording would otherwise sum to ~200%). Shares, not absolute
+    # percentages, are the comparison currency (per the methodology rule).
+    if total > 0:
+        buckets = {k: v / total * 100.0 for k, v in buckets.items()}
+        comms = {k: v / total * 100.0 for k, v in comms.items()}
+
+    ranked = sorted(buckets.items(), key=lambda kv: kv[1], reverse=True)
+    comm_ranked = sorted(comms.items(), key=lambda kv: kv[1], reverse=True)
+
+    if args.json:
+        json.dump({"total_pct": round(total, 2),
+                   "buckets": {k: round(v, 3) for k, v in ranked},
+                   "comms": {k: round(v, 3) for k, v in comm_ranked}},
+                  sys.stdout, indent=2)
+        sys.stdout.write("\n")
+        return
+
+    print(f"# subsystem ranking (self%, total accounted = {total:.1f}%)")
+    print(f"{'BUCKET':<18}{'SELF%':>8}")
+    for name, pct in ranked:
+        print(f"{name:<18}{pct:>8.2f}")
+    print()
+    print(f"# by thread comm")
+    for name, pct in comm_ranked:
+        print(f"{name:<18}{pct:>8.2f}")
+    print()
+    # grep-friendly one-liners for the wrapper / quick scraping.
+    for name, pct in ranked:
+        print(f"@BUCKET@ {name} {pct:.2f}")
+    if buckets["unattributed"] > 5.0:
+        print(f"# WARNING: unattributed {buckets['unattributed']:.1f}% > 5% — "
+              f"add/Tune a NATIVE_RULES regex.", file=sys.stderr)
+
+
+if __name__ == "__main__":
+    main()
@@ -0,0 +1,23 @@
+# Device profile: Apple M2 Max on Asahi Linux (Fedora Asahi, kernel 6.19).
+# Sourced by tools/perf/profile_run.sh ($REPO is defined before sourcing).
+#
+# This is the convenient first profiling target (full perf access). The SD865
+# handheld gets its own devices/sd865.env later — same scripts, no code change.
+
+# Where the clang-perf build put the runner binaries.
+BIN_DIR="$REPO/build-perf/bin"
+
+# perf sampling frequency (Hz). 999 ~ avoids 60 Hz vblank harmonics. Drop to 499
+# if `perf record` warns about throttling on the Apple PMU (verify at go/no-go #1).
+FREQ_DEFAULT=999
+
+# Cycle event. M2 exposes TWO PMUs (apple_avalanche_pmu = P-core, apple_blizzard_pmu
+# = E-core). Plain "cycles" expands to BOTH → two event sections in the report that
+# sum to ~200%. PCSX2 pins its hot threads to P-cores (it logs "enabling thread
+# pinning"), so the P-core PMU captures the vast majority of emulation samples (go/no-go
+# #1 2026-06-24: 2K P-core vs 76 E-core samples). Pin to the P-core PMU for a single,
+# clean event section. VERIFIED WORKING 2026-06-24 (277 JIT symbols resolved).
+CYCLES_EVENT="apple_avalanche_pmu/cycles/"
+
+# Optional: root dir for ISOs, referenced as $ISO_ROOT in scenes/*.env.
+# ISO_ROOT="$HOME/games/ps2"
@@ -0,0 +1,192 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-3.0+
+#
+# profile_run.sh — one-command, repeatable, attributed CPU profile of the PCSX2
+# ARM64 port. Step 0 of the neither cherry-pick funnel: a CURRENT bottleneck
+# baseline. Built/validated on M2 Asahi first; the SAME script runs on SD865 with
+# a different devices/<label>.env (no code change).
+#
+# Two drivers:
+#   eerunner  — whole-system (EE-JIT + MTVU + IOP + VIF + real GS), perf-recorded,
+#               JIT-symbolized via jitdump, bucketed into a subsystem ranking.
+#   gsrunner  — GS-only deterministic @HWSTAT@ frame-time + thread-% cross-check
+#               (no perf/bucketing; gsrunner replays GIF packets, runs no EE/VU JIT).
+#
+# Methodology (CLAUDE.md): wallclock is the throughput truth, FPS is noisy; >=2 runs,
+# report the median. Bucket *shares* are more stable than absolute cycles across the
+# MTVU-nondeterministic liverun, so we median per-bucket share across runs.
+#
+# Usage:
+#   tools/perf/profile_run.sh --device m2max-asahi --scene uya-gameplay --renderer both --runs 3
+#   tools/perf/profile_run.sh --device m2max-asahi --driver gsrunner --gs-dump test-dumps/foo.gs.zst
+#
+set -euo pipefail
+
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO="$(cd "$HERE/../.." && pwd)"
+
+DRIVER=eerunner
+SCENE=""
+DEVICE=""
+RENDERER=""           # vk | null | both ; default from scene RENDERERS, else vk
+RUNS=3
+OUT="$HOME/pcsx2-profiles"
+FREQ=""               # overrides device FREQ
+GS_DUMP=""            # gsrunner driver
+LOOP=3                # gsrunner loop count
+
+die() { echo "error: $*" >&2; exit 1; }
+
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --driver)   DRIVER="$2"; shift 2;;
+    --scene)    SCENE="$2"; shift 2;;
+    --device)   DEVICE="$2"; shift 2;;
+    --renderer) RENDERER="$2"; shift 2;;
+    --runs)     RUNS="$2"; shift 2;;
+    --out)      OUT="$2"; shift 2;;
+    --freq)     FREQ="$2"; shift 2;;
+    --gs-dump)  GS_DUMP="$2"; shift 2;;
+    --loop)     LOOP="$2"; shift 2;;
+    -h|--help)  grep '^#' "$0" | sed 's/^# \{0,1\}//'; exit 0;;
+    *) die "unknown arg: $1";;
+  esac
+done
+
+[[ -n "$DEVICE" ]] || die "--device <label> required (see tools/perf/devices/)"
+DEVENV="$HERE/devices/$DEVICE.env"
+[[ -f "$DEVENV" ]] || die "no device profile: $DEVENV"
+# shellcheck disable=SC1090
+source "$DEVENV"   # provides BIN_DIR, FREQ (default), CYCLES_EVENT, optional ISO_ROOT
+: "${BIN_DIR:?devices/$DEVICE.env must set BIN_DIR}"
+: "${CYCLES_EVENT:=cycles}"
+[[ -n "$FREQ" ]] || FREQ="${FREQ_DEFAULT:-999}"
+
+# --- precondition: perf sampling must be permitted -----------------------------
+if [[ "$DRIVER" == "eerunner" ]]; then
+  PARANOID="$(cat /proc/sys/kernel/perf_event_paranoid 2>/dev/null || echo 99)"
+  if [[ "$PARANOID" -gt 1 ]]; then
+    die "perf_event_paranoid=$PARANOID (need <=1). Fix:  sudo sysctl kernel.perf_event_paranoid=1"
+  fi
+  command -v perf >/dev/null || die "perf not found"
+fi
+
+ts() { date +%s.%N; }
+
+###############################################################################
+# gsrunner driver — deterministic GS-only @HWSTAT@ cross-check (no perf record)
+###############################################################################
+if [[ "$DRIVER" == "gsrunner" ]]; then
+  [[ -n "$GS_DUMP" ]] || die "--gs-dump <dump.gs[.zst]> required for gsrunner"
+  GBIN="$BIN_DIR/pcsx2-gsrunner"
+  [[ -x "$GBIN" ]] || die "missing $GBIN (build it: cmake --build build-perf --target pcsx2-gsrunner)"
+  RUNDIR="$OUT/$DEVICE/gs-$(basename "$GS_DUMP" | tr './' '__')"
+  mkdir -p "$RUNDIR"
+  echo "== gsrunner -perf $GS_DUMP (loop=$LOOP) =="
+  "$GBIN" -surfaceless -perf -loop "$LOOP" "$GS_DUMP" 2>&1 | tee "$RUNDIR/stdout.txt" | grep '@HWSTAT@' || true
+  echo "artifacts: $RUNDIR/stdout.txt"
+  exit 0
+fi
+
+###############################################################################
+# eerunner driver — whole-system perf profile
+###############################################################################
+[[ -n "$SCENE" ]] || die "--scene <id> required for eerunner (see tools/perf/scenes/)"
+SCENEENV="$HERE/scenes/$SCENE.env"
+[[ -f "$SCENEENV" ]] || die "no scene profile: $SCENEENV"
+# shellcheck disable=SC1090
+source "$SCENEENV"  # provides ISO, SAVESTATE, FRAMES, RENDERERS, LABEL
+: "${ISO:?scenes/$SCENE.env must set ISO}"
+: "${SAVESTATE:?scenes/$SCENE.env must set SAVESTATE}"
+: "${FRAMES:=300}"
+[[ -n "$RENDERER" ]] || RENDERER="${RENDERERS:-vk}"
+[[ "$RENDERER" == "both" ]] && RENDERER="vk null"
+
+EBIN="$BIN_DIR/pcsx2-eerunner"
+[[ -x "$EBIN" ]] || die "missing $EBIN (build: cmake --build build-perf --target pcsx2-eerunner)"
+# ( ... || true ) so pipefail sees grep's status, not eerunner --help's exit 1
+# (eerunner treats --help as an unknown arg: prints usage, then exits non-zero).
+( "$EBIN" --help 2>&1 || true ) | grep -q -- '--perf-jitdump' \
+  || die "$EBIN has no --perf-jitdump — rebuild from the patched source (clang-perf)."
+[[ -f "$ISO" ]]       || die "ISO not found: $ISO"
+[[ -f "$SAVESTATE" ]] || die "savestate not found: $SAVESTATE"
+
+# jitdump dir is EmuFolders::Cache = ~/.config/PCSX2/cache (set by eerunner).
+JITDIR="${PCSX2_CACHE:-$HOME/.config/PCSX2/cache}"
+
+for REND in $RENDERER; do
+  SDIR="$OUT/$DEVICE/$SCENE/$REND"
+  mkdir -p "$SDIR"
+  echo "== $LABEL [$SCENE] renderer=$REND frames=$FRAMES runs=$RUNS =="
+  for k in $(seq 1 "$RUNS"); do
+    RUNDIR="$SDIR/run$k"; mkdir -p "$RUNDIR"
+    rm -rf "$JITDIR"/pcsx2-perf-* 2>/dev/null || true
+    echo "-- run $k/$RUNS --"
+    t0="$(ts)"
+    # Production-representative knobs: async MTGS, MTVU on, EE jit. (eerunner test
+    # harness env vars — not shipped gates.) SPU2 stays Null (audio excluded by design).
+    # NOTE: do NOT use --per-thread — on the Apple M2 PMU it follows only the
+    # blocked main thread and captures ~0 samples. Default inherit mode samples all
+    # worker threads (CPU/MTVU/GS) and still records comm/tid per sample, so the
+    # by-comm axis survives. --call-graph omitted (this perf rejects =no; flat is what
+    # we want anyway). Verified working on m2max-asahi 2026-06-24.
+    EERUNNER_SYNCMTGS=0 EERUNNER_MTVU=1 EERUNNER_EE=jit \
+      perf record -e "$CYCLES_EVENT" -F "$FREQ" -k mono \
+        -o "$RUNDIR/perf.data" -- \
+        "$EBIN" --liverun --renderer "$REND" --frames "$FRAMES" \
+                --savestate "$SAVESTATE" --iso "$ISO" --perf-jitdump \
+        >"$RUNDIR/stdout.txt" 2>&1 || true
+    t1="$(ts)"
+    echo "$(awk "BEGIN{printf \"%.3f\", $t1-$t0}")" > "$RUNDIR/wallclock.txt"
+
+    # Pull in the jitdump this PID produced so JIT symbols resolve.
+    JD="$(ls -t "$JITDIR"/pcsx2-perf-*/jit-*.dump 2>/dev/null | head -1 || true)"
+    if [[ -n "$JD" ]]; then cp "$JD" "$RUNDIR/jit.dump"; fi
+    ( cd "$RUNDIR" && perf inject --jit -i perf.data -o perf.jit.data 2>/dev/null ) || \
+      cp "$RUNDIR/perf.data" "$RUNDIR/perf.jit.data"
+    perf report -i "$RUNDIR/perf.jit.data" --stdio --percent-limit 0 -g none \
+      > "$RUNDIR/report.txt" 2>/dev/null || true
+    python3 "$HERE/bucket_perf.py" --json "$RUNDIR/report.txt" > "$RUNDIR/buckets.json" || true
+    # Progress line: report samples + the top bucket from buckets.json (NOT a grep of
+    # report.txt — the @BUCKET@ markers only exist in bucket_perf.py's table mode).
+    SAMP="samples=$(awk '/^# Samples:/{print $3; exit}' "$RUNDIR/report.txt" 2>/dev/null)"
+    TOP="$(python3 -c "import json;b=json.load(open('$RUNDIR/buckets.json'))['buckets'];k=max(b,key=b.get);print(f'{k} {b[k]:.0f}%')" 2>/dev/null || echo '?')"
+    echo "   wallclock=$(cat "$RUNDIR/wallclock.txt")s  $SAMP  top=$TOP"
+  done
+
+  # Median wallclock + median per-bucket share across the K runs -> summary.md.
+  python3 - "$SDIR" "$REND" "$SCENE" "$LABEL" "$DEVICE" "$FREQ" "$FRAMES" "$RUNS" "$CYCLES_EVENT" <<'PY'
+import json, glob, os, statistics, sys
+sdir, rend, scene, label, device, freq, frames, runs, event = sys.argv[1:10]
+bj = sorted(glob.glob(os.path.join(sdir, "run*", "buckets.json")))
+wc = []
+for d in sorted(glob.glob(os.path.join(sdir, "run*"))):
+    p = os.path.join(d, "wallclock.txt")
+    if os.path.exists(p):
+        try: wc.append(float(open(p).read().strip()))
+        except ValueError: pass
+agg = {}
+for f in bj:
+    try: data = json.load(open(f))
+    except Exception: continue
+    for k, v in data.get("buckets", {}).items():
+        agg.setdefault(k, []).append(v)
+med = {k: statistics.median(v) for k, v in agg.items() if v}
+ranked = sorted(med.items(), key=lambda kv: kv[1], reverse=True)
+med_wc = statistics.median(wc) if wc else float("nan")
+out = os.path.join(sdir, "summary.md")
+with open(out, "w") as o:
+    o.write(f"# {label} — {scene} (renderer={rend})\n\n")
+    o.write(f"- device: **{device}**  · runs: {runs}  · frames: {frames}  · perf -F {freq} -e {event}\n")
+    o.write(f"- median wallclock (incl. startup+savestate-load): **{med_wc:.3f} s**\n")
+    o.write(f"- ⚠️ audio OUTPUT excluded (eerunner forces SPU2 Backend=Null); the SPU2 DSP core still runs and is counted.\n")
+    o.write(f"- renderer note: `vk` = whole-system (on M2 the GPU-driver/kernel buckets are the host Asahi stack, NOT an SD865 proxy); `null` = CPU-emulation shape with the GS thread dropped (closest to the SD865-relevant CPU cost, but GIF/PATH3 under-consumed so VU1/XGKICK are distorted).\n\n")
+    o.write("| bucket | median self% |\n|---|---|\n")
+    for k, v in ranked:
+        o.write(f"| {k} | {v:.2f} |\n")
+    if med.get("unattributed", 0) > 5:
+        o.write(f"\n⚠️ unattributed {med['unattributed']:.1f}% > 5% — tune bucket_perf.py NATIVE_RULES.\n")
+print(f"wrote {out}")
+PY
+done
+echo "done. summaries under $OUT/$DEVICE/$SCENE/"
@@ -0,0 +1,9 @@
+# Scene: Katamari Damacy — intro/menu (light; isolates EE/IOP overhead).
+# NOTE: no dedicated cinematic save captured yet — points at the only Katamari
+# savestate on this box (slot 01). TODO: capture a distinct intro/menu save.
+LABEL="Katamari — cinematic"
+ISO_ROOT="${ISO_ROOT:-/home/bmd/dev/ps2/My Sony PlayStation 2 (USA) Collection}"
+ISO="$ISO_ROOT/Katamari Damacy (USA).iso"
+SAVESTATE="$HOME/.config/PCSX2/sstates/SLUS-21008 (FA7E3081).01.p2s"
+FRAMES=600
+RENDERERS="vk null"
@@ -0,0 +1,11 @@
+# Scene: Katamari Damacy — dense katamari (max object count; EE object-update +
+# VU transform stress). Capture late in a level with a large ball / many props.
+# Assets are copyrighted (not checked in); these paths are this dev box's real layout.
+LABEL="Katamari — worst gameplay"
+ISO_ROOT="${ISO_ROOT:-/home/bmd/dev/ps2/My Sony PlayStation 2 (USA) Collection}"
+ISO="$ISO_ROOT/Katamari Damacy (USA).iso"
+# slot 01 = the only Katamari save on this box (fresh 2026-06-23). TODO: confirm it's a
+# dense-objects scene, or capture a worst-case one.
+SAVESTATE="$HOME/.config/PCSX2/sstates/SLUS-21008 (FA7E3081).01.p2s"
+FRAMES=600
+RENDERERS="vk null"
@@ -0,0 +1,11 @@
+# Scene: Ratchet & Clank: Up Your Arsenal — in-engine cinematic / FMV (clean baseline,
+# isolates EE/IOP/IPU overhead away from VU1/VIF combat stress).
+# Assets are copyrighted (not checked in); these paths are this dev box's real layout.
+LABEL="R&C UYA — cinematic"
+ISO_ROOT="${ISO_ROOT:-/home/bmd/dev/ps2/My Sony PlayStation 2 (USA) Collection}"
+ISO="$ISO_ROOT/Ratchet & Clank - Up Your Arsenal (USA) (En,Fr,Es).iso"
+# slot 01 = mid-FMV (validated go/no-go #1 scene). TODO: re-capture a true in-engine
+# cinematic if you want one distinct from the FMV.
+SAVESTATE="$HOME/.config/PCSX2/sstates/SCUS-97353 (45FE0CC4).01.p2s"
+FRAMES=600           # same N every run
+RENDERERS="vk null"  # vk = representative; null = scalar-EE diagnostic
@@ -0,0 +1,10 @@
+# Scene: Ratchet & Clank: Up Your Arsenal — worst-case combat (VU1/VIF + GS stress).
+# Assets are copyrighted (not checked in); these paths are this dev box's real layout.
+LABEL="R&C UYA — worst gameplay"
+ISO_ROOT="${ISO_ROOT:-/home/bmd/dev/ps2/My Sony PlayStation 2 (USA) Collection}"
+ISO="$ISO_ROOT/Ratchet & Clank - Up Your Arsenal (USA) (En,Fr,Es).iso"
+# slot 02 = largest in-game state (best-guess busy scene). TODO: confirm this is the
+# worst-case combat save, or point at the slot you captured for it.
+SAVESTATE="$HOME/.config/PCSX2/sstates/SCUS-97353 (45FE0CC4).02.p2s"
+FRAMES=600           # same N every run
+RENDERERS="vk null"
@@ -0,0 +1,7 @@
+# Temporary wrapper-validation scene (slot-01 savestate; happens to be an FMV).
+# NOT a curated benchmark scene — replace with proper cinematic/gameplay saves.
+LABEL="R&C UYA — wrapper validation (FMV)"
+ISO="/home/bmd/dev/ps2/My Sony PlayStation 2 (USA) Collection/Ratchet & Clank - Up Your Arsenal (USA) (En,Fr,Es).iso"
+SAVESTATE="/home/bmd/.config/PCSX2/sstates/SCUS-97353 (45FE0CC4).01.p2s"
+FRAMES=600
+RENDERERS="vk"