vtlb_Miss raised the exception under the arm64 recompiler and returned,
on the assumption that the rec would pick cpuRegs.pc up at the next
dispatch. Nothing picks it up: the block runs on and its tail writes its
own branch target over the vector PC, while cpuException has already
latched Status.EXL. cpuException leaves EPC alone whenever EXL is
already set, so from the first swallowed miss onward every exception
keeps its predecessor's EPC, and the guest kernel's syscall epilogue
erets to an address belonging to the fault.
Report the miss and continue instead, as the x86 rec already does. The
guest's handler still does not run, so a title that needs demand paging
still cannot work under the rec; a stray miss now stays local to the
instruction. ee_rec_tlb_divert_tests.cpp describes the divert and stays
disabled.
With nothing left to set s_recTlbMissOccurred, the poll after every
interpreter call goes too. It could not have covered the raise anyway:
only the interpreter-call sites had it, so a flag set at an inline
access sat there until an unrelated later recCall consumed it and
diverted on a pc belonging to neither.
This gives up the one case the rec did handle, a miss in a branch delay
slot reaching the vector through the cpuRegs.branch bracket epilogue.
Its recompiler half moves to the disabled file; the interpreter half
stays live, beside a new test pinning the rec's behaviour. The bracket
itself stays: the tlb_fallback_* handlers raise from a delay slot on
their own.
eeFprSyncSlotFormat keys on CHECK_FPU_FULL, so the relocated file is
what both of iFPUd's rungs compute over, not mode 3 alone. The comments
that name a mode where they mean the tier are renamed to the tier.
iFPUd widened every operand with ToDoubleFrom: eleven instructions
branching on exponent field 0xff, since a PS2 single with that exponent
is an ordinary large number IEEE reads as Inf or NaN. Against a
relocated slot the widening is one Fmul, and a compare goes from 34.75
host instructions to 8.75. CVT.W loses its NaN fix-up the same way,
leaving Fcvtzs.
Which format the file is in follows the clamp mode, so modes 0-2 keep
the architectural word in the low half. FPRreg's accessors read the
format and the emitters read the mode; the two engines meet at the
boundaries the word is still observable at -- MFC1, MTC1, MOV.S, the two
conversions, LWC1, SWC1 and the allocator's fill and spill.
The unscale factor takes q11 and d10's mask moves to the relocated
layout, so the allocator's callee-saved range is q12-q15.
The slot is 64 bits and the architectural register is 32, so a word view
of the union is only right while the two coincide. Word() and SetWord()
replace f/UL/SL and are the identity, so nothing that runs changes; what
it buys is that the compiler names every place that reads a slot as a
word, which is the set the next commit has to relocate.
The x86 tier is not built here and still reads the members directly.
Both MMIO load paths — the const-paddr shortcut and the backpatch stub — widen
an unsigned sub-64-bit handler return with Uxtw(x0, w0), mirroring the
Sxtb/Sxth/Sxtw right above them. But the sign-extending three lower to sbfm,
which genuinely accepts a narrow source, and Uxtw lowers to ubfm, which does
not: UBFM has no W-source/X-destination form at all. Its 64-bit encoding is
UBFM Xd,Xn,#0,#31 and the width is carried by the destination alone. vixl
asserts the two operands match, and that assert is live in every Debug build
and compiled out of Release.
The emitted word is the same either way — the operand field holds only the
register number and sf comes from Rd — so this assembles to exactly the
UBFM X0,X0,#0,#31 that was meant. Release has always been correct, which is why
no nightly ever showed it. An assert-enabled build instead aborts on the CPU
thread the first time a game takes an unsigned load through an MMIO/handler
page, which is seconds into a boot: it makes Debug unrunnable rather than
wrong.
Pass x0 on both sides.
Eleven commented-out defines in iR5900-arm64.h, each selecting the interpreter
for one opcode category (branch, jump, move, shift, ALU, arith-imm, mult/div,
memory, COP0, FPU, COP2), consumed by twelve #ifdef/#else/#endif pairs across
eleven files. Using one meant editing the header and rebuilding.
pcsx2-eerunner --rec-fallback <groups> does the same bisect at runtime with no
rebuild and no source edit, over a full VM boot, so it stays game-faithful.
The transform keeps every #else body byte for byte - the diff is deletions
only, no added or reindented lines. Also drops two comments that referenced
the defines.
recompiler_tests: 1513 passed, 2 skipped, 0 failed.
The const-paddr MMIO shortcut flushed cpuRegs.pc on every access. Only the
two unmapped-page handlers can raise a guest exception -- vtlb_Miss ->
cpuTlbMiss -> cpuException derives EPC from that pc -- so for every
registered hardware handler the per-access mov/movk/str was dead weight,
paid once per access in an MMIO burst. x86 never flushes pc at its MMIO
sites at all.
Add vtlb_IsUnmappedHandlerID() and flush pc only for those two handlers,
plus the forceEventTest path: EE-counter reads end the block with the
g_branch=2 event-test exit, and FLUSH_EVERYTHING (0x1ff) excludes FLUSH_PC
(0x200), so this seam's store is the only thing that gives DispatcherReg a
resume pc. ConstCounterReadEventTestExitResumesAtNextPc pins that carve-out;
EeRecTraps.LoadTlbMissInDelaySlotSetsCauseBdAndBranchEpc pins the retained
unmapped-handler flush.
The decision reads handler identity at compile time to settle a runtime
property, but introduces no new assumption: the shortcut already resolves
its callee from the same compile-time mapping and hard-codes a direct BL to
it, so a mapping change that could falsify the pc decision would already be
sending the call to the wrong handler. Noted at the site, since that
equivalence is the whole basis for the trim being safe.
Full suite 1423/1423.
Taken from https://github.com/yaps2/yaps2/pull/10 by pstef
(https://github.com/pstef).
Co-authored-by: pstef <3462925+pstef@users.noreply.github.com>
The vtlb load/store seams flushed const-tracked GPRs with delete_const=true,
so the first vtlb seam in a block destroyed every const mark. An MMIO burst
-- N accesses off one lui/ori const address -- then degraded accesses 2..N
from the const-paddr shortcut (a direct BL to the registered handler) to the
dynamic fastmem/softmem path. The DBZ3 GS-priv block at 0x0029474c (5x sd to
0x120000xx) emitted one BL plus four dynamic stores where x86 emits five BLs.
Add FLUSH_CONST_KEEP as a modifier on FLUSH_CONSTANT_REGS: write dirty const
values back to memory, but keep the tracking marks. FLUSH_VTLB pairs the two
and replaces FLUSH_CONSTANT_REGS at every vtlb seam.
The marks are valid to keep because the callee set at these seams -- the
registered MMIO handlers and vtlb_memRead/Write -- cannot write guest GPRs.
x86 relies on the same invariant more aggressively: its equivalent sites use
FLUSH_FULLVTLB (0x000) and flush nothing at all, values included. We keep the
value writeback because the softmem paths re-read guest GPRs from cpuRegs
memory post-flush and vtlb exception paths expect memory to be current; that
writeback is what the Tier-2G bisect (f4601964b) found load-bearing, and it
is preserved here. Interpreter seams must NOT set the modifier -- the
interpreter does write cpuRegs.GPR, so its marks go stale.
This makes several previously-dead paths live for the first time, since
recLQ/recSQ/recLQC2/recSQC2 and the post-flush GPR_IS_CONST1 folds in
recLoad/recStore never saw a live const mark before. _allocGPRtoNEONreg
already handles both directions (MODE_READ materializes a const as a 128-bit
memory load plus an Ins of the constant into lane 0; MODE_WRITE deletes the
mark), which the new ConstQuadStoreMaterializesAndAliasLoadKillsConst fixture
pins along with the rs==rt alias case.
Note the const-MMIO shortcut sites move from FLUSH_INTERPRETER to
FLUSH_VTLB|FLUSH_PC, which drops FLUSH_FREE_XMM and so lets the GE-15
retention keep FPREG/FPACC slots mapped in q10-q15 across the handler call.
That is safe here -- MMIO handlers do not RMW cpuRegs.fpr -- and makes the
shortcut consistent with the softmem seams. Dropping FLUSH_ALL_X86 changes
nothing: iFlushCall frees all caller-saved GPRs unconditionally.
FLUSH_FREE_VU0 is inert on arm64; only x86's iFlushCall consumes it.
ConstAddrMmioBurstStaysOnShortcut is red before this change (accesses 2..N
land in the faulting-PC set) and green after. Full suite 1423/1423.
Taken from https://github.com/yaps2/yaps2/pull/10 by pstef
(https://github.com/pstef).
Co-authored-by: pstef <3462925+pstef@users.noreply.github.com>
The hand-rolled COP2 macro FMAC clamps reloaded the ±FLT_MAX bounds from
_cpuRegistersPack.cop2Rec at EVERY clamp site — up to 6 q-loads per guest
op, 8398 static instructions across the UYA EE corpus and 9-21% of every
hot COP2 block (5.1% of EErec exec-weighted instructions on the M2
sample profile). x86 never shows this class because SSE folds the operand
(minps xmm, [mem]); AetherSX2-arm64 keeps the bounds register-resident.
Dedicate q25 = maxFloat.4S / q26 = minFloat.4S, excluded from the EE NEON
allocator pool (like q8/q9) and from the COP2 macro-mode mVU pool
(microRegAlloc::reset(cop2mode), the NEON twin of the x26/x27 EE-pin
gate). Re-materialization is 2 Dups from the pinned callee-saved s8/s9
scalars — no memory access (minFloat[i] == maxFloat[i] | 0x80000000 ==
-FLT_MAX exactly). The clamp itself is now a bare Fminnm+Fmaxnm.
Compile-time validity discipline (s_cop2ClampConstsValid):
- false at block start; first clamp site emits the 2 Dups;
- invalidated by iFlushCall (any flushtype — every real C-call seam);
- NOT invalidated by the VPU_STAT sync seams: the shared sync stubs
re-materialize unconditionally on their taken path (always sound —
q25/q26 can hold nothing else; the fast path touches no NEON);
- NOT invalidated by fastmem: vtlbGetLiveRegisterMasks ORs q25/q26 into
the recorded fpr_bitmask while valid, so a backpatched slowmem thunk
preserves them like any live register;
- NOT invalidated by the mVU-reuse macro wrappers (pool-gated, no C
calls);
- forks and superblock side exits carry the flag via BranchCompileState
and Cop2VfCacheScope.
M2 census (fresh 6000f UYA slot-02 liverun A/B, identical 7644-block
set): clamp-const loads 8398 -> 0, replaced by 498 lazy establishments;
EE corpus -7,446 insns (-1.83%) / -29.1 KiB; hot-90 set bytes -6.0%
sample-weighted; COP2-dense physics blocks -15..-19% (0043C718 0.806,
0043A840 0.853). Flagship 003F7690 unchanged as expected (its density
residual is GPR-quad seam round-trips, the SL-14 item). Unlike the
reverted S4-4 outlining (insns-for-bytes trade, A77-neutral), this
removes instructions AND bytes together on serial clamp->FMAC dependency
chains.
Gates: 1401/1401 recompiler_tests incl. 8 new EeVu0Cop2ClampResidency
contract tests (establishment counts, taken-seam clamp correctness, fork
sharing, sync-stub re-Dup byte-scan, EE+mVU pool exclusion probes);
8000-seed EE fuzz green; UYA slot-02 --stepdiff signature identical to
the known-benign baseline (same timer skips, same 0x004c295c terminal).
Co-Authored-By: Claude <noreply@anthropic.com>
recLQC2/recSQC2 still paid iFlushCall(FLUSH_CONSTANT_REGS) on the
fastmem path — the legacy q0-detach shape — which frees every
caller-saved GPR/NEON allocator entry mid-block. UYA's hottest COP2
staging block (0x3F7690) pays it four times in a row. Rework both to
the GE-14 recLQ/recSQ shape: address before the const flush (keeps
the const-Rs fold), _flushConstRegs(true) only, and the quad staged
through RQSCRATCH (q30, never allocator-tracked) so no q0 detach is
needed. The softmem/faulting-PC fallback keeps the legacy full-flush
+ q0 shape.
VF memory is current at both sites without any flush: LQC2/SQC2 are
VF-cache-classifier-false, so recompileNextInstruction flushed the
COP2 VF compile cache before the emitter runs, and VF regs are never
EE-allocator-tracked. The fault path is the proven GE-07 live-mask
thunk (handles non-q0 data registers, saves live allocator entries).
Measured (M2 census, UYA 6000f, vs the S4-2 capture): static EE
bytes -0.16%, 146 blocks shrink / zero grow, EE_003F7690 -104B;
exec-weighted ldr_gpr_unpinned 5.04% -> 4.13% (the reload churn the
seam caused downstream), neon q ld/st -0.3pt combined, cop2_vf flat
by design. A compile-time VF-cache event trace + policy resimulation
sized the ledger's original "VF residency across macro chains" idea
at ~0.07% of cycles ceiling (flagship block: zero) — the seam, not
the residency policy, was the prize.
Tests: three new LQC2/SQC2 residency contracts (dirty caller-saved
scalars + const-folded base, macro-result-then-SQC2 ordering, dirty
resident MMI quad across an LQC2/SQC2 pair). Gates: 1390 tests, 8k
fuzz seeds, vucorpus bins catalog-exact, UYA stepdiff signature
identical, SotC stepdiff baseline-identical (its .01 signature
drifted since the S4-2 session but pre/post agree byte-for-byte at
block 0x006f7168 — pre-existing class, still uncharacterized).
Co-Authored-By: Claude <noreply@anthropic.com>
Add "yaps2 Dev Team" copyright to the files we authored. Net-new files
(all 42 pcsx2/arm64/ codegen/ProgCache/persist sources, the recompiler
test suite + harness, and the vurunner/eerunner tools) never existed
upstream, so they carry yaps2 sole credit. RecStubs.cpp predates the
fork and was heavily extended, so it keeps PCSX2 credit and adds yaps2.
The five pre-existing arm64 files we only lightly touched (AsmHelpers,
Vif_Dynarec, Vif_UnpackNEON) stay PCSX2-only. GPL-3.0+ license lines are
unchanged throughout; this is authorship attribution only.
The LWL/LWR, LDL/LDR, and fused-LDL/LDR handlers allocated a scalar
ARM64TYPE_GPR home for Rt unconditionally — a GE-M2 I1 violation when Rt
is pinned. Pre-residency-flip templates retired the slot at the next ALU
consumer (whose pin-aware writeback reconciled the mirror), so the dual
state was transient; under the resident-ALU templates the slot persists
and pin-preferring readers (_eeGetGPRSourceReg) keep serving the
PRE-load mirror value until the next seam — silent wrong values from any
ulw/uld into a pinned reg, the R&C UYA weapons/hit-detection corruption
and matching soft-locks.
Route the merge through eeUnalignedRtWriteHome(): a pinned Rt merges in
the pin itself (a live 128-bit home or pending const is reconciled into
the pin first; under lazy-dirty writing the pin IS the guest write),
unpinned Rt keeps the READ|WRITE allocator slot. Enforce I1 at the
creation site in _allocArm64GPR so the next offender aborts in its own
backtrace instead of corrupting silently.
Found by the new EE JIT-vs-interp fuzzer (seed 6); regression tests
cover LDR/fused-LDL/LDR/LWL/LWR into pinned regs with an in-block ALU
consume — the consume is what makes the stale window observable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
recLQC2's VF write-back and recSQC2's VF read materialized a 48-bit
absolute address (mov+movk+movk) per access; VURegs is imm12-reachable
from the pinned RVU0 base, so armVU0Mem gets the same access in one
instruction (4 -> 1). Every other VF/VI access in the COP2 emitters
already went through armVU0Mem — these two sites were the stragglers.
In the UYA SD865 emitted-code capture these two sites account for
~1500 of the 10252 absolute-address triples in EE blocks (~4.5K
instructions).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three raw scalar sites remained (the load/store paths already used
_eeMoveGPRtoR widely):
- recStoreLoadResult: deposit the load result into rt via _eeStoreGPRDestReg
(the fastmem pinned-dest path still elides to the canonical Str when result
IS the pin).
- softmem store value: read rt via _eeMoveGPRtoR (matches recPrepStoreValue),
which also handles const and rt == 0.
- SQ softmem address: read rs via _eeMoveGPRtoR. This is a real coherence fix
for the flip — the path flushes only FLUSH_CONSTANT_REGS, which does NOT free
the callee-saved x28 pool slot, so a raw memory load of a dirty resident rs
would go stale (the scalar companion to the armMergeEEResidentIntoQuad rt
quad-load merge landed in 0e).
The raw quad load stays armLoadEERegPtrRaw + armMergeEEResidentIntoQuad by
design (intentionally-stale-then-merged lower half). Behavior-preserving on the
memory-template baseline; coherence-safe under the flip. Mirrors PCSX2's x86
allocator-aware VTLB codegen (pcsx2/x86/ix86-32/iR5900-32.cpp, iCore.cpp).
Gated by recompiler_tests (1279 pass, incl. load/store + coherence net).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The raw quad-load sites (SQ / QMFC2-QMTC2 / MMI memory-path) load a 128-bit
guest GPR from cpuRegs memory and fix up a stale lower half from a dirty pin
via armMergeEEPinIntoQuad. Once scalar residency is flipped on, that lower half
can also live dirty in an ARM64TYPE_GPR slot the site never flushed (recVTLB SQ
only iFlushCall(FLUSH_CONSTANT_REGS), which leaves scalar slots resident).
Add armMergeEEResidentIntoQuad — a side-effect-free superset of the pin merge
that also Ins-es a dirty scalar slot's lower 64 into lane 0 (pin and scalar
slot are mutually exclusive by I1). Route the three emitter quad-load sites
through it. recVTLB SQ's quad load switches to armLoadEERegPtrRaw so the
intentionally-stale-then-merged load doesn't trip the I3 tripwire.
(_allocGPRtoNEONreg already displaces a dirty scalar slot on its fill path;
this closes the raw-load gap.) New SqAfterScalarWriteMergesDirtyLowerHalf test
pins the behavior; 1279/1279 recompiler_tests pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
LQ loads straight into an allocated NEONTYPE_GPRREG slot (MODE_WRITE,
canonical cpuRegs.GPR store elided under dirty tracking) and SQ stores
from the allocator-resident quad (MODE_READ - zero-emit reuse when the
value is already resident from an LQ or MMI op). This is x86
recLoadQuad/recStore(128)'s alloc-callback model; a reference ARM64 PS2
implementation does the same. The per-site iFlushCall - which evicted the
ENTIRE allocator (all NEON + caller-saved GPRs) at every LQ/SQ - is gone
from the fastmem path; only the constant flush remains (the fault thunk's
live-mask machinery, GE-07, covers register state). Softmem and the Rt==0
LQ discard keep the legacy flush + q0 shape.
The 128-bit backpatch thunk is generalized from its data_register==0
assert to any q register: the fast path Ldr/Str targets the resident
slot directly; the slow path bridges to the vtlb_memRead/Write128 q0
ABI with a Mov on the appropriate side of the call (the load dest is
excluded from the thunk's NEON save mask, so the restores cannot
clobber the landed result).
Coherence rests on the existing MMI-Rd protocol: _writebackNEONreg
routes GPRREG quads through armStoreEEGPRQuad (pin refresh at
writeback), scalar templates retire quads via _deleteEEreg before
touching pins/memory, and _eeMoveGPRtoR probes NEON residency ahead of
the pin fallback.
M2 static census, same-day paired 600-frame liveruns (blocks
19715->19715 / 10822->10840): SotC 1,223,421 -> 1,046,870 EE-block
insns (-14.4%), UYA 651,160 -> 550,613 (-15.4%) - the largest single
reduction of the GE campaign; LQ/SQ eviction storms were the dominant
static tax. Emitted bytes -14.4%/-15.4%.
Tests: 9 new coherence guards (resident-dest MMI chains, pinned-reg
lower-64 reads, dest==base aliasing, lazy-dirty pin merge on SQ, LQ/SQ
round trip, same-dest reuse, MMIO fault paths through the generalized
thunk both directions); 1221/1221. Stepdiff SotC/UYA signatures
unchanged from the P1-P3 gates (2 benign timers + pre-existing
0x00124c60).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
recLoad's pinned-dest fast path (WS-C4) emits the Ldr straight into the
pin and only afterwards retires the dest's allocator state via
recStoreLoadResult -> _deleteEEreg. When the dest also has a dirty NEON
quad (an MMI Rd), that late retirement writes the quad back and
armStoreEEGPRQuad's pin refresh clobbers the just-loaded result with the
quad's stale lane 0 - the loaded value is silently lost.
Retire the quad (writeback + free) before emitting the pin load. This
also keeps the dest slot out of the fault thunk's NEON save mask, so the
slow path can't restore stale state over the C-handler result either.
TDD: PaddwThenLwSamePinnedDestPreservesLoadedValue reproduces the
mechanism red on the unfixed tree (PADDW v0 / LW v0 - the pre-existing
reachable route); LqThenScalarLoadIntoSamePinnedDest guards the LQ
producer shape. Both green with the fix; 1212/1212.
Found while validating GE-14 (allocator-resident LQ dests make every
'lq rt ... lw rt' sequence hit this): SotC's guest main thread returned
to address 0 within the first frames of a liverun.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
recLWC1 loaded into w0 and stored to fpr[ft] memory; the next FPU op
paid a Ldr back plus the store→load-forward stall — 2 mem ops per LWC1
on the hottest float-load idiom. x86-master and a reference ARM64 PS2
implementation both pass a dest-alloc callback into the vtlb read emitter
so the load targets the allocated FPR directly.
New vtlbFastmemReadFPR32: inline LDR S<n>, [RFASTMEMBASE, w9, UXTW]
with is_fpr backpatch info. The RecStubs thunk needed NOTHING — its
is_fpr load tail (Fmov S<n>, w0) and is-load-dest save-mask skip were
already in place; this is the first 32-bit user. ft allocates
MODE_WRITE (wholesale overwrite, recMTC1 rule) before the emit so the
dest rides the live-mask snapshot; the value stays resident for the
following arith and flushes at the next seam. Softmem/faulting-PC
fallback keeps the old w0+store shape (per-compile choice, no mixing).
Census (M2, UYA slot 02): 536,366 -> 533,880 EE-block insns with GE-12
(-0.46% for the pair).
Tests: EeRecLoadStore.Lwc1LoadsToResidentSlotThenArith,
Lwc1OverwritesStaleDirtyResidentDest (stale-dirty-slot writeback
hazard), Lwc1FaultPathLandsInResidentSlot (MMIO fault → thunk Fmov tail
+ dirty-NEON survival + faulting-PC witness). 1207/1207.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
recSWC1 flushed-and-freed any NEON-resident ft, then Ldr'd the value
back from fpr[] memory — a Str/Ldr round-trip that also killed FPR
residency at every SWC1. Resident slots now Fmov the value straight to
w10 (zero memory traffic) and stay mapped; the store needs only the
VALUE, so fpr memory may stay stale while the slot lives (fastmem
faults save/restore it via the GE-07 masks; the softmem path's
iFlushCall still evicts-with-writeback after w10 is loaded).
New test drives the resident-store fast path, a genuine MMIO fault with
the slot live across the thunk (witnessed via vtlb_IsFaultingPC), and a
post-fault consumer of the still-resident slot. recompiler_tests
1181/1181.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
recLoad/recStore ran iFlushCall(FLUSH_CONSTANT_REGS) on EVERY GPR
load/store — and iFlushCall unconditionally frees all caller-saved
allocator GPRs plus ALL NEON slots (4a817b0cb). On the fastmem path no C
call happens, so this existed purely for the fault case — and it clipped
FPU/MMI register residency at every interleaved load/store. A reference
ARM64 PS2 implementation keeps residency here and covers faults with
backpatch live-register masks; our RecStubs thunk has had the identical
mask machinery all along, but the masks were ~always zero because the
eviction preceded mask capture.
Fastmem sites now keep allocator state live:
- _flushConstRegs stays (load-bearing at L/S sites — Tier-2G f4601964b).
- Per-register coherence replaces the blanket eviction: a dirty
NEON/GPR copy of Rs (address) or store-Rt (value) is written back via
_flushEEreg(reg, false) — mapping kept; that writeback is also what
refreshes a stale pin against a dirty MMI quad (same ordering the old
eviction provided). Load-Rt invalidation was already
recStoreLoadResult's job.
- x0 (an allocator pool member) is evicted before an unpinned-dest load
since it becomes the load result. w9/w10/x8/x17 are outside the pool.
- Softmem sites keep the full flush (their result/argument convention
needs the C seam). 128-bit LQ/SQ sites are untouched (conservative
until GE-14 handles quad residency).
Thunk audit for the newly-nonzero masks: save skips the load data
register, the result Mov precedes the restore loop (so a saved x0
allocator value and a pinned-dest result coexist), pins are not
allocator state and ride preserve_most, RECCYCLE is spilled around the
call, and NEON saves are full 128-bit.
Two new mutation tests force a GENUINE fastmem fault (MMIO INTC_STAT
through a runtime address) with dirty 128-bit MMI state live across the
thunk C call, and assert via vtlb_IsFaultingPC that the fault path
actually ran (a vacuous softmem pass fails the witness).
recompiler_tests 1180/1180; UYA stepdiff shows only the pre-existing
benign 0x0013d208 timer divergence.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The three caller-saved pins move from argument-register territory
(x4/x6/x7) into preserve_most's spared x9-x15 range, which the vtlb
dispatchers (vtlb_memRead/Write<T> + 128-bit, vtlb.h) preserve by
contract:
- The seven warm-path seams (4 recVTLB softmem/128-bit slow paths +
3 RecStubs thunk slow paths) switch to
armReloadEEPinsAfterPreserveMostCall / armFlushEEPinsBeforePreserveMostCall,
which emit NOTHING for this table -- warm vtlb seams now carry zero
pin traffic (neither's economics, achieved). Object-verified: every
vtlb_memRead/Write prologue opens with the unconditional
str x15 / stp x14,x13 / stp x12,x11 / stp x10,x9 save, which holds
regardless of what the dispatchers call internally (VIF dynarec,
IntCHackCheck, ...). Plain-AAPCS seams (const-paddr MMIO shortcuts,
vu0Sync family, waitMTVU, stubs, hooks) keep the 3-Ldr clobber
reload via the table walk.
- x4-x7 return to the EE allocator pool (net pool size unchanged at
11); S3's 4473a4b7e already removed every hardcoded w4-w7 scratch
use, so nothing conflicts with allocator residency. x28 stays the
only callee-saved EE pool member (the Arm C ordering invariant is
unchanged). IOP pool re-admits x11-x13 alongside x26/x27 -- IOP is
reachable only through C seams whose reloads restore the pins.
- mVU-side w11/w12/w13 uses are all micro-mode-only (verified):
gprT3/w11 sits in NEON_ADD2SS, mVUsetupFlags' 4-distinct shuffle,
VU branch emitters, and the shared SFLAGc exit thunk; macro mode
hand-rolls its arithmetic natively in iCOP2 (incl. native VCLIP)
and routes only the 12 Lower.inl ops (zero w11-w13 refs) through
the mVU adapter. mVUallocSFLAGd's w11 default has no callers. The
vixl scratch pool is x16-only (x17 removed in armStartBlock).
M2 SotC census (600f, s4->s5 pin map, same 19,715 blocks):
1,088,470 -> 1,088,087 EE-block insns (-383). Predicted "reload bucket
drops by exactly 3x the emitted softmem/128-bit slow-path seam count
(faulting-PC blocks only), total ~flat": actual reload 5,178 -> 4,800
= -378 = 126 seams x 3 Ldrs; byte delta -1,532 = 383 insns x 4 exactly;
every other bucket bit-identical except the pin_as_operand/other
re-keying that follows the register-number swap (cross-map runs on
both dumps confirm identical instruction shapes). The static win is
deliberately small -- Arm D's economics are the runtime fault path
(each thunk execution drops 3 Ldrs) and enabling Arm E's lazy re-trial
(warm vtlb seam bill is now zero).
Tests: full recompiler_tests 1130 green (new
TierTwoQuadWriteThroughThenReadBack covers the 637701474 PMADDW pin-
refresh class on the new homes; the existing softmem/const-paddr seam
tests now ride the emit-nothing path end-to-end). Mutation note
recorded in the test file: skipping x11-x13 in the GENERIC clobber
reload stays green in-harness -- under write-through a clobber reload
only matters when the callee physically trashes the register, so the
generic-seam proof for spared-range pins remains object-level +
stepdiff/corpus. SotC stepdiff: byte-identical to base/Arm C (same
pre-existing timer-derived $s0 propagation at block 0x00124c60, not an
arm artifact).
Census tooling: PIN_MAPS "s5" added and made the default.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The four inline unaligned RMW handlers (LWL/LWR, SWL/SWR, LDL/LDR, SDL/SDR)
were the ONLY consumers of callee-saved allocator temps in EE codegen, and
recUnalignedStoreDouble's non-fused path claimed all THREE (x26/x27/x28) at
once. That blocks EE-SRA 3 Arm C, which re-homes the $v1/$a0 tier-1 pins into
the callee-saved pool. This narrows every handler to a single callee-saved
temp (12 -> 4 total; max simultaneous demand 3 -> 1), leaving x28 free for the
unaligned path after C takes x26/x27.
Two mechanisms:
- Only a value that crosses the read's slow-path C call needs a callee-saved
home. _allocArm64GPR auto-sets `needed`, which already protects a parked
temp from the Rt alloc's eviction — so the loaded-word / fused temps that
are written only AFTER the read become plain temps (recUnalignedWord, both
recUnalignedLoadDouble memTemps, the fused SDL/SDR pair).
- The two store handlers keep the FULL effective address in the sole
callee-saved temp and recompute the aligned address (for read and write)
and shift8/s from it after the read; SDL/SDR additionally re-derives Rt
per path via _eeMoveGPRtoR (the read writes only x0, never Rt's memory).
shift8 lands in w10, the store-value scratch, provably free between the
read and the write.
Fused SDL/SDR fix: with addrTemp now a plain temp it can be x0, so the address
must be moved into w9 BEFORE the value is loaded into x0 (the callee-saved
home used to preclude that alias). Caught by SdrSdlPairUnalignedDwordStore.
Census-neutral (M2 SotC 600f, write-through): 1,092,277 -> 1,092,252 EE-block
insns (-25, noise). Unaligned stores are rare enough that the recompute cost
does not register.
Tests: the two existing all-alignment pressure sweeps (the FlatOut 2 / Black
hang guards) plus new softmem twins MultiRegUnaligned{Dword,Word}CopyBlock-
Softmem -- fastmem-off disables pair fusion, so they exercise the NON-FUSED
LDL/LDR/SDL/SDR reworks (three-temp -> one-temp) that the fused pairs never
reach. Full recompiler_tests 1116 green; SotC stepdiff spot-check clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The JIT-internal readers that bypass armLoadEERegPtr's scalar-only pin
substitution would observe a stale lower half under lazy-dirty: the NEON
dual-residence plain fill, mmiLoadReg, recSQ's quad source, and QMTC2 get an
Ins-lane-0 merge from the pin (armMergeEEPinIntoQuad — no-op in write-through,
keeps laziness intact vs. flushing). CTC2's word read and PMADDW's SL[ss]
lane reads convert to armLoadEERegPtr — correct under both modes and a free
Ldr->Mov substitution under write-through; upper-half (SL[2]) reads fall
through to memory, which is always canonical for the unmirrored half.
The other seam-inventory readers were already safe: LWL/LWR/SWL/SWR/LDL/LDR/
SDL/SDR merge reads route through _eeMoveGPRtoR (pin-aware), and the NEON
fill's const/dirty-scalar arms already Ins the newest value.
recompiler_tests 1112/1112 with the flag off.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Groundwork for pins-canonical-between-seams mode, all behind the compile-time
EE_PIN_LAZY_DIRTY flag (default 0 = write-through, byte-identical emission):
- armFlushEEGPRPins / armFlushEEClobberedPins: flush-ALL / flush-caller-saved
pin mirrors to canonical memory. No dirty tracking — the lrps2 lesson says
compile-time dirty subsets are unsound; seams flush everything.
- iFlushCall: FLUSH_ALL_X86-carrying modes (interpreter fallbacks, branch
tails) flush all pins; lighter modes don't (their inline-fastmem hit paths
make no call) — the caller-saved flush is instead emitted at every actual
call site that pairs with armReloadEEClobberedPins (the reload reads
canonical memory; unflushed it would resurrect stale values over dirty
caller-saved pins). 20 sites paired across the rec + backpatch thunks.
- Dispatcher stubs: JITCompile / DispatcherEvent flush all pins before
recRecompile / recEventTest (compile-time hooks and savestate/VM-exit read
guest GPR memory); block-discard / page-reset flush the caller-saved set.
- mVU macro-mode COP2 bridge gains the matching flush-before
(armEmitEEClobberedPinFlushForCOP2) around waitMTVU.
- iCOP2 vu0Sync sequences: flush hoisted before the FIRST call of each
sequence, and the previously brace-less if/else bodies are braced — the
flush must not run after the pins are already clobbered.
recompiler_tests 1112/1112 with the flag off.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
C3: a pinned Rt is passed as the recorded per-site value register instead of
being copied into w10/x10 pre-flush — the fastmem Str / softmem call consume
it post-iFlushCall where the pin is coherent; the thunk and softmem
normalize read-only.
C4: a pinned load dest is passed as the fastmem data register, so the load
lands directly in the pin (Ldrsb/sh/sw x-view or Ldrb/h/w w-view — exactly
the guest sign/zero-extension) and recStoreLoadResult's canonical Str is all
that remains (the mirror Mov self-skips on src==pin). On a fault the thunk
reconstructs into the recorded dest after armReloadEEClobberedPins, so
caller-saved pin dests get the load result last. Softmem keeps x0 (C-call
return); LDL/LDR merge paths untouched (RMW through x0 by design).
Census (SotC): mov_from_pin 22,509 -> 16,638, mov_to_pin 34,656 -> 20,767,
total -20,098; exact categories stable. Cumulative since EE-SRA 2 start:
-10.3% emitted EE instructions. recompiler_tests 1112/1112.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
