Rs == Rt+1 puts the two sources next to each other in the GPR array, so
a 16-byte read at &GPR.r[Rt] + sa funnelled them without a TBL. It was
the slower of the two sequences on both cores at every sa, and worst on
the twelve adjacent pairs that have a pinned source - the register range
a funnel-shift memcpy loop uses.
armFlushEEGPRPin had no other caller.
The sa mask stays, but it no longer guards a host address. Every writer
of SA - MTSA, MTSAB, MTSAH - already keeps four bits, so the two
oversized-sa tests could not fail; one goes with the path it named and
the other says what it covers.
QFSRV used the same store-and-reload trick x86 needs for a
cross-register byte shift. A two-register TBL indexed by a byte ramp
plus a broadcast sa does the same shift without the reload - recQFSRV's
comment has why the reload, not the spill, was the expensive part.
The ramp lives in the register pack next to the COP2 constants; R5900.h
explains why.
The two AX-03 tests keep their assertions and change their names: the
buffer they were named after is gone.
Inspired by Whatcookie's work on arm64 for rpcs3.
Both the interpreter and the arm64 recompiler modelled each PMADDW/PMSUBW
lane as two independent 32-bit accumulations. That loses the carry between
LO and HI, needs a truncating `/ 0xFFFFFFFF` where an arithmetic shift
belongs, grew a +0x70000000 lane-0 addend to compensate, and leaves LO
sign-extended from the wrapped intermediate rather than from the
architectural result. The x86 recompiler has always done it correctly
(recPMADDW: PMULDQ + PADDQ + PMOVSXDQ against the packed accumulator), so
the three engines disagreed; this brings the other two into line with it.
Scored against the 64 ps2autotests PMADDW/PMSUBW captures, which preset
HI/LO to nonzero values and so make the width of the accumulate observable:
two 32-bit halves + errata 54/64
packed 64-bit accumulate 64/64
The errata components are not separable. Reintroducing any one of them
alone into the corrected model scores far worse than leaving all three in:
packed + voodoo addend 52/64
packed + truncating divide 3/64
packed + carry dropped from HI 33/64
The truncating divide's off-by-one very nearly cancels the missing carry
on these operands, which is why the shipping code only lost 10 cases. A
partial fix here would have been much worse than none.
On arm64 the correct form is also the smaller one: Smull plus a 64-bit
add against the spliced accumulator replaces the lane splitting, the SDIV
by 0xFFFFFFFF and the whole conditional-addend block, and Rd.UD[dd] turns
out to be the raw result. Verified bidirectionally: reverting either
engine alone leaves 7 tests failing, since h.Run() auto-diffs the JIT
against the interpreter.
Tests: the six EeRecMmi vectors that pinned the errata are rewritten to
pin the carry, the borrow, the per-half sign extension and the absence of
the lane-0 addend; all six fail on the unpatched baseline. The
PMADDW/PMSUBW allowance in EeMmiConsoleConformance is deleted, so all 64
cases now assert normally.
Also corrects a comment in iMMI-arm64.cpp claiming MMI2_RECOMPILE is
never defined -- Config.h:1650 defines it unconditionally, which makes
that part of pcsx2/x86/iMMI.cpp the shipping x86 implementation rather
than dead reference code.
Idea by pstef.
recQFSRV has a fast path for Rs == Rt+1 that reads the contiguous 256-bit
{Rt:Rs} window straight out of cpuRegs.GPR with an unaligned raw Ldr. Its
comment claimed the window was "memory-coherent after the flushes above",
but those flushes are mmiFlushReg -> _deleteEEreg, which reconciles
const-prop and the scalar/NEON slots and never touches the pins.
Under lazy-dirty the pin is authoritative for UD[0] and armStoreEERegPtrRaw
elides the canonical store entirely for a pinned lane-0 write, so a pinned
source's lower half in memory is routinely stale mid-block. Nine GPRs are
pinned, which makes four adjacent pairs both-pinned -- ($at,$v0) ($v0,$v1)
($v1,$a0) ($a0,$a1) -- plus eight more with one pinned operand: exactly the
register range a funnel-shift memcpy loop uses. Failure mode is wrong data,
not a fault.
Every other raw quad-load site fixes this by merging the pin into lane 0
after the load, which cannot work here because the read straddles two guest
registers. Flush the two pins the window covers instead -- it covers exactly
r[Rt] and r[Rt+1], since sa <= 15 over their 32 bytes -- via a new
armFlushEEGPRPin. That keeps the fast path (0-2 extra Str) rather than
falling back to the ~10-instruction temp-buffer path, and the flushed pins
stay authoritative.
This was the last raw address-of-GPR read in pcsx2/arm64/; the GE-M2e sweep
in 3bc64ac11a covered the mergeable sites and missed this one. Also fixes
the comment, which is what made the hole look deliberate.
Tests: two red-on-unfixed cases dirtying a pinned Rt and a pinned Rs, plus a
non-adjacent green control that proves the divergence belongs to the fast
path. recompiler_tests 1442/1442.
A probe was built with the ps2dev toolchain, run on a PS2, and streamed
its results back over ps2link's `host:` channel. Its two records are the
oracle behind autocases_sa.h.
SA is four bits wide. `mtsa 0x10` leaves SA at 0, `mtsa 0xFFFFFFFF` at
0xF, and MFSA reads the masked value back, so the mask belongs at the
write. PCSX2 stored the raw 32 bits, which was the root cause of three
different QFSRV behaviors for sa >= 16: UB in the interpreter, a
guest-controlled out-of-bounds host read in the x86 recompiler, and a
defensive mask in the arm64 one - and upstream recMTSA masks on its
const path but not its runtime path, so x86 disagrees with itself too.
Masking at MTSA makes all of it unreachable at once.
Leaving it would mean permanently suppressing a JIT-vs-interp diff.
EeRecTraps.MtsaCopiesFullRegister and MtsaConstFoldsAtCompile asserted
the old behavior and were updated to the console's value.
Two defects found and recorded
1, link branches with rs == $31 judge the condition on the pre-link
value on console: `bltzal $ra` with $ra = -1 is taken. Their no-link
twins agree on all six probe values, so the link write does not feed the
comparison. PCSX2 links first in six functions (EE BGEZAL / BLTZAL /
BGEZALL / BLTZALL, IOP psxBGEZAL/psxBLTZAL); 12 rows x 2 engines
diverge. This settles the question
IopLsuBranchConsoleConformance.LinkBranchesWithRaAsSourceArePinned left
open. The assembler refuses to encode these at all, which is why no
capture had them - they went in as raw .words, each verified by
disassembling it back. `jalr $31,$31` rode along as a positive control
and passed: the console jumps to the OLD $ra, corroborating the psxJALR
ordering fix on a second CPU.
2, PCCR's writable mask is 0x800FFBFE - exactly the layout R5900.h
documents, with pad0, pad1 and the eleven Reserved bits reading back as
zero. PCSX2 stores it verbatim; 14 rows x 2 engines diverge. PCR0/PCR1
are fully 32-bit writable, so PCSX2 is already right there.
DISABLED_AllSaPerfMatchesConsole is the graduation tripwire for both.
Add "yaps2 Dev Team" copyright to the files we authored. Net-new files
(all 42 pcsx2/arm64/ codegen/ProgCache/persist sources, the recompiler
test suite + harness, and the vurunner/eerunner tools) never existed
upstream, so they carry yaps2 sole credit. RecStubs.cpp predates the
fork and was heavily extended, so it keeps PCSX2 credit and adds yaps2.
The five pre-existing arm64 files we only lightly touched (AsmHelpers,
Vif_Dynarec, Vif_UnpackNEON) stay PCSX2-only. GPL-3.0+ license lines are
unchanged throughout; this is authorship attribution only.
The raw quad-load sites (SQ / QMFC2-QMTC2 / MMI memory-path) load a 128-bit
guest GPR from cpuRegs memory and fix up a stale lower half from a dirty pin
via armMergeEEPinIntoQuad. Once scalar residency is flipped on, that lower half
can also live dirty in an ARM64TYPE_GPR slot the site never flushed (recVTLB SQ
only iFlushCall(FLUSH_CONSTANT_REGS), which leaves scalar slots resident).
Add armMergeEEResidentIntoQuad — a side-effect-free superset of the pin merge
that also Ins-es a dirty scalar slot's lower 64 into lane 0 (pin and scalar
slot are mutually exclusive by I1). Route the three emitter quad-load sites
through it. recVTLB SQ's quad load switches to armLoadEERegPtrRaw so the
intentionally-stale-then-merged load doesn't trip the I3 tripwire.
(_allocGPRtoNEONreg already displaces a dirty scalar slot on its fill path;
this closes the raw-load gap.) New SqAfterScalarWriteMergesDirtyLowerHalf test
pins the behavior; 1279/1279 recompiler_tests pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* EE/FPU: PS2 guard-bit ADD/SUB masking on the fast path
The single-precision FPU fast path emitted bare Fadd/Fsub for
ADD/SUB/ADDA/SUBA and the MADD/MSUB/MADDA/MSUBA accumulate, omitting the
exponent-difference mantissa masking that reproduces the EE FPU's lack of
guard bits. x86 applies FPU_ADD_SUB there unconditionally
(FPU_CORRECT_ADD_SUB=1, x86/iFPU.cpp), and the arm64 DOUBLE path
(iFPUd-arm64.cpp) already had it; this adds it to the fast path via
fpuEmitGuardedAddSub().
The masking is emitted one of two equivalent ways, selected at build time by
FPU_GUARD_MASK_STUB (iR5900-arm64.h):
0 = inlined into each add/sub block
1 = emitted once as a shared stub (g_fpuGuardMaskStub), reached by bl
The common case (|exp diff| <= 1) skips masking in both.
The interpreter's ADD_S/SUB_S is a bare host float+float (fpuDouble() returns
float, no masking), so the masked result differs from the interpreter for
|exp diff| >= 2 and matches PS2 hardware / the x86 JIT instead. Tests are
therefore JIT-only: ee_rec_fpu_guardbit_tests.cpp (6 tests), verified against
the unmasked baseline (5 masking tests fail there, pass masked; the |diff|==1
boundary matches the interpreter).
* EE/FPU: native RSQRT.S
RSQRT.S was the last FPU op deferring to the interpreter via recFPUCall; it
is now native (recRSQRT_S_xmm), computing Fd = Fs / sqrt(|Ft|) in the shape
of the neighbouring recDIV_S_xmm/recSQRT_S_xmm: swap host FPCR to the
nearest-rounding FPUDivFPCR around the sqrt+div, copy both operands into
temps (EEREC_D may alias EEREC_S/EEREC_T), handle the zero-divisor special
case (Ft exp field 0 -> sign(Ft) | 0x7f7fffff), and write the FCR31 sticky
flags the interpreter sets: clear I|D each op, set D|SD when Ft is zero and
I|SI when Ft is negative. Mirrors x86 recRSQRThelper1.
Differential scope (executed, 40k random pairs + a standalone C model):
native single-precision RSQRT matches x86 (xSQRT.SS/xDIV.SS) and the EE FPU
bit-for-bit; the interpreter's positive-divisor path computes Fs/sqrt(Ft) in
double (libm sqrt returns double, so the divide promotes) and lands exactly
<=1 ULP away on ~10% of inexact quotients. Zero and negative-divisor paths
match the interpreter exactly (its negative path rounds sqrt(|Ft|) to a float
temp first, so that divide is single-precision); flags match on every input.
Tests: ee_rec_fpu_rsqrt_tests.cpp (10) -- a 3000-iter Run() differential
fuzzer over the zero/negative domain, a 3000-iter <=1 ULP fuzzer over the
positive domain (JIT and interp run on separate harnesses, since Run()'s
auto-diff trips on the 1-ULP gap), exact-ratio / register-aliasing /
denormal-divisor / sticky-flag differential cases, a JIT-only positive
single-precision value (1/sqrt(1.5): native 0x3f5105eb vs interp double
0x3f5105ec), and a DISABLED tripwire pinning the interp divergence. The
JIT-only value test fails on the interp-defer baseline (gets 0x3f5105ec) and
passes native (0x3f5105eb). The two pre-existing RSQRT tests in
ee_rec_fpu_tests.cpp stay differential (exact results). Full recompiler_tests
1155/1155.
---------
Co-authored-by: pstef <3462925+pstef@users.noreply.github.com>
PSLLW/PSRLW/PSRAW/PSLLH/PSRLH/PSRAH memory-bounced through
mmiFlush+Ldr+op+Str (3 insns + 2 mem ops) and evicted NEON residency at
every immediate shift. Route them through the MMI_2OP_SETUP XMM template
like their sibling ops: a single resident Shl/Ushr/Sshr qd,qt,#sa.
sa==0 (incl. halfword sa&0xf masking) degenerates to a register move,
skipped when the allocator aliases rd onto rt.
New edge tests: sa==0 copy, Rd==Rt aliased shift, and the aliased
zero-effective-sa no-op. recompiler_tests 1186/1186.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
recPMADDWLane committed Rd.UD[dd] with a raw Str, bypassing armStoreEERegPtr
— the only such bypass in the tree. With a pinned Rd the mirror stayed stale
after PMADDW/PMSUBW, so any subsequent pin-served read (branch compare,
address base, ALU source) used the pre-PMADDW value. Latent since the S3 pin
campaign; surfaced by the lazy-dirty test run (where the seam flush then
clobbered the correct memory with the stale mirror — 6 red tests).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The JIT-internal readers that bypass armLoadEERegPtr's scalar-only pin
substitution would observe a stale lower half under lazy-dirty: the NEON
dual-residence plain fill, mmiLoadReg, recSQ's quad source, and QMTC2 get an
Ins-lane-0 merge from the pin (armMergeEEPinIntoQuad — no-op in write-through,
keeps laziness intact vs. flushing). CTC2's word read and PMADDW's SL[ss]
lane reads convert to armLoadEERegPtr — correct under both modes and a free
Ldr->Mov substitution under write-through; upper-half (SL[2]) reads fall
through to memory, which is always canonical for the unmirrored half.
The other seam-inventory readers were already safe: LWL/LWR/SWL/SWR/LDL/LDR/
SDL/SDR merge reads route through _eeMoveGPRtoR (pin-aware), and the NEON
fill's const/dirty-scalar arms already Ins the newest value.
recompiler_tests 1112/1112 with the flag off.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
PMADDW/PMSUBW were REC_FUNC interpreter calls under the claim that the
PS2 multiplication errata is un-emittable natively. It isn't: the errata
divisor is the CONSTANT 0xFFFFFFFF, so a 64-bit SDIV against positive
0x00000000FFFFFFFF is exactly the interp's C `temp2 / 4294967295` (s64
truncation toward zero; positive divisor rules out the INT64_MIN/-1
case), and the lane-0 "division voodoo" (+0x70000000 on boundary Rt when
Rs != Rt) is two logical-immediate tests + Umull-zero + Ccmp/Csel,
branchless — other mac backends emit this natively too; scalar per-lane
sequence modeled on our recPMADDUW, using only
non-allocatable scratches (w8/w9/w10/x17).
Semantics note pinned by test: the interp's LO update
`(s32)low32(temp) + LO.SL[ss]` overflows s32 for some inputs — UB the
shipping compiler resolves as a 64-bit add of sign-extended halves
(LO.SD[0] = +0x80000000, not a wrapped sext). x86 runs the identical
interp path via REC_FUNC, so the JIT emits the 64-bit-add form
(Add x, x, Operand(w, SXTW)) to match the cross-arch reference.
Validation: 7 boundary-vector tests hand-derived to sit on quotient
crossings, validated green against the interpreter BEFORE the native
swap, then green on the native path; mutation checks (voodoo disabled;
SDIV replaced by >>32) each go red on exactly the intended tests.
Full suite 1085 green. Per-lane commit order preserves the interp's
Rd/Rs/Rt aliasing behavior (lane 1 reads UD[1] fields lane 0 never
writes) — covered by the Rd==Rs aliasing test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both recQFSRV paths added the raw 32-bit cpuRegs.sa to a host base (fast
path off &GPR.r[Rt], slow path off the 32-byte temp buffer) before the
128-bit load. Our MTSA is deliberately a full 32-bit copy (3c381989e,
matching the interp), so a game doing MTSA with a value >= 16 walks the
load out of bounds — a guest-controlled host OOB read. x86 avoids this
only because recMTSA masks sa & 0xf at store time; we combined interp-
faithful MTSA with x86's raw-index QFSRV.
Fix: And sa, 0xf at consumption in both paths. MTSA/MFSA keep the full
value (round-trip pinned by test); QFSRV behaves as x86's effective
sa & 0xf for oversized values (the interp is shift->=64 UB there, so
memory safety + x86 equivalence wins over matching host-dependent
garbage).
TDD: QfsrvOversizedSaClampedOn{FastPath,TempBufferPath} red on unfixed
code (fast path read bytes of the neighboring guest GPR), green after;
in-range MTSA+QFSRV+MFSA auto-diff test guards the no-op case.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Pin the two hottest guest GPRs into callee-saved host registers:
x22 mirrors GPR.r[29].UD[0] ($sp), x23 mirrors GPR.r[31].UD[0] ($ra).
Memory stays canonical — every guest-visible write still stores to
cpuRegs.GPR and refreshes the mirror at the same emission point — so
there is no new C-call or block-exit contract, no const-prop coupling,
and no allocator involvement. Reads that would load UD[0] from memory
use the mirror instead:
- armLoadEERegPtr substitutes Mov-from-pin for the Ldr (covers the
allocator MODE_READ load, _eeMoveGPRtoR's memory fallback, and every
op-file read), deleting the dependent load at the head of scalar ops.
- The scalar ALU/imm/shift memLoad helpers return the pin register
directly (zero instructions for a pinned source); memStore helpers
take the value register so pass-through cases (DADDU rd, rs, $zero;
DSLL rd, rt, 0; ...) store the pin with no intermediate Mov.
- armStoreEERegPtr write-through: Mov for 64-bit stores, Bfi for
UL[0]/UL[1] halves, reload-from-memory for odd shapes. New
armStoreEEGPRQuad covers the 128-bit writers (NEON writeback, MMI,
LQ, QMFC2) with a lane-0 UMOV refresh.
- Pins reload from memory at every JIT entry and after the C calls
that can write guest GPRs: recEventTest (savestate load),
recRecompile (ELF hooks), recCall/recBranchCall (interpreter
fallbacks), eeloadHook/eeloadHook2.
- x22/x23 leave the dynamic allocator pool (ALLOCATABLE_MASK). They
were the only pin-capable callee-saved regs: x26-x28 are the mVU
macro-mode VI cache and the vtlb unaligned helpers' 3 simultaneous
callee-saved temps. COP2 macro mode never touches x22/x23 (flags go
through the s_cop2DenormStatusFlag memory scratch, not gprF2/F3);
the mVU micro dispatcher saves/restores x19-x28.
This is the P2 extraction from the neither/LRPS2 RegCache deep-dive
(2026-07-01): the safe subset of static pinning — reads free, writes
keep the str — without neither's cross-block dirty-pin hazard.
SD865 A/B (codegen_ab.sh, 5 runs, --renderer null): SotC insns -0.06% /
cycles -1.36% (IPC 1.826->1.850), UYA insns -0.50% / cycles -1.36%
(IPC 2.390->2.411) — the guest-reg load latency was not OoO-hidden on
the A77, confirming the deep-dive's prediction. M2: SotC cyc -0.67%,
UYA cyc +0.86% (wider OoO window hides the loads; device is the gate).
Tests: new ee_rec_pinned_gpr_tests (5 cases: scalar reads, scalar
write-through + read-back, 128-bit MMI write-through, vtlb-load
write-through, CFC2 half-word Bfi path), each red under a deliberate
pin-update mutation and green with it restored. Full recompiler_tests
1052/1052.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Full ARM64 EE dynarec — dispatcher, iFlushCall, block emitter and opcode subgroups
(Arit/Branch/Jump/LoadStore/Misc/Move/MultDiv/Shift/Templates), the COP0/COP2/FPU/MMI
coprocessor codegen, the register allocator core (iCore), VTLB codegen, and the EE
block-analysis pass. EE GPRs are allocated in NEON registers, per the approach used
in a reference ARM64 PS2 implementation.
Co-Authored-By: Ryan Walklin <ryan@testtoast.com>
Co-Authored-By: Brian Degenhardt <bmd@bmdhacks.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
pstef
bmdhacks
pstefandpstef
