Both polarities of the gamefix. With it on, a rewritten immediate has to
reach the block already compiled; with it off, the rewrite has to force
a recompile. Each row asserts the compile count as well as the result,
so neither can pass by recompiling behind the value it checks.
Three things the harness needs handling for. The first re-entry through
RunJitPreserveBlockCache compiles a second block variant, because it
enters on the pipeline state the previous run left rather than the
post-Reset one, so the block only goes warm on the re-entry after that.
LoadProgram writes VU.Micro directly and so bypasses the vtlb path that
calls mVUclear, which leaves the stale quick slot serving the old
program unless the test runs the invalidation itself. And the pair that
carries the immediate writes VI[REG_I] again on the way out, as does the
delay slot LoadProgram appends, so the register reads back 0 and the
assertion belongs on the VF result.
x86's doIbit picks between folding the immediate into the block and
loading it from micro memory; arm64 only ever folded.
Scarface (SLUS-21111), the game the gamefix is listed for, writes a
per-object transform into VU1 micro memory as I immediates. Under the
recompiler its trees and much of the geometry around them took whichever
object's transform compiled first. The interpreter re-reads the word on
every dispatch and was unaffected.
vtlb_Miss raised the exception under the arm64 recompiler and returned,
on the assumption that the rec would pick cpuRegs.pc up at the next
dispatch. Nothing picks it up: the block runs on and its tail writes its
own branch target over the vector PC, while cpuException has already
latched Status.EXL. cpuException leaves EPC alone whenever EXL is
already set, so from the first swallowed miss onward every exception
keeps its predecessor's EPC, and the guest kernel's syscall epilogue
erets to an address belonging to the fault.
Report the miss and continue instead, as the x86 rec already does. The
guest's handler still does not run, so a title that needs demand paging
still cannot work under the rec; a stray miss now stays local to the
instruction. ee_rec_tlb_divert_tests.cpp describes the divert and stays
disabled.
With nothing left to set s_recTlbMissOccurred, the poll after every
interpreter call goes too. It could not have covered the raise anyway:
only the interpreter-call sites had it, so a flag set at an inline
access sat there until an unrelated later recCall consumed it and
diverted on a pc belonging to neither.
This gives up the one case the rec did handle, a miss in a branch delay
slot reaching the vector through the cpuRegs.branch bracket epilogue.
Its recompiler half moves to the disabled file; the interpreter half
stays live, beside a new test pinning the rec's behaviour. The bracket
itself stays: the tlb_fallback_* handlers raise from a delay slot on
their own.
A TLB miss on an inline load or store leaves the arm64 EE rec at
exception level in user code. Eight cases, all disabled, because the rec
does not do this yet and the route to making it is staged: x86's
behaviour first as a floor, then the divert built back up with these
dropping their prefix one at a time.
Found from the other end. `3D Pinball Space Cadet (PS2) (3.0) (RA)`
hangs after "Parsing complete. Finalizing...", and the visible fault is
a thread id of 0xff966c22 arriving at a caller whose syscall returned 1.
That is strlen's `subu v0,v0,a0` on the correct v0, reached because the
kernel's syscall epilogue eret'ed into the middle of strlen: 131072 of
the run's next 162766 exceptions were taken with EXL already set, so
none of them updated EPC. Upstream of all of it is one swallowed miss on
a strlen(NULL). Two working hypotheses died on the way — that the
pinned-GPR cache lost v0, and that the call-ret shadow stack mispopped —
both refuted by reading the state at the moment of damage rather than by
reasoning about the emitters.
The tests are the chain in four instructions, plus the load and store
halves of both inline emitter pairs, plus the flush the divert needs.
Three separate defects fall out of the one missing poll. The block runs
past the faulting load. EPC names the instruction after it, because
cpuTlbMiss skips its `pc -= 4` for the rec while the rec's own cursor is
already one instruction ahead outside a delay slot — the delay-slot case
comes out right only because two errors cancel. And Cause is whatever
exception came last, describing a different instruction than EPC does.
Two findings shape the fix rather than the tests, so they are recorded
here.
Reaching the vector with guest state intact costs one writeback, not a
general flush: iFlushCall(FLUSH_VTLB) already precedes every inline
access and frees the caller-saved hosts, leaving only x28, the
allocator's single callee-saved host. A boot with fastmem off puts a
live dirty guest GPR there at 489 sites and nothing else anywhere.
The fastmem backpatch thunk is the part with no clean answer. It is
generated at fault time and cannot name the live guest values of the
block around it, so it cannot divert. A census of the 12411 fastmem
sites emitted during that boot says how much per-site state a precise
one would need: 9355 have nothing live and dirty, 2480 have between one
and seven GPRs, and 576 involve the NEON file.
Lesson, from a guard test that was written wrong first. Three dirty
registers before the faulting load is not enough to make the allocator
reach x28, so that test passed with the writeback deleted — it guarded
nothing. It takes sixteen live guest values at once. A test that guards
a writeback has to create the pressure that puts something in the
register the writeback exists for, and the way to find out is to delete
the code and watch.
The VU's largest value is 0x7FFFFFFF, one binade above FLT_MAX, the same
range the EE FPU has. So an exponent-255 word is an ordinary number on
the way in and on the way out, and "overflow" starts above it rather
than above FLT_MAX.
Both engines put the boundary a binade lower. vuDouble() rewrites an
exponent-255 operand as 0x7F7FFFFF and VU_MAC_UPDATE() calls every
exponent-255 result an overflow; the arm64 COP2 macro emitters clamp the
result to +/-FLT_MAX and raise neither O nor U. microVU's per-op operand
clamps approximate the same thing from a list of games rather than a
rule.
68 rows off an SCPH-90000 through VU0 macro mode, scored per engine and
per column, with what each engine cannot yet reproduce recorded per case
so a fix trips the test as loudly as a regression. Nothing is fixed
here.
Two of the rows are structural rather than about range. An overflowed
product does not become 0x7FFFFFFF before the accumulate: an addend of
-0x7FFFFFFF cannot cancel it. An underflowed product does become zero
before it. And the multiplier is the EE's, with the same one-ULP deficit
decided by ft's mantissa alone.
The harness grows two things the rows need: a VADDA encoder, and an
opt-out from Run()'s VU0 JIT-vs-interp auto-diff for tests that score
each engine against a hardware capture instead of against the other
engine.
DJbox 2
Ratchet & Clank 2 13
Tourist Trophy 6
Monster Hunter 12
The Taxi 2 1
Pride FC 4
Jak and Daxter - The Precursor Legacy 14
Jak II 10
Jak 3 6
Final Fantasy X 15
Driv3r 6
Shadow of the Colossus 5
Gran Turismo 4 19
Tales of the Abyss 4
Disney/Pixar Up 5
Need for Speed - Undercover 9
Mortal Kombat - Shaolin Monks 3
Tokyo Xtreme Racer Zero 5
Yu-Gi-Oh! - The Duelists of the Roses 3
Devil May Cry 3 Special Edition 5
Constantine 3
Sega Ages 2500 Virtual On 1
Tales of Destiny - Director's Cut 4
Colin McRae Rally 3 2
Stuntman 5
Deadly Strike 1
Virtua Fighter 4 4
R:Racing Evolution 5
Xenosaga Episode III 7
Steambot Chronicles 11
Fatal Frame 10
Playmobil - Hype - The Time Quest 3
Exponent-0xFF operands through VMADDw and VMSUB at every dest mask, one
case per half of MADDw's set. MADDx/y/z runs the same shape with its
broadcast lanes in range as a scope control.
arm64 hand-rolls these ops instead of going through the mVU emitter, so
microVU's per-op clamp argument is retyped in iCOP2-arm64.cpp. The two
rows conditional on isCOP2, MADDw's cACC|cFt|cFs and MSUB's cFs, were
missing.
Shadow of Destiny moved its player character a fraction of a step per
frame under the recompiler, at every eeClampMode, on the MADDw row.
Clamping ACC also changes results against the interpreter for values
above FLT_MAX, but is what the x86 COP2 path already does.
Exact was reachable through the GameDB and the INI only: the desktop and
Big Picture pickers stopped at Full, and cleared fpuExactMode behind
themselves. Both now read and write it with the other three bits, so the
mode can be picked and a config already on it displays as itself.
The Big Picture choice list is sized from its own array. It had taken
the EE array's length for the VU pickers too, which held only while the
two arrays were the same length.
eeFprSyncSlotFormat keys on CHECK_FPU_FULL, so the relocated file is
what both of iFPUd's rungs compute over, not mode 3 alone. The comments
that name a mode where they mean the tier are renamed to the tier.
DIV.S, SQRT.S and RSQRT.S ran on host doubles on every recompiler rung,
which makes them correctly rounded; the EE's divide/square-root unit is
a digit recurrence with no rounding step and is not. The interpreter has
run the recurrence since the model landed, so mode 4 now calls the same
two functions out of line rather than reproducing them. eeDivide and
eeSqrtBits stop being static for it.
RSQRT.S composes the two with an ordinary single in between, as silicon
does, and its intermediate crosses the square root's call through the
island's scratch. Only the value moves: the negative-operand and
zero-divisor arms, and every flag they set, are the code they were.
Modes 1 to 3 keep the host instruction and the FPUDivFPCR swap.
The rsqrt sign table gains a mode-4 leg, which owes every row; the
filter for the twenty rows mode 3 misses by one ULP stays where it is,
now as that mode's own pin. The divide unit's console table gains a
mode-4 leg beside its interpreter one.
The multiply array's out-of-line call carries its own spill of the
allocator's live caller-saved homes and its own pin flush/reload pair.
The divide unit needs the same frame around calls of its own, so the two
halves become emitIslandEnter/emitIslandLeave and the multiply is
rewritten on top of them.
Enter takes a byte count for scratch above the saved registers, which
nothing needs yet at zero and RSQRT.S will: its dividend has to outlive
the square root's call.
A standalone probe of the exponent band the FP matrix corpus does not
reach: a large operand at exponent 255 against an addend walked across
the guard mask's erasure boundary and the narrowing boundary just above
it, through all eight add/sub-family forms. Both engines and both rungs
of the DOUBLE path are scored separately, and the table is checked for
rows of each polarity so an engine that never raises O cannot pass it.
raiseOrClearOU() was handed a sum recomputed from the unmasked operands
while the destination got the masked one, so an operand the guard mask
had erased could still move O. The add/sub family now rounds and flags a
single value, eeGuardedSum()'s, and the accumulates hand their second
raise the sum eeMulAccumulate() actually formed instead of rebuilding it
from the ACC and the unrounded product.
The overflow test moves with it, into eeRoundsOutOfRange(), so the value
path and the flag path agree about the band where a result narrows back
onto kEeFpuMax.
madFlushedProduct() goes with the rebuild it existed for: the product
reaches the adder through eeMulRound(), which already returns a signed
zero for an underflowing one.
Both narrowing helpers compared the wide result against kEeFpuMax and
saturated on anything above it. A result can be above it and still
narrow back onto it, and then nothing saturated and O must stay clear;
ADD/SUB and the accumulates reach that band whenever the guard mask
leaves the smaller operand its leading bit and nothing else. Masking the
low 29 mantissa bits before the comparison is that narrowing, under the
arithmetic FPCR. DIV/SQRT/RSQRT round to nearest and pass flags=false,
so their test is unchanged.
The mask sits on the arm that has already branched away for 2^128 and
up, so the in-range path is untouched.
The mask keys on the exponent difference and clears low mantissa bits,
and a widened slot carries both: the exponent field moves from bits
23..30 to 52..62 and mantissa bit k to bit k+29, so FPU_ADD_SUB_D --
already the form the accumulates use -- is the same law with the shift
29 further left. That leaves nothing wanting the architectural single,
so both operands go straight through SlotToDouble, and the narrowing
pair and the widening's exponent-0xff test go with the single-domain
form. ADD.S and SUB.S drop from 127 host instructions to 93, ADDA.S and
SUBA.S from 132 to 98.
recFPUOp was the only caller reaching the arm that masks ft, which the
console corpus never sampled, so the new witness table covers it against
the interpreter and the console rows now also run through the DOUBLE
path.
ToPS2FPU_Full's saturating and flushing arms are for results that leave
the PS2's range, and a root does not, so both sat dead behind their
compares. The body goes from 54 host instructions to 23, under the 26
the single-precision path spends.
The same three moves as DIV: the sign test reads bit 63, Fcmp against
zero reads the slot, and the saturated result assembles from
kEeFprSignBit and kEeFprMaxBits. |ft| becomes unconditional, since it is
a no-op on the positive arm and is also the copy that keeps the guest
slot intact — which is what narrowSrc was there for.
108 host instructions to 75.
narrowSrc, ToDouble and ToDoubleFrom are down to one caller between
them: the ADD/SUB guard mask, which rewrites the word.
recDIVhelper1 narrowed both operands so it could compare them against
zero and xor their signs in the word domain. Fcmp against zero reads the
slot the same way — FZ flushes an EE denormal there too — and the sign
sits at bit 63, so the saturated result assembles from kEeFprSignBit and
kEeFprMaxBits without a word ever being formed. Both arms leave a slot
now, which costs the caller a register move where it paid for a widen.
102 host instructions to 69, and four temp NEON registers to two.
The body narrowed ft to the architectural single, tested the sign bit
there, then widened that single back to a double through ToDouble's
exponent-0xff branch. The slot carries the sign at bit 63 and holds the
value scaled by a power of two, so the test reads the slot as it stands
and the widening is SlotToDouble's one exact multiply.
71 host instructions to 54.
The double tier narrowed both slots to words, wrapped each in a 64-bit
double pattern, ordered them with Fmax/Fmin and widened the winner back.
The relocation is order-preserving, so the ordering key the fast path
already uses reads the slot unchanged, and Csel picks between the two
untouched registers.
24 host instructions to 14, and two temp NEON registers to none. What is
left over the fast path's 12 is the fprc[31] memory RMW this tier owes.
Both capture-backed tables read the operands the fpmul3 sweep produced,
which is exponent 127 with both operands positive. Every model of the
deficit here reads the significands and nothing else, and nothing said
so.
Sixteen rows from a probe that moves the other two fields: four sign
combinations across five exponent placements for eight separating
operand pairs, of which the extremes are kept. All 160 rows the probe
ran came back one ULP low and no operand pair changed its verdict.
The two rungs differ on two classes of operand and the 8137-case
hardware corpus reaches neither: its zero-tail multiplies all carry ft
mantissas the boundary term cannot read, and its rows inside the array's
band either saturate or come back exact. It scores mode 3 and mode 4 the
same, so nothing in the tree said which of them silicon agrees with.
The fpmul3 capture does. It swept eight fs significands against every
one of the 2^23 ft significands on an SCPH-90000, recording for each
whether the console returned the correctly-rounded product or that
product one ULP low, and those two are the only outcomes it ever saw.
Twenty rows where the boundary term decides and twelve inside the band
are transcribed here with both values, so each asserts mode 4 against
silicon and mode 3 against the rounding it is supposed to keep.
Half of each table has the two values equal, which is the polarity that
stops an emitter that decremented every product from passing.
It patched one product, 0.25 * pi, from the correctly-rounded 0x3f490fdb
to the 0x3f490fda the EE's multiplier returns, so Tales of Destiny stops
hanging. That product is one sample of the multiplier's one-ULP deficit,
and eeClampMode 3 models the deficit for every operand pair -- including
the asymmetry the gamefix reproduced by comparing fs and ft against
their own constants, which falls out of a predicate that reads ft alone.
The seven serials carry the clamp mode instead, and iFPU-arm64.cpp's
multiply is a bare Fmul again.
The toggle goes from the Qt, FullscreenUI, Android and iOS front ends,
along with the GameDB schema enum and the harness helper that set it.
Fix_FpuMultiply and the config bit stay: vu_capture's on-disk gamefix
mask is bit-indexed by GamefixId, so dropping ordinal 0 would
reinterpret every capture already recorded. tbl_GamefixNames keeps the
name, so a GameDB that still lists it parses and does nothing.
Both iFPUd modes emitted the multiplier deficit in full: the Booth term,
the boundary predicate over it, and an out-of-line call to the multiply
array for what neither decides. eeClampMode 3 now emits the Booth term
alone, three instructions off ft's mantissa; a new eeClampMode 4 keeps
the other two. Nothing else differs between them.
Mode 4 reaches the config through the GameDB and the INI and has no
picker entry, but the front ends still write the bit: ApplySanityCheck
rejects a config whose bits are not a whole mode, and a rejected config
falls back to the default rather than to the mode that was picked.
The harness's clamp-mode helpers set whole modes for the same reason.
A product whose tail is non-zero but smaller than the array's 2^15
borrow still loses an ULP, and ft alone does not say which of those it
is; only reconstructing the truncated low columns does. Mode 3 guards
for that band and calls eeMulOneUlpLow, which the interpreter already
reaches inline.
The call is plain AAPCS from inside a block, and the only one this file
emits, so it carries an island: the allocator's live caller-saved homes
are spilled around it and the EE pin mirrors go through their
flush/reload pair.
A slot does not say which format it holds, and the global that does
moves with eeClampMode, so a snapshot decoded under a mode other than
the one it was taken in returns a different number for every non-zero
word. Two harnesses alive at once is enough to reach it: an interpreter
leg read after a mode-3 leg has run.
The format now travels with the bytes, the FPRs are read through the
snapshot rather than through FPRreg::Word(), and a restore re-encodes
them for the file it is going into.
The predicate was a Cmtst against a mask parked in d10, which can
express the Booth term and nothing else, so iFPUd came back one ULP high
wherever the boundary term at the truncation column was the deciding one
-- a class the interpreter models. Building the predicate in GPRs
instead admits both terms, and turns the old shape's implicit tail test,
which worked only because FPUFPCR holds round-toward-zero, into an
explicit one.
Nothing reads d10 after that, so it leaves the reserved list and the
allocator has its callee-saved slot back.
iFPUd widened every operand with ToDoubleFrom: eleven instructions
branching on exponent field 0xff, since a PS2 single with that exponent
is an ordinary large number IEEE reads as Inf or NaN. Against a
relocated slot the widening is one Fmul, and a compare goes from 34.75
host instructions to 8.75. CVT.W loses its NaN fix-up the same way,
leaving Fcvtzs.
Which format the file is in follows the clamp mode, so modes 0-2 keep
the architectural word in the low half. FPRreg's accessors read the
format and the emitters read the mode; the two engines meet at the
boundaries the word is still observable at -- MFC1, MTC1, MOV.S, the two
conversions, LWC1, SWC1 and the allocator's fill and spill.
The unscale factor takes q11 and d10's mask moves to the relocated
layout, so the allocator's callee-saved range is q12-q15.
The header called the range's size the minimum because a 3-operand MMI
op needs three homes at once. The floor is three; the slots above it
keep an FPU multiply-accumulate's four FPR homes inside the
call-surviving range.
EeFuzz.CalleeSavedNeonBudget reads high-water marks of the per-op needed
set from a test-build probe in _clearNeededNEONregs.
The slot is 64 bits and the architectural register is 32, so a word view
of the union is only right while the two coincide. Word() and SetWord()
replace f/UL/SL and are the identity, so nothing that runs changes; what
it buys is that the compiler names every place that reads a slot as a
word, which is the set the next commit has to relocate.
The x86 tier is not built here and still reads the members directly.
EeFpuFormat.h defines the format and AsmHelpers.h emits it. Widening a
slot already held in it is one Fmul against a parked 2^896, where mode
3's ToDoubleFrom branches on exponent field 0xFF. Nothing computes in it
yet.
Arm64JitBuffer.h is arm64_emit_test.cpp's MAP_JIT buffer, unchanged,
moved out so the relocation's emit tests can call the code they emit
too.
The word stays in the slot's low half. Nothing but the stride moves.
fpuRegisters was itself the savestate's wire form; fpuRegistersWire is
now, and the freeze boundary converts.
The clampE probe compiled MADDA then MADD, which leaves mVU_FMACd - the
third emitter body the abi 17 clamp change touched - pinned by nothing.
A second two-op program covers it, and mVU_FMACb's other opType with it.
Rs == Rt+1 puts the two sources next to each other in the GPR array, so
a 16-byte read at &GPR.r[Rt] + sa funnelled them without a TBL. It was
the slower of the two sequences on both cores at every sa, and worst on
the twelve adjacent pairs that have a pinned source - the register range
a funnel-shift memcpy loop uses.
armFlushEEGPRPin had no other caller.
The sa mask stays, but it no longer guards a host address. Every writer
of SA - MTSA, MTSAB, MTSAH - already keeps four bits, so the two
oversized-sa tests could not fail; one goes with the path it named and
the other says what it covers.
502 VU0 macro-mode cases of DIV, SQRT and RSQRT over zero, denormal,
normal, exponent-255 and saturated operands, scored on all four engines.
The three commits before this one each fixed one thing an engine had
wrong; this holds the whole grid rather than the operands that happened
to expose them.
DIV earns its place by separating the two rules those commits turned on:
its Invalid never comes from a sign, and its saturated quotient takes
the xor RSQRT's cannot. An RSQRT-only table would fit either rule.
Q is scored by class rather than by row, with exact per-engine tallies.
The arithmetic gap under it is a separate piece of work, and this is
where it gets a number to move.
vu_rsqrt_divisor_sign_tests states the same rules on hand-picked
witnesses, where they can be read instead of counted.
_vuRSQRT and recCOP2_VRSQRT took divide-by-zero from the divisor and
then added Invalid when the dividend was zero too, so 0/0 came back with
both standing. DIV has never done that on either engine, and mVU_RSQRT
does not either: the causes are exclusive, and the console gives Invalid
alone.
The quotient goes with it. 0/0 was the one zero divisor answered with a
signed zero rather than the saturation value, which is another place the
two modes disagreed.
_vuSQRT and recCOP2_VSQRT decide it with an ordered compare against
zero, which -0 passes; the negative denormals reach them already flushed
to -0 and pass too. mVU_SQRT tested the sign bit and was the only one
raising Invalid for either. The console raises it for all of them.
The compare had a second effect on the macro side, since an unordered
compare is not "greater or equal": a positive operand with exponent
field 255 raised Invalid there and nowhere else.
Three engines, three routes to the same missing bit. _vuRSQRT and
recCOP2_VRSQRT returned from the zero branch before the divisor's sign
was ever tested; mVU_RSQRT did test it first, then assigned the zero
branch's divide-by-zero over the Invalid it had just set.
The saturated quotient's sign went the same way. _vuRSQRT and
recCOP2_VRSQRT xor the operand signs as DIV does and mVU_RSQRT does not,
so the two modes disagreed on every negative zero divisor.
vu_sticky_console_conformance carried four rows recording the flag half
as a known divergence against its capture; they go. mVU_RSQRT changing
shape bumps kMvuCompilerAbiVersion, which evicts every on-disk VU
program cache.
A divisor that is negative and has a zero exponent field is the only
operand class in which the EE's two divide-unit causes can co-occur, and
the corpus has no row in it. A probe put 772 cases on an SCPH-90000; the
table is that capture.
It runs the same operands as sqrt.s, as div.s and as the two-instruction
sequence, so the composition the flag rule follows is asserted rather
than described. Each tier's divergence is asserted as itself: mode 3's
one ULP on a top-binade divisor, the fast path's saturation, and the
top-binade operands it hands to ee_fpu_top_binade_console_tests.cpp.
The divisor's exponent field decided the whole instruction: the zero arm
returned before the sign test, so -0 and the negative denormals came
back carrying the division's cause alone. Hoisted, on the interpreter
and on the fast path. fpuFullMode already tested the sign first and is
untouched.
DenormalDivisorTreatedAsZero asserted the shape the other two shared.
The file's divisor coverage stopped at fMax, so the words the host reads
as an infinity or a NaN had no rows at all.
The second test needs no console column: RSQRT.S and a SQRT.S followed
by a DIV.S round the same root the same way, so they have to agree
wherever the prescale applies.
SQRT.S got a prescale for exponent-255 operands when its own emitter was
fixed. RSQRT.S has a square root of its own inline and never got one, so
it was the last place on the fast path handing those words to Fsqrt.
Every FCR31 assertion in the file put the op alone in its block, which
is the one shape where there is no resident FCR31 to write back over it.
The new rows seed the same four flag states through a CTC1 instead, and
keep the memory-seeded leg beside each one so the pair fails
asymmetrically.
The rest of the COP1 family reaches fprc[31] through fpuTryAllocFCR31,
so a CTC1 or a compare anywhere earlier in the block leaves the register
living in a GPR until the block ends. RSQRT.S kept its own
load/modify/store against memory, and the slot's writeback at the seam
put the old word back on top of it: the clear of I|D and the raise of
I|SI or D|SD were both dropped whenever something in front of the op had
made FCR31 resident. It was the last emitter on the fast path still
going to memory.
The file carried the console's result word for its ten rows and nothing
about FCR31, which is the axis the two preceding commits move. The rows
gain a flags column, the denormal dividend gets two DIV rows of its own,
and DIV.S's clear gets the two pre-state rows that show it.
It never looked at the dividend, which is what decides the cause.
Both single-precision tiers move together: the differential fuzzer in
ee_rec_fpu_rsqrt_tests.cpp diffs the interpreter against the fast path
on exactly this operand class, so neither passes alone.
std::uniform_int_distribution is not specified down to the bit, so the
libc++ CI ran different operands from the libstdc++ one and a failure
there could not be reproduced here. Building an operand in a single |
expression added a second source of drift, its sub-expressions being
unsequenced.
The randomized test skipped pairs whose result leaves the EE's range by
testing the model's output for an exponent of 255. The model computes in
host singles under chop rounding, which saturates rather than
overflowing, so the skip never fired: a pair whose exact difference sits
above FLT_MAX was asserted against a value the model cannot hold, and
the interpreter's top-binade answer read as a failure.
The skip now reads the saturated word, and the pair that found it is a
row of its own.
Parking the mask in d10 removed the Mov/Fmov that materialised it. The
boundary term's instruction-count comparison, a few lines below, was
never updated to match.
Every existing probe compiles at the default clamp mode, where mVUclamp3
and mVUclamp4 emit nothing, so the two-step FMAC change at abi 17 moved
none of the eight existing digests.
maddClampE compiles MADDA then MADD through a new helper that switches
on vuClampMode:2. What it covers, and why two ops rather than three, is
at the call site.
MADD, MSUB and the A-forms emit a multiply step and an accumulate step.
The multiply step ends with mVUclamp4 on the product, and the accumulate
step opens with mVUclamp3 on the same register. Under vuClampMode:2 that
is one dead clamp in the six the pair emits.
The accumulate step now takes a mask naming the operand its caller
already clamped. The two steps are adjacent in one emitter body, so
nothing can clobber the product between them.
kMvuCompilerAbiVersion 16 -> 17, so on-disk caches holding the old shape
evict.
The row and the divUnit probe arrived together in a previous commit, but
the change they describe - writeQreg flushing a denormal quotient to
signed zero - is not in the tree. writeQreg is still a bare Ins and both
DISABLED_*DivUnitFlushesDenormalQToSignedZero tripwires still fail when
run, so the row pinned a shape this repo has never emitted.
Nothing selects row 17 while kMvuCompilerAbiVersion is 16, so the
mismatch was invisible until something bumped, and then it surfaced as a
divUnit drift against whatever unrelated change did the bumping.
Repin it to what the emitter produces. divUnit becomes a live check for
the first time either way: every row at 16 and below carries 0 there,
which the guard skips.
recJR/recJALR were the only x86 TrySwapDelaySlot callers the arm64 port
did not carry over, though every EE branch here already goes through the
same table. A slot the table clears now runs before the jump instead of
between the target capture and the block tail.
EeRecJump's two SwappableDelaySlotHoisted tests already described this
behaviour; on arm64 nothing had been doing it.
recJR/recJALR stored the jump target to cpuRegs.pcWriteback before the
delay slot and SetBranchReg loaded it straight back, so every guest
return and indirect call went through memory. ARM64TYPE_PCWRITEBACK and
its writeback case were already in the allocator for exactly this and
had no users: the port kept the support and dropped the allocation that
x86's recJR/recJALR make.
Park the target in that slot, and have SetBranchReg read back whichever
of the register or its memory spill ends up holding it.
The added tests cover the spill arm. Reaching it takes a delay slot
carrying an interpreter seam, so nothing else in the suite goes there.
QFSRV used the same store-and-reload trick x86 needs for a
cross-register byte shift. A two-register TBL indexed by a byte ramp
plus a broadcast sa does the same shift without the reload - recQFSRV's
comment has why the reload, not the spill, was the expensive part.
The ramp lives in the register pack next to the COP2 constants; R5900.h
explains why.
The two AX-03 tests keep their assertions and change their names: the
buffer they were named after is gone.
Inspired by Whatcookie's work on arm64 for rpcs3.
Both comments predate GetCPUTicks() reading CNTVCT_EL0 and give a tick
scale this host does not have; one of them leaves a plain tick count
looking like a duration. The code under them already divides by
GetTickFrequency(), and is unchanged.
Each of the three spin-then-sleep semaphore loops watches a single
atomic word, but ShortSpin() has no way to know that: it spends its
share of SPIN_TIME_NS in batches of eight isb, and every one of those is
a pipeline flush. arm64 can watch the word itself, so ShortSpinOn()
does, and the loops that hold their whole predicate in one word take it.
Hosts that cannot watch an address keep the old spin.
Inspired by Whatcookie's work on arm64 for rpcs3.
The recurrence returns the truncated quotient or one above it, and on a
large share of operands the remainder alone already says which.
eeDivideSignificand() and eeSqrtSignificand() now recognize those with
one integer division or one integer square root and return, so they no
longer run 24 digit steps. The frame, the caps, the share this covers
and which way the implication runs are at eeDivideCap().
The caps are the ones that used to ship in eeDivideTruncates(), which
the digit recurrence replaced.
eeISqrt48() comes back for the square root's half, its body unchanged
from the version the recurrence retired.
Both loops branched on the digit twice per step to choose the addend.
The selector's two comparisons are already the masks those selects want,
so it returns them and the digit is never rebuilt as -1/0/+1 inside the
step; why that form is at EeSrtDigitMask.
Square root takes the same shape, with both candidate addends and both
candidate roots built off the root before the digit arrives. That also
drops its integer digit.
No result changes.
DIV.S, SQRT.S and RSQRT.S applied a partial truncation law and fell
through to a correctly rounded host divide, which left every operand the
law did not reach one ULP away from the console. The unit is not a
rounding rule at all: it is a radix-2 SRT digit recurrence, so
eeDivide() and eeSqrtBits() now run the digits. What that recurrence
is, and what it was measured against, is at eeSrtDigit().
The recurrence is not ours: it is PS2Float.cpp's Div() and Sqrt() from
GitHubProUser67's proposed PCSX2 soft-float series, carried and since
revised in the pcsx2-reliquary fork, whose only documentation is a DOI.
The three ops are integer now, so ScopedDivRoundMode and eeISqrt48 go
with them and FPUDivFPCR's rounding mode no longer reaches the
interpreter.
The divide-unit tests move with it. They were written around a model
with a residue: the two tripwires are enabled, the console tables assert
the console on every cell, and the engine differentials now pin the
shape of the one-ULP divergence that is left rather than the region the
old law settled.
The interpreter modelled the one-ULP deficit with a closed form over
ft's mantissa. That form only holds where the exact product is
representable: once there is a tail below the ULP the decision needs fs
as well. Those rows came back IEEE.
Replace it with the array - Booth recode, carry-save tree, truncated low
columns - and read the decision off the column the truncation lands on.
eeMulDefectiveFt goes away, so the interpreter now models a superset of
what the arm64 emitters do. The emitters keep their cut of the closed
form; the comment in iFPUd-arm64.cpp says why. The randomized
differential classifies rows by the tail below the ULP and licenses a
second divergence, and a new console-row test pins the class the closed
form could not reach.
A COP1 result strictly below 2^-126 and not zero is not always flushed:
the add/sub family keeps the mantissa bits where normalisation left them
and forces the exponent field to 0, while the multiplies and the divide
clear them. The 72 console rows, how they were sampled and how they rule
out flushing and the true denormal value, are in
ee_fpu_underflow_console_tests.cpp.
Only the mode-3 recompiler had it, so on every one of these inputs the
interpreter was the less accurate engine. eeRoundToSingle takes an
addsub flag and eeGuardedAddSub is what passes it, which covers all
eight add/sub-family ops. The underflow fold also moves ahead of the
(float) cast.
The single-precision fast path computes in single and cannot hold these
values at all, so it is pinned as a divergence.
EeFpuGuardedAddSubConsole's randomized test caught the interaction on
its own. Its model computes in host floats, which FZ has already
flushed, so it cannot express the new rule; underflowing pairs are
skipped there and counted.
Nothing covered ToPS2FPU_Full's `addsub` arm, where a full-mode result
that lands below 2^-126 without being zero keeps the double's mantissa
bits instead of flushing. Rows for the eight ops that reach it (ADD,
SUB, ADDA, SUBA, MADD, MSUB, MADDA, MSUBA) and for MUL/MULA, which take
addsub=false and flush.
The rows are not a claim about silicon: no arithmetic op in the hardware
corpus has a denormal result, on hardware or on either engine, so
nothing can referee the model. They pin what mode 3 does today. The
block comment carries the model, the x86 provenance and how the other
two engines differ.
The previous commit moved eeSqrtBits() to integer arithmetic, which
turned FPUDiv.Roundmode from something the host FPCR delivered for free
into something the function reads and acts on itself. Nothing in the
suite covered that.
Four operands, one per case the truncation law and the rounding mode can
land in, plus an assertion that two of the four rows move with the mode
at all.
The comment at eeSqrtBits() now covers the one mode where the law and
the mode disagree, toward-positive-infinity, and which of the two wins
there.
Eleven rounds of console captures went looking for the whole rounding
rule of the EE's divide/square-root unit and did not find it. One
fragment of it is settled: past a per-branch bound on how far the exact
result sits below the upper candidate, the unit truncates. eeDivide()
and eeSqrtBits() now apply that fragment and keep the correctly rounded
answer everywhere else. A fragment can ship because the implication runs
one way. The frame, the caps and the captures behind them are in the
block comment at eeDivideTruncates().
Both recompilers keep the host's correctly-rounded fdiv/fsqrt, so the
interpreter now leaves them behind on those rows. The three fuzz
differentials stop asserting that the engines agree and assert instead
the shape they may differ in.
Two rsqrt.s rows of the console capture regress, and are named in
ee_fpu_divunit_console_tests.cpp. Both were right by cancellation:
a root one ULP high, then a division that rounded up to the word silicon
reached by truncating a smaller divisor. Modelling the square root
removed one half of the pair; the other half is a division with u =
6,884,762 against a cap of 10,043,841, inside the region the law does
not settle. Both rows now have a sqrt.s column that matches silicon
- the console's sqrt.s 4938608B is 445941C1, and the tree produces
exactly that.
SQRT_S now calls eeSqrtBits(). It carried its own copy of the zero case
and of the exponent-255 prescale, which is the drift that helper existed
to prevent. The prescale is retired with it: |Ft|/4 and the doubled
result were only ever a way to keep an ordinary EE binade inside a host
single, and in integers exponent 255 is the k = 23 path.
ScopedDivRoundMode goes too, since the integer path reads FPUDivFPCR
directly.
ee_fpu_divunit_exhaustive_tests.cpp is new. It pins what the exhaustive
captures rule out - the unit is not a rounding rule, and the decision
is not a function of (branch, divisor, u, nu2(T+1)) - next to the
silicon witnesses for the part that shipped.
rsqrt(EEMAX, EEMAX) is not an RSQRT question. Two captures on the
SCPH-90000, 2231 operand pairs run through sqrt.s, rsqrt.s and div.s,
say two things about the unit underneath it.
RSQRT.S is sqrt-then-divide and nothing cleverer: rsqrt.s Fs, Ft came
back bit-identical to div.s Fs, S with S the sqrt.s Ft silicon had just
produced, on every row of both captures, with a plain 24-bit single in
between. That is what RSQRT_S already computes, and it is now pinned.
Neither step is correctly rounded. sqrt.s lands one ULP low on a large
minority of arbitrary operands and never high, div.s misses in both
directions, and rsqrt.s compounds the two into two ULPs. The error is
deterministic, but it is not a rounding mode and not a function of
either operand alone, so there is no cheap model to adopt.
ee_fpu_divunit_console_tests.cpp carries 87 console rows and the tallies
behind all of this. It asserts the sqrt-then-divide decomposition in the
engine, the 193 op-rows where the unit is exact, the 68 where it is not
and by how many ULPs, and that the fast path does not drift off the top
binade. DISABLED_InterpMatchesConsoleOnEveryRow is the tripwire a real
model of the unit would have to pass; run today it fails on exactly
those 68 assertions.
The RSQRT_S comment blamed the row on the two-step rounding above it; it
is corrected to what was measured.
The exponent-255 clamp came off the EE arithmetic in previous commits
and was left on the compares, on the grounds that clamped and unclamped
compares can only differ when both operands are in the top binade and
that no captured row is like that. That reason went into FPU.cpp and it
is wrong: one clamped operand is enough, because it only has to collide
with an operand that already is 0x7F7FFFFF.
C_cond_S reads eeToDouble() now, and that was fpuDouble()'s last caller
with anything to do. Its only other site was SQRT_S's else branch, where
the exponent is 1..254 and both of its rewrites are the identity, so
fpuDouble() and fpuOperandBits() are deleted and eeToDouble() is the
single operand entry point in the file.
The fast path keeps its clamp: it compares in host singles, where every
top-binade word is a NaN and an unclamped Fcmp goes unordered. What
making it exact would cost is at ee_fpu_compare_console_tests.cpp.
That file transcribes all 86 compare rows of the capture: every c.f.s,
c.eq.s, c.lt.s and c.le.s case, with the expected FCR31 put through the
same mask CFC1 applies, so each row is a flag test as well as a
condition test. It asserts the interpreter, fpuFullMode
(DOUBLE::recC_*_xmm, which does not clamp) and the fast path separately,
the last including the four rows it is expected to miss.
EeRecFpu.CEqSInfinityClampsToMax asserted CC set for c.eq.s(2^128,
FLT_MAX), the clamp's answer taken for the PS2's. It is rewritten as
CEqSTopBinadeSplitsTheInterpFromTheFastPath, which asserts both tiers
and says which one is the console's. Same inverted-premise shape as the
four MAX/MIN tests and MulSFpuMulHackOffGivesNativeProduct.
CEqSPositiveNaNBothClampToMax becomes CEqSPositiveNaNIsEqualToItself: it
passes because a bit pattern equals itself, not because of any clamp.
Every other FPU test in this directory measures one instruction, so none
of them reaches the later instruction the fast path's result clamp
(fpuClampResult) exists to protect. These rows are producer/consumer
pairs, which do.
The capture, the console rows read off it and the per-engine scores are
in the new file's header. An exponent-255 word turned out to be an
ordinary finite number to every consumer measured, so there is nothing
for the clamp to keep out of the register file.
What it does instead is hide absorption rather than prevent it: `mov
EEMAX` then `x*0` still goes host-NaN*0 -> NaN inside the fast path, and
the clamp folds that to 0x7F7FFFFF where the console says 0, a plausible
wrong answer instead of an obvious one. It only looks protective where
the producer was itself an arithmetic op whose own clamp had already
saturated.
Deleting it scores better and is still not the answer; the trade is at
EeFpuCascadeConsole.FastPathNeverInventsAnAbsorbingNan. Hence no code
changes here.
The comment above the -ffp-contract=off line claimed
# -ffp-contract=fast is fine on x86 (no FMA emitted without -mfma)
and that is false: -mfma is not the only route to the instruction. The
default non-multi-ISA build takes -march=native
(cmake/BuildParameters.cmake, the DISABLE_ADVANCE_SIMD=OFF branch),
which supplies FMA on any host that has it. Upstream PCSX2 v2.7.508
built exactly that way on amd64, emits
MADDA_S 0x5084ae: vfmadd213ss 0x590(%rdx),%xmm1,%xmm0
MSUBA_S 0x5088be: vfnmadd213ss 0x590(%rdx),%xmm1,%xmm0
so their EE FPU interpreter produces different results in a dev build
than in the multi-ISA release builds. Left standing, that sentence is
the reasoning that narrows this line to if(ARCH_ARM64) the next time
someone tidies up.
Also brought the two halves up to date. FPU.cpp has nothing left to
contract: two previous commits replaced its host-float arithmetic with
an explicit model (fpuAddSubGuarded, eeMulAccumulate), so the `_FAValf_
+= fs*ft` hazard the comment described is gone, and contracting it today
moves none of the 1147 capture cases. The VUops.cpp half points at the
SCPH-90000 capture for what the console does, instead of asserting that
the hardware has no FMA.
Comment only; the compile options are unchanged.
VUops.cpp is compiled with -ffp-contract=off (pcsx2/CMakeLists.txt) so
the compiler cannot turn `acc + fs * ft` into a single-rounded fmadd.
The only test behind that line was
EeVu0Cop2Macro.Jak3CameraBasisKernelChainMatchesInterp, which compares
the two engines to each other and so cannot say how many times a
PlayStation 2 rounds.
A capture on an SCPH-90000 settles it over ten operand triples, eight of
which tell a fused accumulate apart from two separate roundings. The
console took the two-rounding answer on all eight, and both engines
already reproduce all ten. The operands, the three candidate results and
the two control rows are in the new test file.
These two were the last arithmetic reading their operands through
fpuDouble() and saturating at posFmax, so the EE's top binade could not
reach them. eeDivide() normalizes both operands and reassembles the
exponent, and eeSqrtBits() gives RSQRT the exponent-255 prescale SQRT.S
already uses. Why neither can take the eeToDouble() route the rest of
the family took is at eeDivide.
Two results move with them. Saturation goes to the EE's own maximum
0x7FFFFFFF, a binade above the fast path's FLT_MAX, so the tiers now
differ by design and the tests that used to diff them run their legs
separately. And RSQRT's zero-divisor sign comes from Fs rather than Ft,
matching upstream x86 recRSQRThelper1 and the console; DIV's xor of the
two operands is untouched.
The interpreter column of
EeRecFpuFull.RsqrtAboveEeMaxSaturatesInsteadOfWrappingToNegativeZero
moves to 0x7FFFFFFF with it, and the zero-divisor and RSQRT tripwires
that named this commit as their condition graduate.
Exponent 255 is an ordinary exponent on this FPU: 0x7F800000 is 2^128,
0x7FFFFFFF is the largest number the machine has, and there is no Inf
and no NaN. That is one binade above IEEE single, so fpuDouble() folded
every such operand down to +/-0x7F7FFFFF on the way in and
clampToEeRange() folded a host infinity back to the same word on the way
out.
ADD.S, SUB.S, MUL.S, their A-forms and the four multiply-accumulates now
read their operands through eeToDouble() and round once through a new
eeRoundToSingle(). Why the double arithmetic in between is exact, and
how the top binade is rounded with no host single to round it in, is at
eeGuardedAddSub, MUL_S and eeRoundToSingle.
The multiplier's one-ULP deficit had to move across with it or it would
have been lost here: it sat on top of fpuDouble() and read the operands
as clamped, which changed ft's mantissa and so changed its own
predicate. eeMulRound applies it to what eeRoundToSingle produces
instead, and the six multiply forms reach it through eeMulProduct's
replacement, eeMulAccumulate.
One behavior changed beyond the range: an overflowing product now ends a
MADD/MSUB, value as well as flag, where the value path used to carry on
accumulating and land a binade away.
The fast path is untouched. It computes in host singles and cannot hold
these values at all; the warning against changing posFmax to suit it is
at the top of ee_fpu_overflow_console_conformance_tests.cpp.
DIV and RSQRT still read operands through fpuDouble and still saturate
through checkDivideByZero's posFmax. A quotient of two singles is not
exact in a double, so the eeToDouble route would double-round under the
divide unit's round-to-nearest; they need the normalise-and-reassemble
treatment the next commit gives them.
The EE adder carries no guard bits to the right of the mantissa, so
a cancelling add or sub comes back one ULP toward zero from the IEEE
answer. x86 FPU_ADD_SUB and both arm64 emitters have masked for that for
years; the interpreter did not, which made it the less accurate engine
on this class.
fpuAddSubGuarded() ports the recompilers' mask into ADD.S, SUB.S,
ADDA.S, SUBA.S, MADD.S, MSUB.S, MADDA.S and MSUBA.S, unconditionally: it
does not read CHECK_FPU_GUARDED, so with fpuGuardedAddSub=false the two
engines disagree here by design, and
EeRecFpuGuardBit.GuardOffDivergesFromInterpreterByDesign asserts both
sides so that is not closed by accident.
fpuDouble() now reads the shared fpuOperandBits(), and MADDA.S/MSUBA.S
name their product instead of accumulating in one expression; the
comment at MADD_S has the rest.
ee_fpu_guarded_addsub_console_tests.cpp is the new console pin for the
class: the capture rows the port moves, the eight family members,
aliasing, a randomised sweep against the x86 model on both engines, and
the FCR31 axis it must not disturb.
Not fixed here: fpuOperandBits() still clamps an exponent-255 operand to
+-0x7F7FFFFF before the adder sees it, so a row with such an operand
gets one step closer to the console without reaching it. That is the
operand-clamp family SQRT.S already left (1a09344ba6).
checkOverflow() asked whether the result had come back as a host
infinity and checkUnderflow() whether it was a host denormal. Neither
ever appears under the FP environment a game runs in, so O and U were
raised only under a rounding mode no game selects: 18 of the 57 rows in
the first-party capture read back with the flags clear where the console
raised them.
Decide from the magnitude of the exact result instead, computed in
double by a new eeToDouble(). The multiply-accumulates round twice and
so raise twice; the block comments at raiseOrClearOU() and
madAccumulandOverflowed() carry the silicon rows the rule was read off.
The value path keeps the old helpers' folding, now in clampToEeRange(),
so this is a flag change end to end.
The family table in ee_fpu_fcr_console_conformance_tests.cpp called all
ten of its "overflow" rows overflows. Eight are +/-FLT_MAX sums landing
exactly on 0x7FFFFFFF, which is representable and which silicon does not
flag: the four adds move to the clear class, and the four
multiply-accumulates get operands that overflow the product instead. The
raise class is six multiplies and two underflows, and the clear class
grows to one row per emitter that owes a clear.
Because the answer no longer depends on the rounding mode, the two
DISABLED tests that said opposite things about that table in the two FP
environments are gone; ExceptionFlagsMatchConsoleExceptTheFastPathRaise
replaces them and runs both. EnginesAgreeOnTheOverflowFlagClear,
NanMathOverflowIsNotAnOperandClampModeDifference and
OverflowFlagsComposeAcrossOneBlock graduate from DISABLED, and
InterpreterRaisesOverflowAndUnderflowLikeTheConsole and
FullModeMatchesConsoleOnEveryRow are new.
DISABLED_UnderflowFlagsNeedFzOff stays disabled: what is left in it is
the denormal-result value, which no capture can settle, since the
console's FPU has no denormal results to capture.
The fast path is not touched. It clears O and U and raises neither, so
the tests pin its FCR31 as the interpreter's minus exactly O|U|SO|SU
rather than as a row list. A raise there needs the double arithmetic
fpuFullMode already pays for; the reverted fast-path emitter took the
cheap route and derived O from a host-Inf predicate, so it was
a function of eeRoundMode and fired on rows that are not overflows, for
+8 host instructions on every arithmetic op.
Exponent 255 is an ordinary binade on the EE - no Inf or NaN, and the
representable max is 0x7FFFFFFF rather than FLT_MAX - so an exponent-255
operand never needed saturating. The interpreter clamped it to
+/-FLT_MAX inside fpuDouble anyway, and landed short of the console:
sqrt.s 7F800000 -> 5F7FFFFF, silicon 5F800000
sqrt.s 7FFFFFFF -> 5F7FFFFF, silicon 5FB504F3
sqrt.s 7FC00000 -> 5F7FFFFF, silicon 5F9CC471
It now computes sqrt(|Ft|/4)*2, which recSQRT_S_xmm already emits. Why the
factor is 4 is at SQRT_S in FPU.cpp.
The two conformance tests that pinned the clamp pin the console value
instead, over every exponent-255 shape rather than the three the capture
holds.
Both this branch's engines gated SQRT.S's I|SI on `exp != 0 && sign`.
The console gates on the sign bit alone, so -0 and the negative
denormals, which flush to -0 and produce an ordinary +0, raise
invalid-operation there too. Those were the only two operand classes
that lost the flag.
From the first-party capture that records FCR31 alongside the result:
sqrt 80000000 : console 00000000/01020041 both engines 00000000/01000001
sqrt 80000001 : console 00000000/01020041 both engines 00000000/01000001
The rule holds across all 38 SQRT.S rows. A positive qNaN raises
nothing.
The fix is a deletion on both sides: the Tst(0x7F800000)/B.eq pair in
the arm64 fast path, and the flag set hoisted out of FPU.cpp's
negative-normal arm. x86's recSQRT_S_xmm has always tested MOVMSKPS's
sign bit alone (iFPU.cpp:1767), which is why upstream-x86-jit is the one
column in the capture that answers both rows correctly; the arm64
FULL-mode DOUBLE::recSQRT_S_xmm tested the sign alone too and was
unaffected.
A previous commit fixed the value half of a case where the interpreter
returned _FtValUl_ & 0x80000000 and so answered -0 where the console
answers +0.
EeRecFpu.SqrtSInvalidFlagFollowsTheSignBitAlone is the ten-row sign x
exponent matrix from the capture, each engine scored on the full FCR31
word. The flag half of EeRecFpu.SqrtSOfNegativeZeroIsPositiveZero moves
there; it had asserted the opposite on nothing but the two engines
agreeing.
Two leads from the same sweep, recorded in the capture's handoff and not
addressed here. The arm64 JIT reports 01000001 on every RSQRT.S row in
the capture, including ordinary-negative operands, where interp, x86-jit
and hardware all say 01020041; recRSQRT_S_xmm does contain the I|SI set,
so that looks like a lost flag write rather than a missing one. And
hardware raises I|SI, not D|SD, on RSQRT's 0/0 rows.
A 128-case first-party console capture for the three remaining areas of
the adjudication request that had no oracle: how far downstream a Q, P
or flag result is readable, VU data-memory addressing at the top and
across the VU0->VU1 window, and whether an XGKICK packet read wraps.
autocases_vulat.h carries the capture; the headers of
vu_pipeline_console_conformance_tests.cpp and
vu_memory_xgkick_console_conformance_tests.cpp list what it established,
in the order the tests assert it.
Every number and every rule we already model came back correct, apart
from the two below.
Two divergences are pinned as DISABLED tripwires, each naming its cause
at the test: InterpreterPublishesStickyDivideFlagsEarly (the interpreter
publishes the sticky half of a divide-unit cause bit at distance 5,
where console and microVU publish it with the quotient at 7) and
Vu0HasNoElementaryFunctionUnit (we dispatch the EFU family on VU0 and
model a P register for it; hardware has no such unit).
A 61-case first-party console capture for four areas that had no oracle:
the branch terminators, the integer pipe, MAX/MINI and the clip
register. autocases_vubranch.h carries the capture; the header of
vu_branch_console_conformance_tests.cpp lists what it established, in
the order the tests assert it.
Three known divergences are pinned as DISABLED tripwires, each naming
its cause at the test: MacUnderflowBitMatchesConsole (both engines drop
the MAC underflow bit after a flushed product), DBitStopMatchesConsole
(the arm64 recompiler does not implement the D bit) and
InterpreterRunsDelaySlotBeforeTraceStop (the interpreter stops a pair
early and never runs the delay slot).
The only hardware evidence for the EE's top exponent was ps2autotests'
fpu/fcr.cpp, which runs MUL.S(0x7F7FFFFF, 0x7F7FFFFF) but prints the
result with %f and so records the string "NaN". This captures the bits:
57 EE FPU rows and 8 VU0 macro-mode rows from a real PS2 over ps2link,
every value a raw word. The rule they establish is at the top of
autocases_fpuovf.h; the generator checks it in exact rational arithmetic
across all 47 arithmetic rows.
Every console divergence in the capture is shared by both engines and
deliberate, since the fast path cannot hold the EE's top binade at all,
so the conformance test carries the console column as data, classifies
the divergences at the top of the file, and leaves them to the
hardware-alignment stage.
One gap is not deliberate: recSQRT_S_xmm is the only emitter in
iFPU-arm64.cpp that never clamps its operand, and no clamp mode closes
the rows it splits. Recorded as a divergence with a DISABLED tripwire,
not fixed here.
pcsx2/FPU.cpp maintains FCR31's O and U on every EE FPU arithmetic op:
checkOverflow saturates an infinite result to +/-fMax and raises O with
the sticky SO, clearFPUFlags takes them back down. The recompiler's fast
path maintains neither, so an overflowing MUL.S leaves FCR31 reading
0x01000001 where console says 0x01008011. The interpreter is the side
that matches the console, so the recompiler is the side that has to
move.
kFamCases sweeps the whole class, every arithmetic op that touches O or
U, rather than the two rows the capture happens to reach. Five tests go
in DISABLED, because the emitter that would have closed them was
measured and reverted; what it got wrong and what a redesign owes are in
the comment above DISABLED_EnginesAgreeExceptOnTheOverflowFlags.
DISABLED_UnderflowFlagsNeedFzOff holds the U half, which needs FZ off
and brings the denormal-value question with it.
The one cross-engine row left, "NAN math", is not about the missing
flags: ADD.S on two raw exp-255 words has the engines computing
different things before any flag logic runs, and
CHECK_FPU_EXTRA_OVERFLOW aligns it exactly.
DISABLED_NanMathOverflowIsAnOperandClampModeDifference attributes it to
the operand-clamp mode axis instead of leaving it unexplained in
kFcrEngineDivergences.
Every console-conformance corpus scores each engine against the hardware
capture separately. That is deliberate - a JIT-vs-interp differential is
blind to what both engines get wrong together - but it left the
mirror-image blind spot: nothing asserted the engines match each other.
Two engines returning different wrong answers get recorded as known-bad
twice and look settled. That is how 111 EFU divergences hid.
The MMI, SA/branch/perf and EE LSU corpora get a plain agreement test
with no allowances. FCR31 gets one with two recorded divergences:
outside the EATAN family b55b1b7510 already lists, it is the only corpus
where the engines differ, and on both rows the interpreter is the side
that matches silicon, returning 0x01008011 where the arm64 recompiler
returns 0x01000001 and drops O and SO. FPU.cpp raises the pair from
checkOverflow(), iFPU-arm64.cpp models no FCR31 exception flags at all,
and the x86 emitter's FPUflagO/FPUflagSO writes are commented out, so
the fix belongs in the recompilers.
The split only shows at round-to-nearest, which is why this was filed as
unanimous-but-wrong before.
The VU has no denormal encoding, and the interpreter models that by
running every div-unit result through vuDouble, whose exponent-0 arm
returns a bare signed zero. Both arm64 pipes write the raw quotient
instead: COP2 macro through cop2EmitSyncFDiv, micro through writeQreg.
Each is the single tail all three of DIV/SQRT/RSQRT pass through, so the
class is one site per pipe.
Deficient: arm64. Correct: interp. Two tripwires, one per pipe:
VuStickyConsoleConformance.DISABLED_Arm64Cop2DivUnitFlushesDenormalQToSignedZero
VuStickyMicroConsoleConformance.DISABLED_MicroDivUnitFlushesDenormalQToSignedZero
They replace a macro-only tripwire that had stopped tripping. It was
captured before the harness moved into the production FP environment,
where DenormalsAreZero flushes the quotient before either engine can
differ, and after that it passed silently, as a disabled test, while the
defect it named stood. Both replacements scope ScopedFpEnv{IeeeNearest}
and assert FZ clear on the register their pipe actually runs under.
A software flush was written in both pipes and reverted: the VU FPCR's
FZ already does it in every shipping configuration, and it only
half-closes the wider gap the micro tripwire records.
That attempt exposed a hole in the ABI-digest backstop: none of its
seven probe programs contains a lower-pipe div-unit op, so the whole
mVU_DIV/mVU_SQRT/mVU_RSQRT emitter, and writeQreg behind them, could be
rewritten without moving a digest. A divUnit probe now covers it.
ArithmeticStillChopsUnderTheAmbientMode was added as a negative control:
ADD.S does not belong to the divide unit, so it must keep chopping under
the ambient mode, and nothing else in the suite would catch a fix that
widened the FPUDivFPCR swap to the whole FPU.
It was dead. The operands were 1.0 + 2^-25, justified in the comment by
"the exact sum needs 26 mantissa bits, so it rounds to 0x3F800001 at
nearest". The exact sum sits a quarter ULP above 1.0, well below the
halfway point, so round-to-nearest returns 1.0 as well -- 0x3F800000
under both rounding modes and both settings of fpuGuardedAddSub. The
control asserted a value that could not change, and passed for a reason
unrelated to what it claimed to test.
New operands 1.0 + (1 - 2^-24) do separate the two modes. The test
asserts the chopped result, then re-runs the same operands under a
scoped round-to-nearest ambient FPCR and asserts the other one, so
a later edit that flattens them fails here instead of passing quietly.
DIV.S and RSQRT.S special-case a divisor whose exponent field is zero
and saturate instead of dividing. That saturated value differs from the
console in two independent ways, magnitude and sign. The capture rows
and the rules read off them are in the new file's header.
Neither is simply a bug to fix. The console value already ships, in the
FULL path, and the fast path's +FLT_MAX is a deliberate compromise. The
gap is the interpreter, which has no FULL path at all, so an
eeClampMode:3 game gets the console value from the recompiler and the
fast-path one from the interpreter.
Hence four tests rather than two. One pins the JIT's console-exact
FULL-mode result, which nothing covered before; it is the only one
enabled here. One pins the fast path's non-console value in the default
mode, so a later attempt at either tripwire cannot change that mode by
accident. The last two are the tripwires for the interpreter gap,
magnitude and sign split apart so they can graduate independently, the
sign one carrying the DIV rows as a control.
The default-mode test ships disabled as well: the interpreter still
reads DIV/RSQRT operands through fpuDouble(), so it cannot agree with
the fast path yet. "Fix: DIV.S and RSQRT.S, the last two ops holding the
operand clamp" re-enables it.
The interpreter ran DIV.S, SQRT.S and RSQRT.S under the ambient FPUFPCR
rather than the divide unit's FPUDivFPCR, so it chopped where both
recompilers and the console round: one ULP low on every inexact result
in the default mode a game runs in. It now takes the same swap, gated
the way the emitters gate it; ScopedDivRoundMode in FPU.cpp has the
rest.
The witness this was first reported against, 1.0 rsqrt 1.5 landing on
0x3F5105EC where hardware gives 0x3F5105EB, has two independent causes.
749392120f fixed one. The value came back when the harness moved to the
production FP environment and was read as that fix half-landing, "the
divide is still in double". It is not: fpuDouble returns float and the
quotient was always single.
Four EeFpuOverflowConsole cases go DISABLED here: their SQRT.S rows were
written against the interpreter's truncated values. "Tests: EE FPU
overflow against hardware" and "Fix: SQRT.S raises invalid on -0 and
negative denormals" later in the series bring them back.
It re-lands later in this series, so the FCR conformance file goes back
to covering the console capture and nothing else. Out with it:
EnginesAgreeOnTheOverflowFlagClear, the kFcrEngineDivergences list, the
FamCase table and RunFamCase, and the five DISABLED tripwires over them.
The comments around the divergence tables described an emitter that
derives all four ZSUO predicates from the raw result's exponent field,
and a MAC table emptied because of it. Neither is in this tree:
cop2EmitFlagUpdate extracts a sign bit and a zero bit per lane and
nothing else, both tables still carry their rows, and the U/O tests are
still DISABLED. Restate the comments against the code that is here, and
correct the STATUS table's row split, which was already wrong.
The micro-mode note had drifted onto kMacroMacDivergences; put it back
over kMicroDivergences.
Add a tripwire for the DIV unit, the same defect one pipe over: the
interpreter runs its quotient through vuDouble, which flushes a denormal
Q to signed zero, and the arm64 DIV/RSQRT emitters store the raw
quotient.
On the positive-divisor path the interpreter divided by the unrounded
double that libm sqrt returns, so its quotient came out 1 ULP from the
EE FPU and from both recompilers. The negative path already rounded
through temp.f; both paths now share the same two expressions. The
comment at RSQRT_S() names the recompilers and carries the operands and
both results.
PCSX2's D-cache tag holds a host pointer -- CacheLine::load stores the
translated `ppf` -- where hardware holds a guest physical address, and
DXSTG copied CP0 TagLo into that field unmasked, so writeBackIfNeeded
dereferenced a word the guest chose. It now translates the tag page the
way a fill does and takes isValidPFN from the same translation; the
comment at the case has the rest.
That widens the write-back precondition: a DXSTG naming a resolvable
page arms a line that was never filled, where before only a fill could
set bit 11. That is the hardware behaviour.
The exposure was narrow. The wild store needed isValidPFN standing from
a real fill, and fills are gated by EnableEECache, which ships off; the
tag write itself was never gated. Cache.cpp is shared, so x86 was
affected identically, and both recCACHE() bodies are empty, so the
recompilers reach this through the interpreter.
DISABLED_DxstgDirtyStaysInsideGuestMemory graduates.
Pcsx2DxstgRedirectsAHostWrite, which pinned the defect, becomes
DxstgWriteBackTargetsTheTaggedGuestPage, and
DxstgOnAnUnresolvablePageDeclinesTheWriteBack is new.
Pcsx2AcceptsDirtyFromDxstgButCannotReportIt used the redirect as its
instrument and now reads guest memory.
The Booth predicate is keyed to fixed bit positions of ft, but the
truncation column moves one bit between the binades -- and every
zero-tail row I measured sat in the lower binade, at fs = 2^23. So the
region where the decrement fires with the product in the upper binade
had never been observed on silicon. The fpmul3 capture already had it.
The counts and the row provenance are at the tables.
One dead end worth recording, since the data invites it: do not split
this by binade, split it by T. fs = 0xFFFFFF looks like the predicate
carries no information there (0.1907% low with Booth on, 0.1976% with it
off), but it has exactly one T == 0 row, so that is measuring the
unmodelled 0 < tail < deficit class instead. At fs = 0x800001, lower
binade and also almost entirely that class, Booth is strictly necessary:
13,248/13,248 on, 0/262,144 off.
emitDefectiveFmul materialised the 0x2AA Booth-digit mask on every
mode-3 multiply (mov + fmov). It is now placed into d10 for the whole
JIT session by _DynGen_EnterRecompiledCode, next to the s8/s9 clamp
scalars and under the same AAPCS64 argument: the low 64 bits of d8-d15
are callee-saved, so a pinned constant survives every C call with no
compile-time tracking. Six instructions become four:
cmtst d30, d11, d10 ; d11 = narrow guest ft, d10 = the parked mask
fmul d0, d0, d1
fcmeq d31, d0, #0.0
bic v30.8b, v30.8b, v31.8b
add d0, d0, d30
Why a pinned callee-saved register beats a caller-saved one behind a
liveness flag, and why q10 specifically, is on NEON_RESERVED_FPU_MULMASK
in iCore-arm64.h. It costs one of the six call-surviving FPR homes.
The console's multiply array does not round correctly. When the exact
product has nothing below the single's ULP to absorb it, the result
comes back exactly one step closer to zero -- and whether it does is
decided by ft's mantissa alone, so mul.s is not commutative. iFPUd, the
eeClampMode:3 codegen, computed the IEEE product; the interpreter has
modelled the law since the previous commit.
Six instructions per multiply, at both emit sites: recMULop, and
recMaddsub's multiply stage, which round through different helpers. The
mechanism, the FPCR dependency it rests on, and the one term of the
measured predicate it knowingly drops are documented at
emitDefectiveFmul.
kGuardMaskWitnesses moved on 35 of its 72 rows -- fs is 1.0 throughout,
so every product there is zero-tailed. Each was re-derived against the
measured law rather than re-pinned against the emitter that moved it;
the one row where the dropped boundary term decides is called out at the
table.
The console's multiply array is not a correctly-rounding multiplier: it
comes back exactly one step closer to zero on a large fraction of
operands, and which operands depends on operand order. Upstream states
the rule in a comment (x86/iFPU.cpp) and never tests it; FpuMulHack is
a one-point sample of it. What is modelled, what deliberately is not,
and where the rule was measured are at eeMulDefectiveFt.
All six multiply forms route their product through the new eeMulProduct,
which also localises the fpuDouble() calls the sites used to make
inline. MADDA/MSUBA gain a temp for it; the accumulate itself is
unchanged, and still does not round-trip the product through fpuDouble
the way MADD/MSUB do.
MulSFpuMulHackOffGivesNativeProduct asserted the opposite -- that the
IEEE product is the console's. Silicon returns 0x3F490FDA for those
operands and 0x3F490FDB reversed, so the gamefix matched hardware in
both orders all along.
The deficit only reaches what fpuDouble() hands it. An exponent-0xff
operand is clamped to +/-Fmax before the multiply and the product
saturates at FLT_MAX, so this engine still has no EE top binade; that
gap is fpuDouble's and is untouched here.
Every full-mode widening site paid an Fmov to copy an allocator-resident
operand into a temp before ToDouble, for one reason: ToDouble's
exponent-0xff has to place the exponent-lowered single somewhere before
Fcvt, and it parked it in the register it was reading.
ToDoubleFrom(dst, src) puts it in the destination's S lane -- a temp the
caller already owns -- so it never writes the source. ToDouble(idx) is
now ToDoubleFrom(idx, idx) and emits the same instructions it always did.
Instructions removed on the common path: 2 per MUL.S/MULA.S, 3 per
MADD/MSUB/MADDA/MSUBA (both operands plus the ACC), 2 per
C.EQ/C.LT/C.LE, 2 per DIV.S. recDIVhelper1 now takes the operand registers
separately from its two write-only temps; its zero-divisor block reads
fs/ft in place.
recMaddsub rounded the product to a PS2 single with ToPS2FPU_Full and then
widened it straight back with ToDouble thirteen instructions later. Nothing
observed the narrow form in between except the guard mask, which only reads
exponent fields -- and those are as extractable from bits 52..62 of a double as
from bits 23..30 of a single.
Two new emitters make the round trip unnecessary: ToPS2FPU_Wide rounds to EE
precision without changing format, and FPU_ADD_SUB_D is the guard mask on
doubles.
Shaves off about 5 executed instructions per MADD/MSUB/MADDA/MSUBA, not
the 19 the round trip costs: the multiply stage still needs its flag
branches, because the O flag is what recMaddsub branches on. Output is
unchanged.
pstef