armsx/ARMSX2
0
0
Fork 0
mirror of https://github.com/ARMSX2/ARMSX2.git synced 2026-08-24 16:50:16 -07:00
Code Issues Packages Projects Releases Wiki Activity

Fix FPU.cpp: model the EE divide/square-root unit's truncation law

Browse Source 
Eleven rounds of console captures went looking for the whole rounding
rule of the EE's divide/square-root unit and did not find it. One
fragment of it is settled: past a per-branch bound on how far the exact
result sits below the upper candidate, the unit truncates. eeDivide()
and eeSqrtBits() now apply that fragment and keep the correctly rounded
answer everywhere else. A fragment can ship because the implication runs
one way. The frame, the caps and the captures behind them are in the
block comment at eeDivideTruncates().

Both recompilers keep the host's correctly-rounded fdiv/fsqrt, so the
interpreter now leaves them behind on those rows. The three fuzz
differentials stop asserting that the engines agree and assert instead
the shape they may differ in.

Two rsqrt.s rows of the console capture regress, and are named in
ee_fpu_divunit_console_tests.cpp. Both were right by cancellation:
a root one ULP high, then a division that rounded up to the word silicon
reached by truncating a smaller divisor. Modelling the square root
removed one half of the pair; the other half is a division with u =
6,884,762 against a cap of 10,043,841, inside the region the law does
not settle. Both rows now have a sqrt.s column that matches silicon
- the console's sqrt.s 4938608B is 445941C1, and the tree produces
exactly that.

SQRT_S now calls eeSqrtBits(). It carried its own copy of the zero case
and of the exponent-255 prescale, which is the drift that helper existed
to prevent.  The prescale is retired with it: |Ft|/4 and the doubled
result were only ever a way to keep an ordinary EE binade inside a host
single, and in integers exponent 255 is the k = 23 path.
ScopedDivRoundMode goes too, since the integer path reads FPUDivFPCR
directly.

ee_fpu_divunit_exhaustive_tests.cpp is new. It pins what the exhaustive
captures rule out - the unit is not a rounding rule, and the decision
is not a function of (branch, divisor, u, nu2(T+1)) - next to the
silicon witnesses for the part that shipped.
This commit is contained in:
pstef
2026-08-09 11:20:53 +02:00
parent 6e46d5a67e
commit 34f757a475
6 changed files with 1384 additions and 150 deletions
Download Patch File Download Diff File Expand all files Collapse all files
pcsx2/FPU.cpp
+165 -56
View File
@@ -411,7 +411,64 @@ static u32 eeGuardedAddSub(u32 a, u32 b, bool issub)
The divisor must already be known nonzero: a zero divisor is a flag question
the callers answer first.
The console does not round to nearest; eeDivideTruncates() below covers the
part of the difference that is settled.
*/
/* The EE divider's truncation law.
The divide/square-root unit is not correctly rounded: the exact quotient
lies between two singles and the unit returns one of them, not always the
nearer, and which one it returns depends on the operands rather than on a
rounding mode. This is the part of that choice the captures settle.
Write the exact division of the two significands (hidden bit restored) as
lt = ma < mb the branch: does the quotient need a shift
num = ma << (23 + lt)
T = num / mb the truncated 24-bit significand
rem = num - T*mb 0 <= rem < mb
u = mb - rem how far the exact quotient sits below T+1
so the unit returns T or T+1 and correct rounding would take T+1 exactly
when 2*rem >= mb. The unit rounds up only when u is small:
u > cap => the unit truncates cap = 2^22 on A>=B
cap = max(2^23, mb-2^22) on A<B
Evidence, all of it first-party captures from SCPH-90000 (FCR0 0x2E40) in
captures/fpmatrix/divsqrt/ in the session archive:
* 150,994,944 rows over eighteen divisors swept exhaustively -- every one
of the 2^23 numerator significands at each -- of which 57,612,965 have
u > cap. Not one of them rounds up.
* 48,799,468 further rows of scattered, band and transverse captures:
6,937,248 distinct divisor significands, divisor exponent fields 110
through 145. Again not one violation.
Rows with u <= cap are unsettled -- 27.5% of them truncate as well, and no
model this project has built predicts which -- so they keep the correctly
rounded answer, which is also what both recompilers produce. The implication
runs one way, so applying the law can only turn a wrong row right.
Only the A>=B half of the cap ever changes a result: on A<B, u > cap already
implies correct rounding truncates. The branch is kept because it is the law
the captures give, and
EeFpuDivUnitExhaustive.TheAlbHalfOfTheCapCannotChangeAnAnswer holds its
shape so a tightened cap cannot silently become live.
Over the exhaustive set this takes the interpreter from 19.49% of quotients
off by a ULP to 15.44%, and over the scattered set from 13.98% to 9.51%, at
the price of disagreeing with both recompilers on those rows.
EeRecFpuDivUnitRounding pins the divergence to this class.
*/
static bool eeDivideTruncates(u32 mb, u32 lt, u32 rem)
{
const u32 cap = lt ? ((mb > (3u << 22)) ? mb - (1u << 22) : (1u << 23)) : (1u << 22);
return (mb - rem) > cap;
}
static u32 eeDivide(u32 a, u32 b)
{
const s32 ea = (s32)((a >> 23) & 0xFF);
@@ -420,6 +477,33 @@ static u32 eeDivide(u32 a, u32 b)
if (ea == 0)
return (a ^ b) & 0x80000000; // zero dividend, sign from both operands
// The exact frame, in integers, so the decision below owes nothing to the
// host's rounding mode. Exponent 255 is an ordinary binade on this FPU, so
// every finite operand reaches here and the hidden bit is always present.
{
const u32 sma = 0x800000u | (a & 0x7FFFFFu);
const u32 smb = 0x800000u | (b & 0x7FFFFFu);
const u32 lt = (sma < smb) ? 1u : 0u;
const u64 num = (u64)sma << (23 + lt);
const u32 T = (u32)(num / smb);
const u32 rem = (u32)(num - (u64)T * smb);
if (eeDivideTruncates(smb, lt, rem))
{
// T is already normalised into [2^23, 2^24), so nothing carries out
// of the significand and the exponent is pure integer. A quotient
// that only reaches the next binade by rounding up therefore does
// not reach it here, as on the console.
const s32 e = ea - eb + 127 - (s32)lt;
const u32 sign = (a ^ b) & 0x80000000u;
if (e > 255)
return sign | 0x7FFFFFFFu; // the EE's maximum, not FLT_MAX
if (e < 1)
return sign; // the EE has no denormals to underflow into
return sign | ((u32)e << 23) | (T - 0x800000u);
}
}
FPRreg ma, mb, q;
ma.UL = (a & 0x807FFFFFu) | (127u << 23);
mb.UL = (b & 0x807FFFFFu) | (127u << 23);
@@ -435,28 +519,90 @@ static u32 eeDivide(u32 a, u32 b)
return (q.UL & 0x807FFFFFu) | ((u32)e << 23);
}
/* sqrt(|Ft|) as EE bits, including the top binade. The exponent-255 arm is the
same |Ft|/4 prescale SQRT.S does inline below, where the reason for it is. */
/* floor(sqrt(x)) for x < 2^48, exactly. The host sqrt only seeds it: x is
under 53 bits, so it converts to a double without loss and lands within one
of the answer, and the fixup loops run unconditionally, so the result does
not depend on the host's rounding mode. */
static u32 eeISqrt48(u64 x)
{
u64 r = (u64)std::sqrt((double)x);
while (r > 0 && r * r > x)
--r;
while ((r + 1) * (r + 1) <= x)
++r;
return (u32)r;
}
/* sqrt(|Ft|) as EE bits, including the top binade.
Integer, for the same reason eeDivide is: the square-root unit is the divide
unit, and it misses the correctly rounded answer under the same law.
Put the operand's significand where the root is a 24-bit integer. With E the
exponent field and m the significand with its hidden bit,
k = 23 if E is odd, 24 if E is even (|Ft| = X * 2^(E-150-k))
X = m << k 2^46 <= X < 2^48
R = floor(sqrt(X)) 2^23 <= R < 2^24
rem = X - R*R, u = (R+1)^2 - X = 2R+1-rem
so the unit returns R or R+1, correct rounding takes R+1 exactly when
X > (R+0.5)^2 -- i.e. rem > R, with no ties possible since (R+0.5)^2 is
never an integer -- and E-150-k is even by construction, so the result's
exponent field 150 + (E-150-k)/2 is exact integer arithmetic.
The truncation law is the one eeDivideTruncates() applies, with the constant
that goes with it:
u > 2^23 => the unit truncates
2^23 is half of sqrt's minimum span 2R+1 >= 2^24 + 1, as 2^22 is half of
div's minimum span mb >= 2^23. Measured by
captures/fpmatrix/divsqrt/scatter/sqgain.c over the two exhaustive console
sweeps (16,777,216 rows, every significand at both exponent parities,
SCPH-90000, FCR0 0x2E40):
* 10,845,747 rows have u > 2^23. Not one of them rounds up.
* An odd-exponent row rounds up at u = 2^23 exactly, so the bound is
attained.
It takes sqrt from 26.20% of roots off by a ULP to 11.55%. The rest is the
same unsolved region as div's, and keeps the correctly rounded answer. */
static u32 eeSqrtBits(u32 t)
{
FPRreg r;
if ((t & 0x7F800000) == 0)
const u32 E = (t >> 23) & 0xFFu;
if (E == 0)
return 0; // +/-0 and the denormals: the EE drops the sign here, and so
// do both recompilers (they take |Ft| first). See
// EeRecFpu.SqrtSOfNegativeZeroIsPositiveZero.
const int k = (E & 1u) ? 23 : 24;
const u64 X = (u64)(0x800000u | (t & 0x7FFFFFu)) << k;
const u32 R = eeISqrt48(X);
const u64 rem = X - (u64)R * R;
const u32 u = (u32)(2ull * R + 1ull - rem);
// The mode the divide unit runs in, honoured here rather than through the
// host FPCR because this arithmetic is integer. sqrt's result is always
// positive, so toward-negative-infinity is the same as toward zero and
// toward-positive-infinity is "up whenever the root is inexact". The
// truncation law sits on top: it can only take the increment away.
bool round_up;
switch (EmuConfig.Cpu.FPUDivFPCR.GetRoundMode())
{
r.UL = 0; // +/-0 and the denormals: the EE drops the sign here
case FPRoundMode::Nearest: round_up = rem > (u64)R; break;
case FPRoundMode::PositiveInfinity: round_up = rem != 0; break;
default: round_up = false; break;
}
else if ((t & 0x7F800000) == 0x7F800000)
u32 sig = (round_up && u <= (1u << 23)) ? R + 1u : R;
s32 e = 150 + (((s32)E - 150 - k) / 2); // the numerator is always even
if (sig == 0x1000000u) // rounded out of the binade
{
FPRreg quarter;
quarter.UL = (t & 0x7FFFFFFF) - 0x01000000; // |Ft| / 4
r.f = 2.0 * sqrt((double)quarter.f);
sig = 0x800000u;
++e;
}
else
{
FPRreg mag;
mag.UL = t & 0x7FFFFFFF;
r.f = sqrt(mag.f);
}
return r.UL;
return ((u32)e << 23) | (sig - 0x800000u);
}
/* The EE's divide/square-root unit rounds to nearest even when the rest of the
@@ -818,7 +964,8 @@ void RSQRT_S() {
}
void SQRT_S() {
const ScopedDivRoundMode div_round;
// No ScopedDivRoundMode: eeSqrtBits() reads FPUDivFPCR's rounding mode
// itself. DIV.S and RSQRT.S still need it for eeDivide()'s host division.
clearFPUFlags(FPUflagI | FPUflagD);
// Invalid-operation keys off the SIGN BIT ALONE. -0 and the negative
@@ -832,45 +979,7 @@ void SQRT_S() {
if ( _FtValUl_ & 0x80000000 )
_ContVal_ |= FPUflagI | FPUflagSI;
if ( ( _FtValUl_ & 0x7F800000 ) == 0 ) // If Ft = +/-0 (denormals included)
{
_FdValUl_ = 0; // +0: the EE drops the sign here, and
// both recompilers already do (they
// take |Ft| before the sqrt). See
// EeRecFpu.SqrtSOfNegativeZeroIsPositiveZero.
}
else if ( ( _FtValUl_ & 0x7F800000 ) == 0x7F800000 )
{
// Exponent 255 is an ordinary binade on the EE -- no Inf, no NaN, and
// the representable max is 0x7FFFFFFF, not FLT_MAX. The fpuDouble()
// clamp that used to stand here therefore handed sqrt a different
// operand rather than a rounded one, and the answer landed two binades
// low: sqrt(2^128) came back as 0x5F7FFFFF where the console gives
// 0x5F800000, and sqrt(+EEMAX) as 0x5F7FFFFF against 0x5FB504F3.
//
// Square-root |Ft|/4 and double it. sqrt halves exponents, so the
// scaled operand (exponent field 253) and the doubled result are both
// ordinary singles and no wider format is needed. 4 is an even power of
// two, so its own square root is exact and the sqrt below stays the
// only rounding step. It is the power-of-two prescale ToDouble() uses
// to carry these operands into FULL mode (iFPUd-arm64.cpp), with the
// factor picked to suit sqrt so it can stay in single precision.
// recSQRT_S_xmm (iFPU-arm64.cpp) emits the same two steps.
//
// RSQRT_S does not get this, deliberately: its two clamped operands
// currently cancel on rsqrt(2^128, 2^128), so unclamping only the sqrt
// breaks that row. It is all-or-nothing and is a separate change.
FPRreg quarter;
quarter.UL = ( _FtValUl_ & 0x7FFFFFFF ) - 0x01000000; // |Ft| / 4
_FdValf_ = 2.0 * sqrt( (double)quarter.f );
}
else
{
// Exponent 1..254 here: zero and the top binade are taken by the
// branches above, so this leg needs no operand rewrite and can stay in
// single precision.
_FdValf_ = sqrt( fabs( _FtValf_ ) ); // sqrt of |Ft|
}
_FdValUl_ = eeSqrtBits( _FtValUl_ );
}
void SUB_S() {
tests/ctest/core/recompilers/CMakeLists.txt
+1
View File
@@ -132,6 +132,7 @@ add_pcsx2_test(recompiler_tests
ee_fpu_cascade_console_tests.cpp
ee_fpu_compare_console_tests.cpp
ee_fpu_divunit_console_tests.cpp
ee_fpu_divunit_exhaustive_tests.cpp
ee_fpu_guarded_addsub_console_tests.cpp
ee_fpu_top_binade_console_tests.cpp
ee_fpu_zero_divisor_console_tests.cpp
tests/ctest/core/recompilers/ee_fpu_divunit_console_tests.cpp
+166 -72
View File
@@ -1,26 +1,25 @@
// SPDX-FileCopyrightText: 2026 yaps2 Dev Team
// SPDX-License-Identifier: GPL-3.0+
//
// The EE's divide/square-root unit is not correctly rounded, and this file is
// The EE's divide/square-root unit is NOT correctly rounded, and this file is
// the measurement that says so.
//
// Provenance: two first-party captures on an SCPH-90000 over ps2link, 1075 and
// 1156 operand pairs, the rows below generated from the capture files rather
// than transcribed. Each pair is run as sqrt.s Ft, rsqrt.s Fs, Ft and div.s
// Fs, S where S is the sqrt.s Ft the same silicon had just produced; the
// second probe adds div.s Fs, Ft straight off the pair, which is the con_div
// column below. FCR31 is cleared before every op and every result is read
// twice. The first probe re-runs byte-identical, 15/15 corpus controls
// reproduce, and the probe's sqrt.s column matches the corpus's own on all 80
// shared operands.
// 1156 operand pairs, each pair run three ways -- sqrt.s Ft, rsqrt.s Fs, Ft,
// and div.s Fs, Ft -- with FCR31 cleared before every op and every result read
// twice. Both runs are byte-identical on a re-run, and their rows reproduce
// every div/sqrt/rsqrt value in the 1147-case corpus they overlap.
//
// 1. RSQRT.S is sqrt-then-divide, exactly. rsqrt.s Fs, Ft was bit-identical to
// div.s Fs, S on 1075/1075 and 1156/1156 rows: no fused
// reciprocal-square-root, and no extra precision carried between the two
// steps -- the intermediate is a plain 24-bit single. That is what RSQRT_S
// in FPU.cpp does (eeDivide of eeSqrtBits), and RsqrtIsSqrtThenDivide below
// is what keeps it that way: an arm64 fast path tempted by FRSQRTE would
// fail it.
// Two things came out of it.
//
// 1. RSQRT.S IS sqrt-then-divide, exactly. On all 1156 rows of the second
// capture, rsqrt.s Fs, Ft was bit-identical to div.s Fs, S where S is the
// value sqrt.s Ft had just produced on the same silicon. There is no fused
// reciprocal-square-root in there, and no extra precision carried between
// the two steps -- the intermediate is a plain 24-bit single. That is what
// RSQRT_S in FPU.cpp does (eeDivide of eeSqrtBits), and
// RsqrtIsSqrtThenDivide below is what keeps it that way: an arm64 fast path
// tempted by FRSQRTE would fail it.
//
// 2. Neither step is correctly rounded. The reference was computed two
// independent ways -- an exact integer model over dyadic rationals, and the
@@ -29,42 +28,64 @@
// singles. Against that reference, silicon comes out one ULP away on a
// large minority of arbitrary operands, capture 1 / capture 2:
//
// sqrt.s 196 / 1075 and 290 / 1156 rows one ULP low, none high
// sqrt.s 196 / 1075 and 290 / 1156 rows one ULP LOW, none high
// div.s 212 low + 22 high / 1075, 153 low + 36 high / 1156
// rsqrt.s 345 / 1075 and 341 / 1156, and 47 of those are two ULPs
// rsqrt.s 345 / 1075 and 341 / 1156, and 47 of those are TWO ULPs
// out because a low root makes the quotient high
//
// Both captures are deliberately enriched for rows that can show the error;
// the uniform-random figure is the 120 unbiased pairs inside capture 1,
// where sqrt.s misses on 33, div.s on 13 and rsqrt.s on 35.
// the honest uniform-random figure is the 120 unbiased pairs inside capture
// 1, where sqrt.s misses on 33, div.s on 13 and rsqrt.s on 35.
//
// The error is deterministic, but it is not a rounding mode and not a
// function of either operand alone: with the divisor held fixed and the
// numerator swept so the exact quotient walks across its ULP, the rounding
// boundary interleaves on 22 of 24 divisors, and on all 12 numerators with
// the roles swapped. A reciprocal-then-multiply model scores 69% against
// correctly-rounded's 84%.
// The error is deterministic -- the two runs of capture 1 are byte-
// identical -- but it is not a rounding mode and not a function of either
// operand alone:
// with the divisor held fixed and the numerator swept so the exact quotient
// walks across its ULP, the rounding boundary interleaves on 22 of 24
// divisors, and it interleaves on all 12 numerators with the roles swapped.
// A reciprocal-then-multiply model scores WORSE than plain
// correctly-rounded (69% against 84%), so that is not the shape either.
//
// So this tree computes the correctly-rounded result, the closest simple model
// of the unit there is; the rows below pin where it lands on silicon and where
// it does not.
// Part of that is now modelled: eeDivide() and eeSqrtBits() in FPU.cpp truncate
// when u, how far the exact result sits below the upper candidate, is above a
// per-branch cap. The law, its caps and the captures behind them are at
// eeDivideTruncates(). The rows below moved:
//
// This is also the answer to corpus case 220, rsqrt EEMAX, EEMAX -- the last
// result-axis row where the interpreter misses the console. It is one sample
// of the divide unit's approximation and the only row in all 1147 corpus cases
// that can see it: every other div/sqrt case there has an exact result below
// the halfway point, where nearest and truncation agree, which is why eight
// DIV.S rows read as round-to-nearest while this one reads as truncation.
// op was now gained lost
// sqrt.s 66/87 -> 87/87 +21 0
// div.s 66/87 -> 70/87 +4 0
// rsqrt.s 61/87 -> 78/87 +19 2
//
// A model of the unit -- the way eeMulRound/eeMulDefectiveFt already reproduce
// the multiplier's deficit -- would collapse SiliconIsOneUlpOffInTheseExactWays
// and InterpMatchesConsoleWhereTheUnitIsExact into one conformance test and
// have to pass the tripwire at the bottom. Plan and acceptance criteria:
// WORKORDER-divsqrt-model.md in the notes tree.
// The two rsqrt losses were rows where the old code was right by cancellation:
// its square root came back one ULP high (correctly rounded, where silicon
// truncates) and its division then rounded up to the same word silicon reached
// by truncating a smaller divisor. Modelling the square root removed one half
// of that pair; the other half, a division inside the region the law does not
// settle, is still wrong. Both rows now have a sqrt.s column that matches the
// console.
//
// Evidence archive: captures/fpmatrix/divsqrt/ in the notes tree -- rsqprobe.c,
// hw-rsq-run{1,2,3}.bin and the scoring scripts -- with
// PROBE-divsqrt-rounding.md beside it.
// rsqrt.s 3895AEC3, 4938608B was 33B06019 (console), now 33B0601A
// rsqrt.s 43CD0CEB, 365AF7C1 was 485DB675 (console), now 485DB676
//
// Not modelled: the region u <= cap, where 27.5% of div rows truncate as well
// and no coordinate system this project has built predicts which. See
// FINDINGS-div-round11-exhaustive.md; the missing coordinate is a carry
// propagation distance in the trial product, known in shape and not in form.
//
// Corpus case 220, rsqrt EEMAX, EEMAX, is still the last result-axis row where
// the interpreter misses the console, and is still one sample of that unmodelled
// region -- its division is div.s 7FFFFFFF, 5FB504F3 with u = 3,571,369 against
// an A>=B cap of 4,194,304, so the law does not reach it and the correctly
// rounded answer stands where silicon truncates. It is the only row in all 1147
// corpus cases that can see the unit's approximation at all, which is why the
// whole corpus is byte-identical before and after this change.
//
// This file is still interim. The tripwire at the bottom went from 68 failing
// assertions to 26; enabling it means someone closed the u <= cap region. Plan
// and acceptance criteria: WORKORDER-divsqrt-model.md in the notes tree.
//
// Evidence archive: captures/fpmatrix/rsqprobe.c, hw-rsq-run{1,2,3}.bin and
// PROBE-divsqrt-rounding.md in the notes tree.
#include "harness/EeRecTestHarness.h"
#include "harness/MipsEncode.h"
@@ -260,31 +281,95 @@ TEST(EeFpuDivUnitConsole, RsqrtIsSqrtThenDivide)
}
// ---------------------------------------------------------------------------
// 2. Where the hardware unit happens to be exact, this tree must reproduce it.
// 2. How much of the unit the interpreter now reproduces, and where what is
// left over lives.
//
// The per-op counts are drift guards: they will move when someone models
// more of the unit. The assertion inside the loop is the one that matters:
// every row the interpreter still misses must be a division whose `u` lies
// at or below the cap, inside the region the truncation law does not
// settle. A miss above the cap refutes the law rather than widening it.
// ---------------------------------------------------------------------------
TEST(EeFpuDivUnitConsole, InterpMatchesConsoleWhereTheUnitIsExact)
namespace {
// The frame the model is stated in, recomputed rather than shared with FPU.cpp
// so this file cannot inherit the arithmetic it is checking.
struct DivFrame
{
int checked = 0;
u32 T, rem, u, cap;
int lt;
};
DivFrame Frame(u32 a, u32 b)
{
const u32 ma = 0x800000u | (a & 0x7FFFFFu);
const u32 mb = 0x800000u | (b & 0x7FFFFFu);
const int lt = ma < mb ? 1 : 0;
const u64 num = static_cast<u64>(ma) << (23 + lt);
const u32 T = static_cast<u32>(num / mb);
const u32 rem = static_cast<u32>(num - static_cast<u64>(T) * mb);
const u32 cap = lt ? std::max<u32>(1u << 23, mb - (1u << 22)) : (1u << 22);
return { T, rem, mb - rem, cap, lt };
}
} // namespace
TEST(EeFpuDivUnitConsole, WhatIsLeftOverIsAllInsideTheUnsettledRegion)
{
int match[3] = {}, total[3] = {}, fixed_by_the_model = 0;
for (const ConsoleRow& r : kRows)
{
for (Op op : { OP_SQRT, OP_DIV, OP_RSQRT })
{
if (Con(r, op) != Ieee(r, op))
const u32 got = RunInterp(r, op), con = Con(r, op), ieee = Ieee(r, op);
++total[op];
if (got == con)
{
++match[op];
if (con != ieee)
++fixed_by_the_model; // silicon disagrees with correct
// rounding here and we followed silicon
continue;
++checked;
EXPECT_EQ(RunInterp(r, op), Con(r, op))
}
// SQRT.S has no unmodelled rows left on this table at all, so a
// square-root miss is a regression and not a known gap.
ASSERT_NE(op, OP_SQRT)
<< "sqrt.s fs=" << std::hex << r.fs << " ft=" << r.ft
<< " got " << got << " console " << con
<< " -- the sqrt column was exact on all 87 rows";
// RSQRT.S is sqrt-then-divide, so its division is by the root the
// interpreter just produced, not by Ft.
const u32 divisor = (op == OP_DIV) ? r.ft : RunInterp(r, OP_SQRT);
const DivFrame f = Frame(r.fs, divisor);
EXPECT_LE(f.u, f.cap)
<< OpName(op) << " fs=" << std::hex << r.fs << " ft=" << r.ft
<< " (" << r.what << ")";
<< ": missed with u=" << std::dec << f.u << " ABOVE cap=" << f.cap
<< ", which the truncation law says cannot happen";
}
}
EXPECT_EQ(checked, 193) << "the exact-row population moved; re-derive it "
"from the capture rather than editing this number";
EXPECT_EQ(match[OP_SQRT], 87) << "sqrt.s";
EXPECT_EQ(match[OP_DIV], 70) << "div.s";
EXPECT_EQ(match[OP_RSQRT], 78) << "rsqrt.s";
EXPECT_EQ(total[OP_SQRT] + total[OP_DIV] + total[OP_RSQRT], 261);
// Anti-vacuity. If the model were inert the interpreter would just be the
// correctly-rounded column again and this counter would be 0, so every
// number above would be describing nothing.
// 21 sqrt + 4 div + 19 rsqrt; the net gain on the table is 42 because two
// rsqrt rows that used to be right by cancellation are not any more.
EXPECT_EQ(fixed_by_the_model, 44)
<< "rows where silicon and correct rounding differ AND the interpreter "
"followed silicon -- 0 means the truncation law stopped firing";
}
// ---------------------------------------------------------------------------
// 3. Where it is not, the miss is one ULP per rounding step and this tree sits
// on the correctly-rounded side of it. Pinned so that any future model of
// the unit has to come here and say so.
// 3. The console's miss against correct rounding is exactly one ULP, and two on
// the composed op. That is a property of the capture, not of this tree, so
// the test stays as it was: it is the fact any future model has to explain.
// ---------------------------------------------------------------------------
TEST(EeFpuDivUnitConsole, SiliconIsOneUlpOffInTheseExactWays)
{
@@ -310,14 +395,6 @@ TEST(EeFpuDivUnitConsole, SiliconIsOneUlpOffInTheseExactWays)
EXPECT_GE(ulps, 1u);
if (ulps == 2)
++two_ulp;
EXPECT_EQ(RunInterp(r, op), ieee)
<< OpName(op) << " fs=" << std::hex << r.fs << " ft=" << r.ft
<< ": this tree computes the correctly-rounded value";
EXPECT_NE(RunInterp(r, op), con)
<< OpName(op) << " reached the console value -- if that is a "
<< "deliberate new model of the divide unit, this file is the "
<< "place to say so and the tripwire below should be enabled";
}
}
EXPECT_EQ(off_sqrt, 21);
@@ -328,12 +405,17 @@ TEST(EeFpuDivUnitConsole, SiliconIsOneUlpOffInTheseExactWays)
}
// ---------------------------------------------------------------------------
// 4. The fast path is the same arithmetic off the top binade, so it must not
// drift from the interpreter there.
// 4. The fast path does not get the model.
//
// Both recompilers inherit the host's correctly-rounded fdiv/fsqrt, so off
// the top binade the JIT must still produce the `ieee_*` column exactly.
// Whether it should take the model too is a separate call with its own
// measurement, as it was for the multiplier's deficit (eeMulRound is
// interpreter-only as well).
// ---------------------------------------------------------------------------
TEST(EeFpuDivUnitConsole, JitAgreesWithTheInterpreterOffTheTopBinade)
TEST(EeFpuDivUnitConsole, TheFastPathStaysCorrectlyRoundedAndSaysSoHere)
{
int checked = 0;
int checked = 0, diverged = 0;
for (const ConsoleRow& r : kRows)
{
if (!r.jit_agrees)
@@ -346,19 +428,31 @@ TEST(EeFpuDivUnitConsole, JitAgreesWithTheInterpreterOffTheTopBinade)
h.SetFprBits(kFs, r.fs);
h.SetFprBits(kFt, r.ft);
h.LoadProgram({ Encode(op) });
h.Run(); // auto-diffs the two engines
h.RunJitNoDiff(); // not Run(): the two engines now differ on purpose
EXPECT_EQ(h.GetFprBitsJit(kFd), Ieee(r, op))
<< OpName(op) << " fs=" << std::hex << r.fs << " ft=" << r.ft;
if (h.GetFprBitsJit(kFd) != RunInterp(r, op))
++diverged;
}
}
EXPECT_EQ(checked, 249);
// 47 of the 249 cells are rows where the interpreter took the truncation
// law and the fast path did not; if this reaches 0 the interpreter's model
// has been lost.
EXPECT_EQ(diverged, 47)
<< "the interpreter is supposed to leave the fast path behind on exactly "
"the rows the truncation law settles";
}
// ---------------------------------------------------------------------------
// 5. The tripwire. Enabling this means someone has modelled the EE's divide/
// square-root unit rather than computing the correctly-rounded answer.
// Nothing in the tree does today and nothing upstream does either -- both
// recompilers inherit the host's correctly-rounded fdiv/fsqrt.
// 5. The tripwire. It fails on 26 assertions today, down from 68 before the
// truncation law landed -- all 26 are divisions inside u <= cap, which test 2
// asserts one at a time. Enabling this means someone closed that region.
//
// Run it with --gtest_also_run_disabled_tests before graduating it: a
// disabled test that has quietly gone vacuous graduates just as easily as
// one that was fixed.
// ---------------------------------------------------------------------------
TEST(EeFpuDivUnitConsole, DISABLED_InterpMatchesConsoleOnEveryRow)
{
tests/ctest/core/recompilers/ee_fpu_divunit_exhaustive_tests.cpp
+863
View File
File diff suppressed because it is too large Load Diff
tests/ctest/core/recompilers/ee_rec_fpu_divunit_rounding_tests.cpp
+108 -13
View File
@@ -31,6 +31,9 @@
#include "Config.h"
#include "common/FPControl.h"
#include <algorithm>
#include <cmath>
#include <gtest/gtest.h>
using namespace recompiler_tests;
@@ -116,11 +119,58 @@ static bool IsTopBinadeTierGap(u32 interp, u32 jit)
(interp & 0x80000000u) == (jit & 0x80000000u);
}
TEST(EeRecFpuDivUnitRounding, DivSMatchesInterpAtInexactQuotients)
// The second divergence: the interpreter now models the divide unit's
// truncation law (FPU.cpp, eeDivideTruncates / eeSqrtBits) while the emitters
// still take the host's correctly-rounded fdiv/fsqrt. The tests below stop
// asserting that the engines agree and assert instead the shape they may differ
// in: only where the law fires (u > cap), only by one ULP, and only with the
// interpreter on the closer-to-zero side.
//
// The predicates are recomputed here rather than exported from FPU.cpp: a
// differential that imports the implementation's arithmetic cannot catch the
// implementation's arithmetic being wrong.
static bool BothNormalOperands(u32 fs, u32 ft)
{
return ((fs >> 23) & 0xFFu) != 0 && ((ft >> 23) & 0xFFu) != 0;
}
static bool DivideTruncates(u32 fs, u32 ft)
{
const u32 ma = 0x800000u | (fs & 0x7FFFFFu);
const u32 mb = 0x800000u | (ft & 0x7FFFFFu);
const int lt = ma < mb ? 1 : 0;
const u64 num = static_cast<u64>(ma) << (23 + lt);
const u32 rem = static_cast<u32>(num % mb);
const u32 cap = lt ? std::max<u32>(1u << 23, mb - (1u << 22)) : (1u << 22);
return (mb - rem) > cap;
}
static bool SqrtTruncates(u32 ft)
{
const u32 E = (ft >> 23) & 0xFFu;
if (E == 0)
return false;
const u64 X = static_cast<u64>(0x800000u | (ft & 0x7FFFFFu)) << ((E & 1u) ? 23 : 24);
u64 R = static_cast<u64>(std::sqrt(static_cast<double>(X)));
while (R > 0 && R * R > X)
--R;
while ((R + 1) * (R + 1) <= X)
++R;
return (2 * R + 1 - (X - R * R)) > (1u << 23);
}
// The interpreter's word is the JIT's with one unit taken off the magnitude.
static bool IsOneUlpTowardZero(u32 interp, u32 jit)
{
return (jit & 0x7FFFFFFFu) != 0 &&
interp == ((jit & 0x80000000u) | ((jit & 0x7FFFFFFFu) - 1u));
}
TEST(EeRecFpuDivUnitRounding, DivSMatchesInterpExceptWhereTheTruncationLawFires)
{
RequireDistinctDivideRoundingMode();
Lcg r{0xD1F5D1F5A5A5A5A5ull};
int checked = 0, tier_gaps = 0;
int checked = 0, tier_gaps = 0, law_gaps = 0;
for (u32 iter = 0; iter < 3000; ++iter)
{
const u32 fsBits = fuzzOperand(r);
@@ -156,9 +206,21 @@ TEST(EeRecFpuDivUnitRounding, DivSMatchesInterpAtInexactQuotients)
}
if (IsTopBinadeTierGap(res[0], res[1]))
{
++tier_gaps;
else
EXPECT_EQ(res[1], res[0]) << "engines disagree on the quotient";
}
else if (res[0] != res[1])
{
++law_gaps;
EXPECT_TRUE(BothNormalOperands(fsBits, ftBits) &&
DivideTruncates(fsBits, ftBits))
<< "the engines parted company where the truncation law does NOT "
"fire -- that is a plain quotient disagreement, not the "
"modelled one";
EXPECT_TRUE(IsOneUlpTowardZero(res[0], res[1]))
<< "the interpreter's model can only ever take the LOWER of the two "
"candidates; interp=" << std::hex << res[0] << " jit=" << res[1];
}
EXPECT_EQ(fcr[1] & kStickyMask, fcr[0] & kStickyMask);
++checked;
if (::testing::Test::HasFailure())
@@ -168,6 +230,9 @@ TEST(EeRecFpuDivUnitRounding, DivSMatchesInterpAtInexactQuotients)
EXPECT_GT(tier_gaps, 0) << "anti-vacuity: the operand pool stopped producing "
"saturating quotients, so the allowance above is "
"dead code that could hide a real divergence";
EXPECT_GT(law_gaps, 0) << "anti-vacuity: no operand pair reached the truncation "
"law, so this test is asserting engine agreement under "
"a different name";
}
// A named witness alongside the fuzzer: 1.0 / 3.0 is one ULP apart between the
@@ -199,10 +264,11 @@ TEST(EeRecFpuDivUnitRounding, DivSOneOverThreeRoundsToNearest)
// ---------------------------------------------------------------------------
// SQRT.S
// ---------------------------------------------------------------------------
TEST(EeRecFpuDivUnitRounding, SqrtSMatchesInterpAtInexactRoots)
TEST(EeRecFpuDivUnitRounding, SqrtSMatchesInterpExceptWhereTheTruncationLawFires)
{
RequireDistinctDivideRoundingMode();
Lcg r{0x5011EE5011EE1234ull};
int law_gaps = 0;
for (u32 iter = 0; iter < 3000; ++iter)
{
// Both signs: SQRT.S takes |Ft| on the negative path and raises I|SI.
@@ -212,18 +278,47 @@ TEST(EeRecFpuDivUnitRounding, SqrtSMatchesInterpAtInexactRoots)
SCOPED_TRACE(::testing::Message()
<< "iter=" << iter << " Ft=" << std::hex << ftBits << " pre=" << pre);
EeRecTestHarness h;
h.EnableCop1();
h.SetFprBits(1, ftBits);
h.SetFcr31(pre);
h.LoadProgram({ee::SQRT_S(2, 1)});
h.Run();
// Two harnesses, not Run(): the engines now differ on purpose, and
// Run()'s auto-diff cannot express "differ in exactly this shape".
u32 res[2] = {}, fcr[2] = {};
for (int jit = 0; jit < 2; ++jit)
{
EeRecTestHarness h;
h.EnableCop1();
h.SetFprBits(1, ftBits);
h.SetFcr31(pre);
h.LoadProgram({ee::SQRT_S(2, 1)});
if (jit)
{
h.RunJitNoDiff();
res[1] = h.GetFprBitsJit(2);
fcr[1] = h.JitSnapshot().fprs.fprc[31];
}
else
{
h.RunInterpOnly();
res[0] = h.GetFprBitsInterp(2);
fcr[0] = h.InterpSnapshot().fprs.fprc[31];
}
}
EXPECT_EQ(h.JitSnapshot().fprs.fprc[31] & kStickyMask,
h.InterpSnapshot().fprs.fprc[31] & kStickyMask);
if (res[0] != res[1])
{
++law_gaps;
EXPECT_TRUE(SqrtTruncates(ftBits))
<< "the engines parted company on a root the truncation law does "
"NOT settle";
EXPECT_TRUE(IsOneUlpTowardZero(res[0], res[1]))
<< "silicon's square root is one ULP LOW or exact, never high; "
"interp=" << std::hex << res[0] << " jit=" << res[1];
}
EXPECT_EQ(fcr[1] & kStickyMask, fcr[0] & kStickyMask);
if (::testing::Test::HasFailure())
return;
}
EXPECT_GT(law_gaps, 0) << "anti-vacuity: no operand reached the truncation law, "
"so this test is asserting engine agreement under a "
"different name";
}
// sqrt(5): 0x400F1BBD to nearest, 0x400F1BBC chopped.
tests/ctest/core/recompilers/ee_rec_fpu_rsqrt_tests.cpp
+81 -9
View File
@@ -99,10 +99,33 @@ static bool IsTopBinadeTierGap(u32 interp, u32 jit)
(interp & 0x80000000u) == (jit & 0x80000000u);
}
// The second allowance, and a wider one, because RSQRT.S is composed.
//
// The interpreter models the divide unit's truncation law (FPU.cpp,
// eeDivideTruncates / eeSqrtBits) and the emitters still take the host's
// correctly-rounded fsqrt/fdiv, so the interpreter applies the law twice --
// once to the root, once to the quotient. The two do not pull the same way: a
// root that comes back one ULP lower makes the quotient larger, so unlike DIV.S
// and SQRT.S the interpreter can land on either side of the fast path here.
// The compounding is silicon's own -- the console capture has 26 rsqrt.s rows
// one ULP off correct rounding and 2 of them two ULP -- and it is bounded at
// two ULP of magnitude, with the sign never in question.
//
// EeFpuDivUnitConsole owns the law itself, the per-op scoreboard, and the check
// that every remaining console miss lives inside the region the law does not
// settle.
static bool IsDivUnitModelGap(u32 interp, u32 jit)
{
if ((interp & 0x80000000u) != (jit & 0x80000000u))
return false;
const u32 a = interp & 0x7FFFFFFFu, b = jit & 0x7FFFFFFFu;
return (a > b ? a - b : b - a) <= 2u;
}
TEST(EeRecFpuRsqrt, DifferentialFuzzZeroAndNegativeDivisor)
{
Lcg r{0x123456789ABCDEF0ull};
int checked = 0, tier_gaps = 0;
int checked = 0, tier_gaps = 0, model_gaps = 0;
for (u32 iter = 0; iter < 3000; ++iter)
{
const u32 fsBits = fuzzOperand(r);
@@ -142,9 +165,16 @@ TEST(EeRecFpuRsqrt, DifferentialFuzzZeroAndNegativeDivisor)
}
if (IsTopBinadeTierGap(res[0], res[1]))
{
++tier_gaps;
else
EXPECT_EQ(res[1], res[0]) << "engines disagree on the result";
}
else if (res[0] != res[1])
{
++model_gaps;
EXPECT_TRUE(IsDivUnitModelGap(res[0], res[1]))
<< "the engines disagree by more than the divide unit model can "
"produce; interp=" << std::hex << res[0] << " jit=" << res[1];
}
EXPECT_EQ(fcr[1] & kStickyMask, fcr[0] & kStickyMask);
++checked;
if (::testing::Test::HasFailure())
@@ -154,17 +184,25 @@ TEST(EeRecFpuRsqrt, DifferentialFuzzZeroAndNegativeDivisor)
EXPECT_GT(tier_gaps, 0) << "anti-vacuity: the pool stopped producing "
"saturating results, so the allowance is dead "
"code that could hide a real divergence";
EXPECT_GT(model_gaps, 0) << "anti-vacuity: no pair reached the truncation law, "
"so the model allowance is dead code too";
}
// ---------------------------------------------------------------------------
// Positive-divisor fuzzer. An exact differential like every other case in this
// file now that both engines are single-precision and share a rounding mode:
// Run()'s auto-diff checks the value, the flags are diffed on top.
// Positive-divisor fuzzer.
//
// Bounded rather than exact, because the interpreter models the divide unit and
// the fast path does not; the bound is two-sided and sign-checked.
//
// It also checks the composition on 3000 random pairs: silicon's rsqrt.s is
// div.s(Fs, sqrt.s(Ft)) with a plain 24-bit single in between, and the
// interpreter has to keep being that now that both steps carry the model.
// EeFpuDivUnitConsole.RsqrtIsSqrtThenDivide has the console rows.
// ---------------------------------------------------------------------------
TEST(EeRecFpuRsqrt, PositiveDivisorMatchesInterpExactly)
{
Lcg r{0x0F0E0D0C0B0A0908ull};
int checked = 0, tier_gaps = 0;
int checked = 0, tier_gaps = 0, model_gaps = 0;
for (u32 iter = 0; iter < 3000; ++iter)
{
// Positive nonzero divisor: clear sign, force a normal exponent.
@@ -201,10 +239,42 @@ TEST(EeRecFpuRsqrt, PositiveDivisorMatchesInterpExactly)
}
if (IsTopBinadeTierGap(res[0], res[1]))
{
++tier_gaps;
else
EXPECT_EQ(res[1], res[0]) << "engines disagree on the result";
}
else if (res[0] != res[1])
{
++model_gaps;
EXPECT_TRUE(IsDivUnitModelGap(res[0], res[1]))
<< "the engines disagree by more than the divide unit model can "
"produce; interp=" << std::hex << res[0] << " jit=" << res[1];
}
EXPECT_EQ(fcr[1] & kStickyMask, fcr[0] & kStickyMask);
// The composition, on the interpreter alone: sqrt.s Ft, then div.s by
// whatever word that produced. Both steps carry the model, so this fails
// if either one is applied inconsistently between RSQRT_S and the two
// standalone ops.
{
EeRecTestHarness hs;
hs.EnableCop1();
hs.SetFprBits(2, ftBits);
hs.LoadProgram({ee::SQRT_S(4, 2)});
hs.RunInterpOnly();
const u32 root = hs.GetFprBitsInterp(4);
EeRecTestHarness hd;
hd.EnableCop1();
hd.SetFprBits(1, fsBits);
hd.SetFprBits(4, root);
hd.LoadProgram({ee::DIV_S(3, 1, 4)});
hd.RunInterpOnly();
EXPECT_EQ(hd.GetFprBitsInterp(3), res[0])
<< "rsqrt.s must stay div.s(Fs, sqrt.s(Ft)) with a plain single in "
"between, which is what silicon does on every measured row; "
"root=" << std::hex << root;
}
++checked;
if (::testing::Test::HasFailure())
return;
@@ -213,6 +283,8 @@ TEST(EeRecFpuRsqrt, PositiveDivisorMatchesInterpExactly)
EXPECT_GT(tier_gaps, 0) << "anti-vacuity: the positive-divisor pool stopped "
"producing saturating results, so the allowance "
"is dead code that could hide a real divergence";
EXPECT_GT(model_gaps, 0) << "anti-vacuity: no pair reached the truncation law, "
"so the model allowance is dead code too";
}
// ---- Exact-result differential cases (value + flags both diffed) -----------