GS: keep the full barrier under framebuffer fetch when primitives overlap

DetermineBarriers dropped both barrier flags whenever framebuffer fetch was
available, on the reasoning that fetch makes them unnecessary. It does not.
Fetch replaces the destination *read*; whether it also orders overlapping
primitives *within* one draw is a per-backend property, and the blending path
depends on that ordering: it switches an overlapping draw to software blending
precisely because fetch is available ("on fbfetch, one barrier is like full
barrier") and requests a full barrier to get per-primitive ordering. Clearing
the flag here on the same reasoning removed the mechanism that supplied it, so
a primitive blended against a destination its predecessor had not written yet.

The two decisions were reading one feature bit as the answer to two different
questions, and only the first is what GL fetch guarantees.

Measured on the GL arm (Mesa 25.3.6, Apple M2 Max, GL_EXT_shader_framebuffer_
fetch), replaying dumps through gsrunner and scoring every pixel against the
software rasteriser, which is an exact GS:

  MGS3          76872 px wrong by >=8  ->    753   (RT-copy path: 759)
  Katamari       8255, 2415 px >=64    ->    806, 30 px >=64  (copy: 8255)
  FlatOut 2                            -> pixel-identical to the copy path
  Dirge of Cerberus                235 ->    234

It was also nondeterministic, which is what pointed at ordering rather than
arithmetic: 18% of the MGS3 frame changed between identical replays, 56464
pixels varying run to run, while the copy path was byte-identical across every
run. That is now zero.

Vulkan's framebuffer fetch *is* VK_EXT_rasterization_order_attachment_access
and Metal's is programmable blending; both order overlapping fragments by
contract, so they keep the barrier-free path and their render passes intact --
making them pay for a barrier would reintroduce the pass breaks that path
exists to remove. Only GL changes. Where a draw's own primitives do not
overlap the question is moot, since a live in-tile read and a pre-draw snapshot
are the same value, so the barrier is still dropped there on every backend:
76 of the 84 affected draws in that MGS3 frame.

The decision moves into GSFramebufferFetchPolicy.h beside the fetch decision
itself, for the same reason that one was extracted -- it is a rule about what a
capability does and does not imply, and it belongs somewhere a reader and a
test can see it whole.

pcsx2/GS/Renderers/Common/GSDevice.h

@@ -1403,6 +1403,7 @@ public:
 		bool dxt_textures         : 1; ///< Supports DXTn texture compression, i.e. S3TC and BC1-3.
 		bool bptc_textures        : 1; ///< Supports BC6/7 texture compression.
 		bool framebuffer_fetch    : 1; ///< Can sample from the framebuffer without texture barriers.
+		bool framebuffer_fetch_orders_overlap : 1; ///< Framebuffer fetch also orders overlapping primitives *within* a single draw, so a full barrier is redundant. Vulkan's rasterization-order attachment access and Metal's programmable blending guarantee this; the GL fetch extensions do not deliver it in practice.
 		bool stencil_buffer       : 1; ///< Supports stencil buffer, and can use for DATE.
 		bool cas_sharpening       : 1; ///< Supports sufficient functionality for contrast adaptive sharpening.
 		bool test_and_sample_depth: 1; ///< Supports concurrently binding the depth-stencil buffer for sampling and depth testing.

pcsx2/GS/Renderers/Common/GSFramebufferFetchPolicy.h

@@ -89,6 +89,49 @@ constexpr GSFramebufferFetchDecision DecideGLFramebufferFetch(bool has_arm_fetch
 	return decision;
 }
 
+// Whether a draw may drop its barrier requirement because framebuffer fetch is available.
+//
+// Fetch replaces the destination READ. Whether it also orders overlapping primitives WITHIN one
+// draw depends on which spelling of it the backend has, and that distinction is load-bearing:
+// GSRendererHW switches an overlapping draw to software blending *because* fetch is available
+// ("on fbfetch, one barrier is like full barrier") and asks for a full barrier to get the
+// per-primitive ordering that path needs. Dropping the barrier on the same reasoning removed the
+// mechanism that supplied the ordering, so a primitive blended against a destination its
+// predecessor had not written yet. It raced: on Mesa 25.3.6 / Apple M2, a 640x480 MGS3 frame came
+// out 101x further from the software rasteriser than the RT-copy path (76872 pixels wrong by >=8
+// against 759), and 18% of the frame changed between identical replays.
+//
+// Vulkan's rasterization-order attachment access and Metal's programmable blending guarantee the
+// ordering by contract, so they still drop the barrier and keep their render passes intact. The
+// GL fetch extensions do not deliver it in practice. Where the draw's own primitives do not
+// overlap the question is moot -- a live in-tile read and a pre-draw snapshot are the same value --
+// so the barrier is dropped there on every backend, which covered 76 of the 84 affected draws in
+// that frame.
+//
+// `prims_may_overlap` must be true when overlap is merely unknown; guessing "no" is what costs
+// correctness, and guessing "yes" costs only a split draw.
+constexpr bool FbFetchDropsDrawBarriers(
+	bool fetch_orders_overlapping_prims, bool prims_may_overlap, bool needs_barriers_for_depth)
+{
+	// Depth feedback reads through a texture, not the colour attachment, so fetch says nothing
+	// about it and its barriers stand regardless.
+	if (needs_barriers_for_depth)
+		return false;
+
+	return fetch_orders_overlapping_prims || !prims_may_overlap;
+}
+
+// The regression: GL fetch does not order overlapping primitives, so an overlapping draw keeps its
+// barrier. Everything else about the old unconditional drop is preserved.
+static_assert(!FbFetchDropsDrawBarriers(false, true, false));
+static_assert(FbFetchDropsDrawBarriers(false, false, false));
+// Backends whose fetch *is* an ordering guarantee keep the barrier-free fast path, overlap or not.
+static_assert(FbFetchDropsDrawBarriers(true, true, false));
+static_assert(FbFetchDropsDrawBarriers(true, false, false));
+// Depth feedback outranks all of it.
+static_assert(!FbFetchDropsDrawBarriers(true, false, true));
+static_assert(!FbFetchDropsDrawBarriers(false, false, true));
+
 // The regression itself: a blocklisted driver, or the user's setting, must survive the Mali
 // profile -- and must not drag the profile to PowerVR on the way.
 static_assert(!DecideGLFramebufferFetch(true, true, true, true, false, true).enabled);

pcsx2/GS/Renderers/HW/GSRendererHW.cpp

@@ -5,6 +5,7 @@
 #include "GS/Renderers/HW/GSHwHack.h"
 #include "GS/Renderers/HW/GSDrawLog.h"
 #include "GS/Renderers/HW/GSTextureReplacements.h"
+#include "GS/Renderers/Common/GSFramebufferFetchPolicy.h"
 #include "GS/GSGL.h"
 #include "GS/GSPerfMon.h"
 #include "GS/GSUtil.h"
@@ -6407,9 +6408,13 @@ void GSRendererHW::DetermineBarriers(GSTextureCache::Target* rt, GSTextureCache:
 		// If we use depth-as-color feedback, then FB fetch can be used for depth also.
 		const bool need_barriers_for_depth = m_conf.ps.IsFeedbackLoopDepth() && features.depth_feedback;
 
-		if (!need_barriers_for_depth)
+		// Fetch replaces the destination read; whether it also orders overlapping primitives
+		// within the draw is a per-backend property, and the software blend path enabled above
+		// depends on that ordering. See FbFetchDropsDrawBarriers for the full reasoning.
+		// PRIM_OVERLAP_UNKNOWN counts as overlapping.
+		if (FbFetchDropsDrawBarriers(features.framebuffer_fetch_orders_overlap,
+				m_prim_overlap != PRIM_OVERLAP_NO, need_barriers_for_depth))
 		{
-			// Barriers aren't needed with fbfetch
 			m_conf.require_one_barrier = false;
 			m_conf.require_full_barrier = false;
 		}

pcsx2/GS/Renderers/Metal/GSDeviceMTL.mm

@@ -1201,6 +1201,9 @@ bool GSDeviceMTL::Create(GSVSyncMode vsync_mode, bool allow_present_throttle)
 	m_features.dxt_textures = true;
 	m_features.bptc_textures = true;
 	m_features.framebuffer_fetch = m_dev.features.framebuffer_fetch && !GSConfig.DisableFramebufferFetch;
+	// Apple's programmable blending reads the tile in rasterization order, so overlapping
+	// primitives in one draw already observe each other and a full barrier adds nothing.
+	m_features.framebuffer_fetch_orders_overlap = m_features.framebuffer_fetch;
 	m_features.stencil_buffer = true;
 	m_features.cas_sharpening = true;
 	m_features.test_and_sample_depth = true;

pcsx2/GS/Renderers/Vulkan/GSDeviceVK.cpp

@@ -3602,6 +3602,11 @@ bool GSDeviceVK::CheckFeatures()
 		(is_mali_vk || is_adreno || GSConfig.EnableAdrenoFramebufferFetch) && !is_xclipse_vk;
 	m_features.framebuffer_fetch = vendor_allows_fbfetch &&
 		m_optional_extensions.vk_ext_rasterization_order_attachment_access && !GSConfig.DisableFramebufferFetch;
+	// The Vulkan spelling of framebuffer fetch *is* rasterization-order attachment access, whose
+	// contract is that overlapping fragments in one draw observe each other in primitive order.
+	// So the ordering a full barrier would provide is already guaranteed, and keeping the barrier
+	// would only reintroduce the render-pass breaks this path exists to avoid.
+	m_features.framebuffer_fetch_orders_overlap = m_features.framebuffer_fetch;
 	m_features.texture_barrier = GSConfig.OverrideTextureBarriers != 0;
 	// No working in-pass render-target self-read (ARMSX2 #442, Qualcomm/Turnip). Force the RT-COPY
 	// path: with texture barriers off, GSRendererHW reads Cd from a separate copy of the target

tests/ctest/core/gs/gs_framebuffer_fetch_policy_tests.cpp

@@ -177,3 +177,61 @@ TEST(GSFramebufferFetchPolicy, ProfileDemotionIgnoresTheVetoInputs)
 		}
 	}
 }
+
+// The barrier half of the same policy (FbFetchDropsDrawBarriers).
+//
+// Same class of bug as the one above, in the opposite direction: not a decision re-made in three
+// places, but one decision standing in for two different questions. "Can I read the destination
+// without a barrier?" and "are overlapping primitives ordered against each other?" were both
+// answered by features.framebuffer_fetch, and only the first of them is what fetch actually
+// guarantees on GL. The blending path asks for a full barrier so overlapping primitives observe
+// each other; DetermineBarriers then deleted it because fetch was available. Measured on the GL
+// arm, that cost 101x the per-pixel error of the RT-copy path and made the frame nondeterministic.
+
+TEST(GSFramebufferFetchPolicy, GLFetchKeepsTheBarrierWhenPrimitivesOverlap)
+{
+	// The regression. GL's fetch orders nothing, so an overlapping draw must keep its barrier --
+	// the software blend path enabled for it reads a destination its own predecessor writes.
+	EXPECT_FALSE(FbFetchDropsDrawBarriers(false, true, false));
+}
+
+TEST(GSFramebufferFetchPolicy, NonOverlappingDrawsDropTheBarrierOnEveryBackend)
+{
+	// With no overlap, a live in-tile read and a pre-draw snapshot are the same value, so the
+	// barrier buys nothing. This is the common case and the reason the fix is nearly free.
+	EXPECT_TRUE(FbFetchDropsDrawBarriers(false, false, false));
+	EXPECT_TRUE(FbFetchDropsDrawBarriers(true, false, false));
+}
+
+TEST(GSFramebufferFetchPolicy, OrderingBackendsKeepTheBarrierFreeFastPath)
+{
+	// Vulkan's rasterization-order attachment access and Metal's programmable blending order
+	// overlapping fragments by contract. Making them pay for a barrier would reintroduce exactly
+	// the render-pass breaks the fetch path exists to remove, for no correctness gain.
+	EXPECT_TRUE(FbFetchDropsDrawBarriers(true, true, false));
+}
+
+TEST(GSFramebufferFetchPolicy, DepthFeedbackBarriersSurviveEveryFetchCapability)
+{
+	// Depth feedback reads through a texture rather than the colour attachment, so no colour-fetch
+	// capability says anything about it.
+	for (bool orders : {false, true})
+	{
+		for (bool overlap : {false, true})
+		{
+			SCOPED_TRACE(testing::Message() << "orders=" << orders << " overlap=" << overlap);
+			EXPECT_FALSE(FbFetchDropsDrawBarriers(orders, overlap, true));
+		}
+	}
+}
+
+TEST(GSFramebufferFetchPolicy, DroppingBarriersNeverDependsOnOverlapAloneWithoutAnOrderingGuarantee)
+{
+	// The invariant that fails if someone reinstates the unconditional drop: without an ordering
+	// guarantee, the answer must track overlap exactly.
+	for (bool overlap : {false, true})
+	{
+		SCOPED_TRACE(testing::Message() << "overlap=" << overlap);
+		EXPECT_EQ(FbFetchDropsDrawBarriers(false, overlap, false), !overlap);
+	}
+}