Arkham City never finished compiling, and neither did LEGO Batman 2. The log
said "LLVM crash recovery invoked" 240 times and then killed main_thread,
which looks like a codegen bug and is not one.
What actually happened: utils::memory_commit failed with ENOMEM inside the
disposable LLVM worker. That thread dying is how run_recoverable_llvm reports
any failure, so an out-of-memory device was indistinguishable from bad
codegen. It cost a long detour through max_map_count, disk space and
overcommit before the errno in the fatal gave it away, so the JIT's allocator
now uses a checked commit and throws a plain "Out of memory" instead.
The fatal part was the symbol resolvers. ppu_initialize ensure()d that every
group's __resolve_symbols was present, but that function lives in the compiled
output: when every module in a group fails, it is simply absent. The ensure
turned a partial compile into a dead main_thread, which discarded the 170
modules that HAD compiled and surfaced as a boot that never ends.
That contradicts the design either side of it -- a module that fails to load
is deliberately not fatal, because a guest function with no compiled code
keeps its dispatcher entry and is interpreted. A missing resolver is now the
same: report it, skip it, let that group interpret. Losing one group's speed
beats losing the boot.
Also tell the user. Out of memory is the only compile failure they can act on,
and the useful action is not obvious: compiled modules are already in the
cache, so starting the game again resumes rather than restarting the work. One
message after the workers join, not one per module -- once memory is short
every remaining module fails identically, and two hundred popups would be
worse than none.
Lowering Max LLVM Compile Threads also avoids it, and is deliberately not
suggested in the message: compile time is already the common complaint, and
halving the workers to dodge a case that is now survivable is a bad trade.
SPU: the ARM64 block checksum folded two thirds of every block through
absolute difference, which is not injective, so adding the same value to
two words left the checksum unchanged and similar job binaries hashed
alike. Plain summation now. This is what Precise SPU Verification was
working around, and that setting is exposed properly instead of only being
reachable by hand editing the config.
SPU: a block is no longer marked permanently failed when the trampoline
rebuild fails. The compiled function was live, the state was not
recoverable for the rest of the session, and the claim could never be
retaken.
RSX: render pass churn cut in heavy scenes, roughly 113 to 85 passes per
frame. On a tile based GPU every pass boundary is a full tile store and
reload. Two Vulkan specification violations fixed, and a read/write hazard
on the render pass path.
RSX: the FIFO no longer burns a core on sched_yield while idle.
Android: ADPF is implemented rather than an inert setting, logcat no longer
allocates and makes an IPC call per line, and Silence All Logs is available
for playable titles.
Audio: Oboe backend, for the per device quirks database and stream recovery
on disconnect and route change.
Ported from ouroboros420/rpcsx: GPU Turbo, power and thermal handling, the
crash and freeze fixes, savestate and WSI surface lifetime, honest RAM VRAM
budgeting, the persistent SPU object cache design, occlusion query and RSX
fixes, frame pacing and tiler tuning.
Ported from rfandango/rpcsx: the Turnip ZCULL deadlock fix and ARM64 SPU
checksum handling.
Individual commits are credited in comments at each site.
run_recoverable_llvm runs code generation on a disposable thread and terminates
it through pthread_exit when LLVM invokes its fatal error handler. bionic does
not force-unwind C++ frames on pthread_exit, so the lock MCJIT holds over the
execution engine is never released and stays held by a thread that no longer
exists.
Every entry point into the engine takes that lock, so the next compile hangs and
so does teardown. One recovered error, and the emulator is finished until it is
killed -- and the error is recovered, which is the point: it is meant to be
survivable.
Their Kotlin log-channel screen is left out; this port has its own.
From MaxsTechReview in PS3Native.
The fallback CPU table was missing cores found in recent handheld SoCs
(Cortex-A510, A715, X3, A520, A720, X4). Because get_cpu_name() bails out
when any detected MIDR is unknown, a single missing core sent the whole
lookup to the cortex-a78 fallback whenever LLVM host detection returned
"generic".
Display names were also reused as LLVM -mcpu values, which happens to work
for the Cortex names but not for Qualcomm Oryon: the display name lowercased
to "x-elite", which is not an LLVM processor, so the JIT silently lost
per-CPU scheduling. Entries now carry an explicit canonical LLVM name
alongside the human-readable one; get_cpu_brand() keeps using the latter.
The Qualcomm entry is named "Oryon" rather than "X-Elite" because MIDR
0x51/0x001 only identifies an Oryon core, not the SoC it sits in.
MIDRs cannot identify the SoC at all, which made bug reports ambiguous.
Android's own SOC_MANUFACTURER/SOC_MODEL are now passed to the core and
logged as a separate "SoC:" line, so SoC identity, core topology and the
resolved LLVM target are three distinct values. The LLVM target reported by
system info now comes from the same resolution path the JIT uses, rather
than from the fallback alone, so it no longer disagrees with the target
actually compiled for.
The Vulkan renderer logs one verdict for the adapter it selected, recording
whether BC1-BC3 support keeps DXT textures compressed or whether they are
decoded on the CPU. It sits in render_device::create rather than where the
flag is resolved, because physical_device::create runs for every GPU of
every instance, and not in TextureUtils, whose fallback branches run per
texture and per mip level.
SoC information travels through a new optional _rpcsx_setSocInfo export
instead of an added _rpcsx_initialize parameter. The core is dlopen()ed and
can be updated independently of the JNI glue, so changing an existing
export's signature would make older glue call it with a garbage argument.
Older glue simply never calls the setter, and newer glue null-checks the
symbol against older cores.
No JIT feature policy and no texture decoding behaviour changed.
Verified on an AYN Odin 3 (ayn CQ8725S, 8x Oryon, Adreno 830) running
Turnip 26.2.99: SoC line reads "ayn CQ8725S (Snapdragon 8 Elite-class)",
the brand line reports Oryon rather than X-Elite, the JIT resolves to
oryon-1, and a single BC verdict reports the GPU path. That BC result
applies to the Turnip driver tested; stock-driver behaviour is unmeasured.
While chasing an unrelated SPURS hang I noticed the JIT publishes
freshly written code on ARM64 with no instruction-cache maintenance at
all. A grep for clear_cache or flushInstructionCache over the JIT layer
comes back empty. The branch-rewrite sites only issue ISB; DSB ISH,
which performs no D-cache clean or I-cache invalidation and is ordered
backwards for self-modifying code besides. On ARMv8 a correct
publication needs the DC CVAU / IC IVAU broadcast sequence; x86 has a
coherent instruction cache, so none of this was ever visible there.
All sites use the bundled asmjit::VirtMem::flushInstructionCache(),
which emits that sequence portably across toolchains.
This covers every publication path I could find:
- MemoryManager1::finalizeMemory() and MemoryManager2::finalizeMemory()
were both no-ops. RuntimeDyld calls finalizeMemory() after writing
code and relies on it for cache maintenance, so LLVM emitted PPU and
SPU code was never flushed. MemoryManager1 serves the primary PPU
JIT, MemoryManager2 the SPU JIT and auxiliary engines. Both managers
now record code section allocations and flush them on finalize. I
confirmed at runtime that the MemoryManager2 path executes (about
12800 calls per cold boot).
- jit_runtime_base::_add() copies asmjit output into executable memory
with no flush.
- jit_runtime::finalize() restores an executable code snapshot in place
during emulator restart with only the ISB/DSB pair.
- spu_runtime::rebuild_ubertrampoline() publishes a hand-written
trampoline via CAS with no flush; the flush now happens before the
publication.
- spu_runtime::make_branch_patchpoint() writes a patchpoint byte by
byte and returns it with only the ISB/DSB pair.
- Both 16-byte branch-site rewrites (dispatch and branch) atomically
overwrite live code and only issued the ISB/DSB pair.
The ISB/DSB pairs adjacent to the new flushes are removed along with
their misleading "flush all cache lines" comments: the flush helper
already issues the trailing barriers, and the pairs never performed
any cache maintenance in the first place.
I want to be upfront that this was not the cause of the hang I was
debugging (a same-item compilation race, fixed separately), and I have
not observed a failure that this change alone fixes. It is a latent
correctness issue on any ARM64 host: nothing prevents another core
from fetching stale instruction bytes for freshly published code.
The PPU/SPU recompilers emit i8mm intrinsics (ummla/smmla, used by the
SPU GBB/GBH gather paths) gated on utils::has_i8mm(). The JIT's MAttrs
list mirrored dotprod/sha3/sve from HWCAP but never added i8mm, and the
resolved -mcpu on Apple silicon is the cortex-a78 fallback (no i8mm), so
the backend aborted with "Cannot select: intrinsic %llvm.aarch64.neon.ummla"
on every game. Mirror i8mm into MAttrs like the other features.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 52d121fee79cc569c9ac273852edd0e493ae51fc)
- Some SPU programs inexplicably fail to compile when TBL2/TBX2 are used.
- As an insane workaround, first try to compile with TBL2/TBX2, if LLVM crashes while compiling, try to compile the same program without TBL2/TBX2.
- On x86, LLVM has robust detection for the CPU name. If a CPU like skylake has AVX disabled, it will fall back to something without AVX (nehalem)
- On ARM, detection is not as robust. For instance, on my snapdragon 8 gen 2, it assumes that we have SVE support, as the cortex-x3 supports SVE.
- If an ARM cpu is paired with other cpus from another generation which doesn't support the same instructions as the cortex-x3, or if the cortex-x3 just has SVE disabled for no apparant reason (in the case of the snapdragon 8 gen 2)
- We need to actually detect that ourselves.
- Beyond SVE also detect support for some instructions that might be useful SPU LLVM when optimized with intrinsics.
jpolo1224
Zulux91
Megamouse
Elad
Rad0van
Malcolm
Marin Baron
DH
RipleyTom
kd-11
Darkhost1999