Tales of Xillia 2 (BLUS31397) polls this syscall in a loop: 20,664 calls in a 19
minute session, 5,123 of them inside a single second. It was logged at warning, so
each one wrote a line to external storage on Android -- the same shape as the SPU
recompiler diagnostics, and enough on its own to stop frames landing.
Reported as issue #76, an intermittent hang every few minutes that ends with
"Game has stopped responding - it is no longer drawing frames". That message comes
from the 30 second frame-stall watchdog, which is what a saturated log writer
looks like from the RSX thread.
Moved to trace. Nothing is lost: a container's size is fixed at creation, and
sys_memory_container_create and _destroy both still log at warning, so the events
that carry information are still recorded. sys_memory_get_user_memory_size in the
same file already has the equivalent treatment upstream -- it only logs when the
values it reports actually change.
Not confirmed as the cause of #76 yet; the reporter has not attached a log, and 19
minutes of play here did not reproduce the hang. It is the largest log source this
title produces by a wide margin.
Booting a game while another was still tearing down left the app frozen on the
last frame of the previous game.
The boot gate asked Emu.IsStopped(), whose default overload is
m_state <= system_state::stopping
so it answers true while the previous VM is still stopping -- and while it is
loading. Kill() only signals the threads and hands the joining to a detached
"Emulation Join Thread", so the state reaches stopping at once and the guard read
that as stopped. The entire wait block was skipped precisely when it was needed,
which is why no "previous VM still running" line was ever logged.
On device: Stopping emulator at 0:10:14, BootGame at 0:10:17, and at 0:10:31 the
join thread was still waiting on PPU[0x1000004] "SPU Interrupt Thread0" -- 17.4
seconds -- with seven SPUs parked in EXIT|w|G-PAUSE and one PPU thread spinning at
100%.
Use the IsStopped(true) overload, which requires system_state::stopped and is
reached only once that join thread has finished. The ten second bound and the
boot-anyway fallback are unchanged, so a teardown that genuinely hangs still gets
reported rather than freezing the UI.
Only the three checks in _rpcsx_boot are changed. Other IsStopped() callers here
want "not running" and the loose overload is right for them.
This does not fix why that interrupt thread fails to exit, which is still open. It
stops a slow teardown from becoming a boot into a half-destroyed VM.
cpu_on_stop() is a teardown hook and nothing enforces that it runs a single time.
A PPU thread that re-enters the stop path without exiting reports again on every
pass: one thread ("SPU Interrupt Thread2") was producing "PPU thread perf stats
are not available." roughly every 10 microseconds, near 100,000 lines a second.
That is survivable on a desktop. On Android the log goes to external storage, so
it pins the log writer and drags down the shutdown it is describing -- the
emulator logged "Stopping emulator..." and never reached "All threads have been
stopped", leaving the next boot stuck on the last frame of the previous game.
Guard the reporting with a flag so it happens once per thread. The flag is
deliberately left out of serialization: it describes this run's reporting, not
guest state.
This does not address why the thread re-enters the stop path, which is a separate
question -- it stops that from being an emulator-wide stall while it is open.
The client thread's inner loop breaks out to the outer sem_rpcn.acquire() for
every state except one: connected and authentified with no game running. The
only blocking wait sits inside the `authentified && !Emu.IsStopped()` branch, so
that case fell through to `while (true)` with nothing to wait on and span a full
core for as long as the user stayed signed in outside a game -- at the menu,
between titles, and throughout shutdown.
Measured on a Retroid Pocket 6 sitting at the library: `RPCN Client` at 100%,
utime 21964 against stime 58, so a userspace spin rather than a syscall storm.
Wait in the fall-through case instead. Breaking out to the outer semaphore would
also stop the spin but nothing releases it when a game starts, so the pings would
never resume.
Only reachable once RPCN actually authenticates, which is why it survived
upstream: signing in is new on Android.
A 15 minute Prototype session wrote 56,881 log lines, and 49,644 of them came from
the SPU recompiler -- peaking at 2,473 lines in a single second, written to
/sdcard. The bursts land exactly when a game is already stalling to compile new
blocks, which is the worst possible moment to add synchronous file writes, and
they stop when compilation finishes. That matches the reported symptom: seconds of
lockup that recover on their own.
Every one of these is per-block or per-instruction:
8963 New SPU block compiled successfully was success
7833 Precompiling fallthrough was notice
4337 Precompiling filler space was notice
3449 SPU block is a loop was notice
2418 MFC_EAH / MFC_Cmd not constant was warning, per INSTRUCTION
1516 Trampoline simplified was error, and is routine
843 SPU Block Dump was notice, and is multi-line
696 GETLLAR pattern entry point was notice
~1400 PUTLLC16 / pattern breakage family was notice and success
Upstream can afford these: a desktop has a fast disk and nobody is writing to
external storage. Demoted to trace, so they stay available by raising the SPU
channel and cost nothing during normal play.
The genuine faults keep their level -- MFC_Cmd invalid size and unknown command
are still errors, and they are rare.
This does not claim compilation is free. It removes the logging so what remains
can be measured, which is not possible while the instrument is this loud.
Adreno waves are 64 lanes, and a workgroup narrower than the wave does not pack
together with its neighbours -- it occupies a whole wave and masks the surplus
lanes off. At 32 that idled half of every wave on every dispatch, and these
kernels run on every texture upload of a kilobyte or more, plus every deswizzle
and detile.
Nothing argues the other way here. The kernels carry no shared memory and no
barriers, so group size is a scheduling hint with no LDS or synchronisation
cost to trade against.
Adreno and Turnip only. The comment this replaces claimed Mali was also 64-wide
and used that to justify holding everything at 32; that is wrong -- Valhall
warps are 16 lanes and Bifrost 4-8, so 32 already spans several of them and
there is no half-empty wave to reclaim. Xclipse is RDNA-derived and prefers
wave32 for compute. Both stay where they were.
Group size is baked into the generated GLSL, so this invalidates shader and
pipeline caches once on first launch after the update.
The mobile branch of the per-vendor group size table picks 32 by falling
through to the NVIDIA case. Adreno and Mali both run 64-wide waves, so 32
plausibly leaves half of each wave idle -- but that is reasoning, not a
measurement, and guessing wrong here costs performance silently.
Read ARMSX3_CS_GROUP_SIZE, which driver_env.txt already plumbs through
setenv(), so both candidates can be compared on one build without a settings
field or a second APK. Powers of two only, clamped to maxComputeWorkGroupSize
and maxComputeWorkGroupInvocations, because an over-large local_size_x fails
shader compilation rather than validation. The default is unchanged; this only
makes the question answerable.
Every vkCreate*Pipelines call passed VK_NULL_HANDLE for the pipeline cache, so
the driver re-did the whole of its own compilation work for every pipeline, in
every run. On mobile that work is a visible stall the first time each pipeline
is seen -- and it was being thrown away at every shutdown.
Give render_device a VkPipelineCache seeded from <cache>/vk_pipeline_cache.bin
and written back at teardown, and hand it to both pipeline creation calls. The
file is keyed on vendorID, deviceID and pipelineCacheUUID, so a driver update
or an adrenotools driver swap rejects the old blob and rebuilds rather than
feeding a driver a cache it cannot read. Oversized blobs are dropped instead of
being allowed to grow without bound -- the cache is shared by every title, so
one cold run is the cheaper failure.
This is orthogonal to the RSX shader cache: that one remembers WHICH pipelines
a title needs, this one makes each one cheap to create.
The shader interpreter was opening a private cache and destroying it on the way
out, which discarded exactly the expensive ubershader compiles. It now borrows
the device's. Teardown order already guarantees the pipe compiler threads are
joined before the device is destroyed, so the readback needs no extra locking.
vkGetPipelineCacheData was missing from the generated Android dispatch table;
regenerated with it, no other entry point changed.
Seventeen commits. The substantial one is kd-11's ROP_OUTPUT_REMAP series
across rsx/fp, glsl and both backends, which ARMSX3 did not have at all.
Two conflicts.
nv4097.cpp: upstream added the ROP remap plumbing to the format-change checks,
we have profiler instrumentation and an ARM64 observe() on the two hot FIFO
reads. Different parts of the same file, so upstream's version is the base and
ours is re-applied on top. The g_xform_const_words increment is included
deliberately: the profiler reports average batch size as words/calls, so
dropping it would have printed 0 rather than nothing, which is worse than an
absent stat.
ISO.cpp: took upstream's magic-read check. It is a real fix --
`!file.read_at(...) == 5` parses as `(!x) == 5`, which is false for every x, so
the guard never fired and a short read left `magic` uninitialised. Our reverted
reader has no check there at all, and read_at returns a byte count in this
version too, so the corrected form applies cleanly.
This does NOT undo the ISO reader revert. The refactor that broke reading for
some users is still reverted; only the one-line magic check comes across.
"Server error 1" is ErrorType::Malformed, and it was our fault rather than the
server's: two buttons posted queries with a required field blank, and the
server rejects the whole query rather than naming the field.
Reset password sends the account's email so the server can mail a token, but
the email box was only rendered while creating an account. Outside that mode it
sent an empty address every time. The box is always shown now.
Resend token sends the password, and Save deliberately clears that box, so
anyone who saved before pressing it sent an empty one. Having typed it a moment
ago is not the same as it being in the field.
Both are checked before sending now, along with account creation, and the
message names the field to fill in rather than failing at the server.
The error text is better too. Malformed, Invalid and the unknown case were
surfacing as a number with no way to act on it. Malformed now says outright
that it is a bug and asks which button was pressed, because if it appears again
the guards above have missed a path.
Not fixed here: sign-in reporting Invalid Password for credentials that work on
desktop. That is a distinct server code rather than a catch-all, and the cause
is not yet known -- desktop stores the password exactly as this does, so it is
not a hashing difference.
The ISO reader revert is confirmed working by the reporter, so this ships it.
Upload signing is now requested explicitly with -Parmsx3.uploadSigning rather
than used whenever keystore.properties happens to exist. Once that file was
created, every release build silently started signing with the upload key, and
an APK signed differently from the one already installed cannot be installed
over it -- so a sideload build becomes something testers cannot install, and
Android's error does not mention signatures.
It was being worked around by hiding keystore.properties by hand before each
build, which is the kind of step that gets forgotten exactly once and then
wastes a tester's evening. build-play-aab.sh passes the flag; nothing else
does, and asking for it without the keystore present is now an error rather
than a silent fallback.
The upstream merge brought a six-commit ISO series that regressed reading on
some images. A tester's ISOs that loaded on 0.9.2 now fail at the very first
header read:
ISO: init: Failed to read region information (region_count=0)
ISO: iso_archive: Corrupt ISO file: Decryption failed
Both of his games fail identically, so it is not one bad dump, and other
people's ISOs still load, so it depends on the image format -- most likely the
3k3y path, which has its own watermark detection and key extraction at 0xF70
and 0xF80, separate from the plain one.
The reader is reverted to its 0.9.2 state. The change implicated is the region
count read moving from char_arr_BE_to_uint(sec0_sec1.data()) to
read_from_ptr<be_t<u32>>(sec0_sec1), alongside decrypt_data moving from raw
pointers to std::span, but this is not a targeted fix: it restores code that
demonstrably worked rather than guessing which line of the refactor is wrong.
is_valid() is kept, because System.cpp calls it and it is one line that has
nothing to do with the failure.
What this gives up is real: the series contained out-of-bounds write and
overflow hardening for ISO parsing. A reader that cannot open a user's games
is the worse of the two, and the hardening should come back with a version of
the refactor that works. The path validation from the same series that lives
outside these two files is untouched.
Unverified against the actual failure -- the ISOs here all load either way, so
this could not be reproduced locally. It is verified not to break what already
worked.
Builds were reporting whichever commit cmake last configured against, not the
one being built. rpcs3/git-version.h is generated at CONFIGURE time, and
build-variants.sh deliberately skips reconfiguring an already-correct build
dir because re-running cmake regenerates LLVM's headers and costs a full
rebuild. So the stamp froze, and every build after that lied about itself.
The gap was wide. HEAD is 20120-da26a455; the header on disk still said
19985-91952ae4, a commit from before 0.9.2.
This is not cosmetic. A tester running 0.9.3 reported that older commit in
their log, which sent an investigation hunting for a regression among upstream
ISO changes their build did not contain, and very nearly had a fix built and
sent for a version they were not running. A build that misreports itself makes
every report from it ambiguous.
stamp-git-version.sh writes the header from HEAD and is called by both build
entry points before anything compiles. It only writes when the contents differ,
so an unchanged HEAD does not force a rebuild, and ninja rebuilds just the
translation units that include it.
The script bundled whatever core happened to be sitting in jniLibs, and
build-variants.sh overwrites that path once per variant, so the file left there
is simply whichever ran last. Running the variants and then rebuilding the
bundle would have shipped an armv8.2 core inside a minSdk 30 bundle:
installable on devices that cannot execute it, failing at dlopen, with nothing
in the failure to point at the cause.
Play serves one bundle to every device, so the ISA floor has to be the lowest
ARMSX3 supports. The legacy core is now stripped into place by this script
every time, and it refuses to run if that core has not been built.
The bundle already shipped was correct -- the legacy core was staged by hand
before it was built -- but only by hand, which is the part worth removing.
The bundle verifier reported libarmsx3-core.so missing from a bundle that
plainly contained it, at base/lib/arm64-v8a/libarmsx3-core.so.
Under `set -o pipefail`, `unzip -l "$AAB" | grep -q x` fails whenever x IS
present: grep exits at the first match, closes the pipe, and unzip takes
SIGPIPE, so the pipeline reports failure. Every check for something that must
be there was inverted, and every check for something that must be absent passed
for the wrong reason -- grep found nothing, read to the end, and no signal was
raised.
Capturing the listing first and piping printf into grep instead only moved
which process took the signal. The listing is matched with `case` now, which
has no subprocess to kill.
Worth stating plainly: this was a verifier that would have reported a clean
bundle whatever went wrong, which is the failure mode the script exists to
prevent. It only surfaced because the one check that should have passed was the
one that failed.
Checking that keystore.properties exists was not enough. A template with the
placeholders still in it passed the check and went on to build, failing much
later inside signing with an error that says nothing about the cause. It now
rejects the placeholders too, so the message arrives before the fifteen minutes
rather than after them.
Also locate a JDK. The script assumed the calling shell had one on PATH, and
without it gradle fails with "Unable to locate a Java Runtime" several steps in,
which points at Java rather than at the environment. Android Studio's bundled
JDK is used when JAVA_HOME is unset, with /usr/libexec/java_home as a fallback
and a clear failure if neither exists.
Two things stood between the flavor split and an uploadable bundle.
The release build type was debug-signed, with a comment saying to swap it
before any public build. It now uses an "upload" signing config when
keystore.properties exists and falls back to the debug key when it does not, so
GitHub alphas stay sideloadable without the key present while Play gets a
properly signed bundle. The keystore and the properties file are both already
gitignored, and nothing reads or echoes them outside the build.
build-play-aab.sh refuses to start without that file and prints the keytool
command to create one, rather than spending fifteen minutes producing a bundle
Play will reject.
Minify is off for the Play bundle and on everywhere else. This is not a
preference: AGP 9.2.1's R8 writes its mapping as mapping.prt, a compressed
per-class archive, while packageBundle still requires a plain mapping.txt, so
an AAB cannot be built with R8 enabled at all. ARMSX2 ships its Play build with
minify off entirely, so this matches existing practice rather than introducing
a compromise, and a 94 MB native core dominates a 76 MB APK -- shrinking the
Kotlin was never where the size is.
Driven by a gradle property, matching how armsx3.minSdk is already threaded
through by build-variants.sh, rather than by the variant API.
Verified: the github release APK still builds at 76.5 MB with R8 running.
An AAB is a ZIP and its entries are compressed, so grepping the archive finds
nothing and reports a clean bundle whatever is inside it. The first version of
this script did exactly that. It passed all five exclusion checks and then
failed to find the applicationId it was supposed to find, which is the only
reason the method got questioned at all -- every check that mattered was a
false pass.
It extracts the bundle now and reads base/manifest/AndroidManifest.xml for the
permissions, and the archive listing for the native libraries. Verified against
a real bundle: com.armsx3.play present, REQUEST_INSTALL_PACKAGES,
MANAGE_EXTERNAL_STORAGE, RECORD_AUDIO and the update provider all absent, and
libarmsx3_lsfg.so not among the shipped .so files.
The manifest is protobuf rather than text, so LC_ALL=C and grep -a stay: the
NUL bytes in it would otherwise make grep declare the file binary and print
nothing, which reads the same as a pass.
Play will not take the build ARMSX3 ships today, and the two runtime flags that
were already here could never have made it acceptable. The manifest said so:
it is the declared PERMISSION in the bundle that gets rejected, not the code
path behind a boolean. So the split is by source set, which is the only thing
that actually removes anything.
github keeps what a sideloaded build should have: the in-app updater, all-files
storage for an arbitrary data folder, and frame generation. play has none of
them, and its applicationId is com.armsx3.play so the two install side by side
instead of over each other.
Moved rather than flagged: com.armsx2.update and its FileProvider resource into
src/github, with no-op composables in src/play so that source set still
compiles. REQUEST_INSTALL_PACKAGES and MANAGE_EXTERNAL_STORAGE now live in the
github manifest, so the play bundle never declares them.
Frame generation is excluded by moving libarmsx3_lsfg.so into
src/github/jniLibs. A packaging block inside a productFlavor is NOT honoured --
it applied to both and dropped the library from the github build too, which the
build caught. Excluding the file is the whole exclusion: the shim is dlopen'd
by name and the core already reports the feature unavailable when it is absent,
which is the same path a device that cannot run it takes.
build-play-aab.sh builds the bundle and refuses to emit one that fails any of
five checks, because a flavor split is only worth as much as the thing that
notices when it stops working. It greps the bundle with LC_ALL=C and -a on
purpose: a NUL byte makes plain grep declare the file binary and say nothing,
which reads exactly like a pass.
build-variants.sh follows the rename to assembleGithubRelease; there is no
flavorless release variant any more.
Both failures in the savedata fixup were logged as fatal while the code
carried straight on -- the loop continues, the emulator starts, and one save
is left as it was. On Android a fatal line goes to logcat at ANDROID_LOG_FATAL,
so it reads like a crash, and it was reported as one.
It is also permanent rather than transient when it happens. Android storage can
refuse a directory rename outright with EPERM, and nothing about launching
again changes that, so it fired on every single launch and looked like a fault
that was getting worse. A user hitting it has no way to tell that their games
are unaffected.
Both are errors now, and both say what the consequence is and what to delete if
it keeps happening, rather than repeating an alarm about something the emulator
has already decided to continue past.
jpolo1224
Megamouse