Check in missing files

This commit is contained in:
Yoshi Askharoun
2026-07-23 01:52:18 -05:00
parent 7d96b4fd64
commit 194725ccb6
10 changed files with 1411 additions and 8 deletions
-7
View File
@@ -21,17 +21,10 @@ mono_crash.*
[Dd]ebugPublic/
[Rr]elease/
[Rr]eleases/
x64/
x86/
[Ww][Ii][Nn]32/
[Aa][Rr][Mm]/
[Aa][Rr][Mm]64/
bld/
[Bb]in/
[Oo]bj/
[Oo]ut/
[Ll]og/
[Ll]ogs/
# Reinclude the Microsoft.Iris.Debug namespace
!**/Microsoft/Iris/**
+13
View File
@@ -0,0 +1,13 @@
using System;
using System.Runtime.InteropServices;
namespace Microsoft.Iris.Render.Interop.Win32;
// Bit-for-bit mirror of Microsoft.Iris.OS.Win32Api.HANDLE (UIX/Microsoft/Iris/OS/Win32Api.cs)
// -- the opaque handle type NativeApi.cs's rich-text/simple-text exports use (hRto/hSto).
[StructLayout(LayoutKind.Sequential)]
public struct HANDLE
{
public IntPtr h;
public static readonly HANDLE NULL = new() { h = IntPtr.Zero };
}
+12
View File
@@ -0,0 +1,12 @@
using System;
using System.Runtime.InteropServices;
namespace Microsoft.Iris.Render.Interop.Win32;
// Bit-for-bit mirror of Microsoft.Iris.Render.HWND (UIX.RenderApi/Microsoft/Iris/Render/HWND.cs).
[StructLayout(LayoutKind.Sequential)]
public struct HWND
{
public IntPtr h;
public static readonly HWND NULL = new() { h = IntPtr.Zero };
}
+26
View File
@@ -0,0 +1,26 @@
using System.Runtime.InteropServices;
namespace Microsoft.Iris.Render.Interop.Win32;
// Bit-for-bit mirror of Microsoft.Iris.OS.Win32Api.LOGFONTW_STRUCT
// (UIX/Microsoft/Iris/OS/Win32Api.cs), including its 32-char inline face-name buffer.
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
public unsafe struct LOGFONTW
{
public const int LF_FACESIZE = 32;
public int lfHeight;
public int lfWidth;
public int lfEscapement;
public int lfOrientation;
public int lfWeight;
public byte lfItalic;
public byte lfUnderline;
public byte lfStrikeOut;
public byte lfCharSet;
public byte lfOutPrecision;
public byte lfClipPrecision;
public byte lfQuality;
public byte lfPitchAndFamily;
public fixed char lfFaceName[LF_FACESIZE];
}
+19
View File
@@ -0,0 +1,19 @@
using System;
using System.Runtime.InteropServices;
namespace Microsoft.Iris.Render.Interop.Win32;
// Bit-for-bit mirror of Microsoft.Iris.Render.Internal.Win32Api.MSG
// (UIX.RenderApi/Microsoft/Iris/Render/Internal/Win32Api.cs) -- the variant EngineApi.cs's
// SpPeekMessage actually resolves to (hwnd is the Render-namespace HWND wrapper, not a bare IntPtr).
[StructLayout(LayoutKind.Sequential)]
public struct MSG
{
public HWND hwnd;
public uint message;
public IntPtr wParam;
public IntPtr lParam;
public uint time;
public int pt_x;
public int pt_y;
}
+436
View File
@@ -0,0 +1,436 @@
# UIXrender.dll — architecture log
Append-only. Do not edit previous entries.
---
## 2026-07-21 — Schema extractor written and run: `tools/extract_splash_schema.py`
**Task:** implement the codegen decision from the entry directly below. Wrote a Python
script, `tools/extract_splash_schema.py`, that parses every `*.cs` file under
`UIX.RenderApi/Microsoft/Iris/Render/Protocols/Splash/` and produces
`tools/splash-schema.json`: for each `Remote*`/`Local*Callback` class, its namespace,
base class, source file, and every `Msg<N>_<Name>` struct's fields (name + C# type),
whether it carries a `BLOBREF` (variable-length payload), and whether the message is
instance-scoped (`this.m_renderHandle`) or class/static-scoped. Neither the script nor
its output is wired into a build yet — this is groundwork only, no native code
generated.
**Results:**
- 74 source files parsed, **56 classes** with message structs found, **283 total
messages**.
- **Coverage check passed**: every `Remote*.cs`/`Local*.cs` file under that directory
(56 of them) produced at least one parsed class — nothing silently dropped by the
regex-based parser. (`I*.cs` interface files and the two `ProtocolSplash*.cs` binder
files correctly produce no message structs, since they don't declare any.)
- **Resolves open question #1 from the entry below**: the wire class-name strings for
all 44 remote classes (not just the 4 `Splash::Messaging::*` ones read by hand
earlier) are now known, extracted from every `_priv_remoteClass_X =
port.InitRemoteClass("Wire::Name")` assignment across the `ProtocolSplash*.cs` binder
classes. Full list in `tools/splash-schema.json`'s `wireClassNameBindings`; confirms
the naming scheme is `Splash::<Area>::<Class>` (`Splash::Messaging::*`,
`Splash::Rendering::*`, `Splash::Desktop::*`) with a platform-backend segment inserted
for Nt/Xenon-specific classes (e.g. `Splash::Rendering::Nt::GdiDevice`,
`Splash::Rendering::Xenon::XeDevice`).
- **New finding, not previously noted**: message IDs are **not** guaranteed contiguous
within a class — 8 of the 56 classes have gaps (e.g. `RemoteContext` is `[0, 2, 3,
4]`, no 1; `RemoteAnimation` has gaps at 28, 30, 38, 40, 42). This is consistent with
messages having been removed/deprecated across the engine's lifetime without
renumbering the survivors, which makes sense given the same engine shipped across
Zune and Xbox 360 over several versions. **Implication for codegen**: the native
dispatch table generated per class must be a sparse `{msgid: handler}` map, not a
dense array indexed by position — using array position instead of the literal
`_priv_msgid` value would silently misroute messages for any of these 8 classes.
- Spot-checked extracted output against source by hand for `RemoteBroker`,
`RemoteContext`, and `RemoteDx9Device` (all three read in full during the prior
session) — exact field-for-field match, including the `_priv_objcbOwner`/
`_priv_ctxcbOwner` callback-reference field pair on `RemoteDx9Device.Msg8_CreateVideoPool`
and the base-class-continues-numbering behavior (`RemoteDx9Device : RemoteDevice`
starts at msgid 8, meaning `RemoteDevice` itself owns 07 — confirmed separately by
checking `RemoteDevice`'s own entry in the schema).
**Not yet done (deliberately stopped here to check in before going further):** the
schema doesn't yet capture each base class's *inherited* message range merged into its
subclasses (each class's `messages` list currently only holds what's declared in that
one file — a consumer needs to walk `baseClass` to get the full set for e.g.
`RemoteDx9Device`), and nothing consumes `splash-schema.json` yet to emit actual C++.
Those are the next two concrete steps once resumed.
---
## 2026-07-21 — Decision: generate the native message/dispatch layer instead of hand-transcribing it
**Context:** follow-up to the same-day architecture entry below. The user asked how to
implement the ~60-class `Protocols/Splash/**` message proxy layer on the native
(`UIXrender.dll`) side, and specifically whether an original IDL/schema exists that we
could reuse.
**Finding:** no. `Cn.CodeGenNameAttribute` (`UIX.RenderApi/Cn/CodeGenNameAttribute.cs`)
is `[Conditional("NEVER")]` — compiled out of the shipped assembly — which means we only
ever had its *output* (the decompiled `Remote*.cs`/`Local*.cs` proxy classes), never the
schema/IDL that presumably generated them at Microsoft. That source is not recoverable.
**Decision:** rather than inventing a new IDL syntax and hand-transcribing schema into
it, or hand-writing each native `Remote*` counterpart directly from its C# by eye, we
will mechanically extract a schema (JSON) from the already-decompiled, already-correct
`Remote*.cs`/`Local*.cs` files themselves (class wire-name, per-method `_priv_msgid`,
field names/types, which fields are `BLOBREF`), then use that schema to generate the
native-side message structs and msgid dispatch tables for the new `UIXrender.dll`
implementation, with stub bodies to fill in incrementally per the project's normal
stub-then-real-logic convention. The managed side (`UIX.RenderApi.dll`) is unchanged —
this only affects how the new native implementation's internals get built.
**Rationale:** the pattern across all ~60 classes is extremely uniform (confirmed in the
prior entry via `RemoteBroker.cs`), so hand-transcribing several hundred near-identical
message layouts is where a slipped field order or msgid would silently corrupt wire
format at runtime instead of failing to compile — mechanical extraction from source we
already know is correct removes that risk class entirely. Tradeoff accepted: this is
more upfront tooling investment than hand-writing the first few classes directly.
---
## 2026-07-21 — Reconnaissance: what UIXrender.dll is and how it talks to managed code
**Task:** starting reverse engineering of `UIXrender.dll` (the native backing library for
the `Microsoft.Iris` / Zune UI engine, "UIX"), following the same stage-1 procedure used
for `ZuneDBApi`. This entry is architecture reconnaissance only — no reimplementation
code written yet. Scope was explicitly limited this session to understanding the
transport, message format, and primitive concepts, per user direction.
**New project location (decided with the user, not yet created):** a new reimplementation
project will live inside this repo (`ZuneDev/MicrosoftIris`, checked out as the nested
submodule `libs/MicrosoftIris` under `ZuneDev/ZuneUIXTools`), alongside the existing
`UIX`, `UIX.RenderApi`, `UIXcontrols` projects — not in `ZuneShell.dll`/`ZuneDBApi`. This
log therefore lives at `logs/UIXrender/` at *this* repo's root, mirroring the
`logs/<Namespace>/<Topic>.md` convention already used in `ZuneShell.dll`.
### Method
`UIXrender.dll` (`UIXrender.dll` at the MicrosoftIris repo root, PE32+ x86-64, 1,752,288
bytes) is a **native, unmanaged DLL** — no managed metadata, so ILSpy can't see inside it
and this is genuinely stage-1 (Ghidra territory) for its actual implementation. However,
almost the entire *public surface and wire format* is recoverable without touching Ghidra
at all, because two managed assemblies in this same repo consume it and were themselves
decompiled straight from the original Microsoft assemblies (ILSpy-quality, per
CLAUDE.md's stage-1 rule that pure-managed decompilations are accurate enough to use
directly):
- `UIX/Microsoft/Iris/OS/NativeApi.cs` (from `UIX.dll`) — low-level OS-ish helpers
(tracing, download, resource/DLL loading, memory, DPI, IME, registry-change
notification), 119 `[DllImport("UIXRender.dll")]` entries.
- `UIX.RenderApi/Microsoft/Iris/Render/**` (from `UIX.RenderApi.dll`) — the actual
render-engine client: `Protocol/EngineApi.cs` (init/transport primitives, 15 exports),
`Extensions/ExtensionsApi.cs` (bitmap/sound asset loading, 7 exports),
`Internal/FormApi.cs` (GDI+ init, 2 exports) live in the `Protocol`/`Internal`/
`Extensions`/`Protocols/Splash` folders and account for most of the rest.
- `UIX.RenderApi/Microsoft/Iris/Render/Extensions/eDebugApi.cs` and
`UIX/Microsoft/Iris/Debug/DebugHelpers.cs` P/Invoke a **second** native DLL,
`UIXsup.dll` (5 exports: `DebugDisplayErrorStack`, `DebugSetTimedWriteLines`,
`DebugSetWriteLinePrefix`, `DebugGetCategoryLevel`, `DebugSetCategoryLevel`,
`DebugBreak`) — out of scope for `UIXrender.dll` itself but worth noting as a sibling
native DLL this repo doesn't have a copy of yet (not present in the repo tree; only
referenced by name).
Cross-checked every `Sp*`/`Debug*` P/Invoke name found across the whole repo (`grep -rh
DllImport --include=*.cs`, 199 unique names) against `UIXrender.dll`'s real export table
(`objdump -x`, 225 named exports incl. `DllMain`) — every referenced name is present in
the export table (sanity check passed: `comm -13` diff is empty), and roughly 26 exports
have no managed caller found yet in this repo tree (e.g. `SpRemoteClientInit`,
`SpRemoteCreateClientStreams`, `SpAnimationInputConsumerConnect`,
`SpDynamicSurfaceConnect`, `SpCreateObject`/`SpDestroyObject`/`SpFindClass`,
`SpAttachWndProc`/`SpDetachWndProc`, `SpGetMessageA`/`W`, `SpPeekMessageA`/`W`,
`SpDx9SoundDeviceCheckCaps`) — these are presumably called from C++ code inside
`UIXrender.dll` itself (internal use) or from a managed consumer not present in this
repo checkout; flagged as an open question, not guessed at further.
### Codename and platform scope
The namespace `Microsoft.Iris.Render.Protocols.Splash.*` and class names like
`ProtocolSplashMessaging`, `RemoteBroker` (bound to the wire class name
`"Splash::Messaging::Broker"`), and the `Sp` prefix on every native export strongly
indicate the internal codename for this engine is **"Splash"** — `Microsoft.Iris` is the
managed-facing brand name, "Splash" is what the wire protocol and (presumably) the C++
implementation call themselves internally. This is a direct reading of the code, not an
inference.
The protocol tree has two platform backends alongside the shared core:
`Protocols/Splash/Desktop/Nt/*` and `Protocols/Splash/Rendering/Nt/*` (Windows desktop —
"Nt" = Windows NT), and `Protocols/Splash/Desktop/Xenon/*` /
`Protocols/Splash/Rendering/Xenon/*` ("Xenon" is Microsoft's public codename for the
Xbox 360 CPU/platform). This confirms the Iris/Splash engine was shared between the Zune
desktop software and the Xbox 360 dashboard (Xbox 360's "New Xbox Experience" UI is
publicly known to have used this engine) — noted for context, not something we need to
implement (only the `Nt`/Windows desktop backend matters for reviving Zune).
### Core primitive types (all confirmed from `UIX.RenderApi/Microsoft/Iris/Render/Protocol/*.cs`)
- **`ContextID`** (`ContextID.cs`) — opaque `uint`, identifies one endpoint of a
connection (`NULL`, `CURRENT` = `uint.MaxValue`, or an allocated id). Each side of a
render-port connection (client vs. the Splash engine) has its own `ContextID`.
- **`RENDERHANDLE`** (`RENDERHANDLE.cs`) — opaque `uint` "pointer" to a remote object,
valid only within one render port. Encoding (`HandleTable.cs`) is a packed bitfield
given a `MessageCookieLayout`: `[unique/generation bits][group bits][object-index
bits]`, low-to-high. **Default layout is 4 group bits + 20 object bits**
(`MessageCookieLayout.Default`), leaving 8 bits for a generation/uniqueness counter
that's incremented every time a handle slot is reused (use-after-free / stale-handle
detection — a slot index can be reused but a stale handle referencing the old
generation will be rejected).
- **`RENDERGROUP`** (`RENDERGROUP.cs`) — same opaque-`uint` shape as `RENDERHANDLE`;
handles are allocated within a group (`HandleTable`'s `HandleGroup` bucket), and a
group's handles can be destroyed as a batch (`RemoteContext.SendDestroyGroup`) — this
is how e.g. an entire subtree of UI objects gets torn down in one message instead of
one message per object.
- **`Message`** (`Message.cs`) — the base wire header for every request:
`{ uint cbSize; uint nMsg; RENDERHANDLE idObjectSubject; }` (12 bytes). `nMsg` is a
**per-class message index** (0, 1, 2, ... — see below), not a global message ID.
`idObjectSubject` is the handle of the *class* or *instance* the message targets.
- **`CallbackMessage`** (`CallbackMessage.cs`) — the reverse-direction header (engine →
client): `{ uint cbSize; uint nMsg; RENDERHANDLE idObjectSubject; RENDERHANDLE
hTarget; }` (16 bytes) — same shape plus an explicit `hTarget` naming which client-side
object/callback-handler the message is for.
- **`BLOBREF`** (`BLOBREF.cs`) — opaque `uint` offset/reference to a variable-length blob
(string, byte buffer, or nested sub-message) appended after a fixed-size message
struct. Used for anything of non-fixed size (see `RemoteBroker.Msg2_CreateClass`
below).
### Message batching format
Individual messages are usually sent one at a time (`SendRemoteMessage`
`SendMessageBuffer``SpBufferOpen`), but `RenderPort.BeginMessageBatch`/
`EndMessageBatch` switch to accumulating multiple messages into a **`MessageHeap`**
(`MessageHeap.cs`) — an arena allocator that carves fixed-size (default 8 OS pages) or
oversized "large" blocks out of `Marshal.AllocCoTaskMem`, each block prefixed by a
**`MessageBatchHeader`** (`{ RENDERHANDLE idPredicateBuffer; uint uOffsetFirstEntry; }`)
and containing a singly-linked list of **`MessageBatchEntry`**
(`{ uint uOffsetNextEntry; }`) records, each immediately followed by one `Message`-shaped
payload. `idPredicateBuffer` chains separate memory blocks together server-side when a
batch spans more than one block (each block after the first is sent as its own buffer,
with the previous block's `RENDERHANDLE` as its "predicate"). This is purely a
client-side batching/IPC-efficiency optimization — the underlying per-message wire
format inside a batch entry is identical to a standalone message.
### The one real send primitive: `SpBufferOpen`
Every outbound path — a single message, a batch, or a raw data buffer (e.g. image
pixels via `RenderPort.SendDataBuffer`) — ultimately funnels through
`EngineApi.SpBufferOpen(BufferInfo* phdrData, void* pvData)`:
```
struct BufferInfo {
ContextID idContextSrc;
ContextID idContextDest;
RENDERHANDLE idBuffer; // NULL unless this is one block of a multi-block batch
BufferFlags nFlags; // IsBatch=1, CopyData=2
uint cbSizeBuffer;
}
```
`SpBufferOpen` is the single P/Invoke that crosses from managed into native code to
*deliver* something — there is no separate "receive" P/Invoke call; delivery is a
**callback**, registered once via `SpWrapBufferProc` (wraps a managed
`MessageBufferEventHandler(IntPtr pData, uint hContext, BufferInfo* pBufferInfo, void*
pvBufferData) : int` delegate into a raw native function pointer) and handed to the
engine at `SpInit`/`SpRenderThreadInit` time via `InitArgs.pfnProcessBuffer`. So the
actual flow is: native engine thread receives/produces a buffer → invokes the
managed callback synchronously on its own thread → managed side dispatches by
`BufferInfo.idContextSrc`/`idBuffer` back into `RenderPort.ProcessMessageBuffer`
casts the buffer as a `CallbackMessage*` → looks up the registered `PortCallback`
handler by `idObjectSubject` (offset by 2 — indices 0/1 are reserved) and invokes it.
This is a thin, synchronous, non-blocking-IPC style API — **not** a classic
send/blocking-recv pump for the local case.
### Local vs. Remote channel (two ways `RenderPort` can be connected)
- **`LocalChannel`** (`LocalChannel.cs`) — same process. `Connect()` just calls
`SpRenderThreadInit`, which spins the entire Splash engine up **on its own native
thread inside the current process** (not a separate process) and returns an opaque
thread handle; teardown is `SpRenderThreadUninit`. All communication after that is via
the `SpBufferOpen`/callback mechanism above — cross-thread, not cross-process. This is
what the normal Zune desktop UI uses.
- **`RemoteChannel`** (`RemoteChannel.cs`) — genuinely out-of-process (or out-of-machine).
Uses a distinct set of `EngineApi` entry points: `SpRemoteCreateServerStreams(session,
TransportProtocol, out send, out receive)` creates named send/receive stream handles,
`SpRemoteWaitServerStreamsConnected` blocks until a client attaches, then
`SpRemoteServerInit` wraps the connected streams into an opaque `pSession` used for
actual traffic; `SpRemoteServerUninit` tears it down. `TransportProtocol.cs` enum:
`VC` (1), `TCP` (2), `UDP` (3), `PIPE` (4) — `VC` is unidentified (possibly "virtual
channel", an RDP-style transport; not confirmed, flagged as open question, not
guessed further). The export table also has `SpRemoteClientInit`/
`SpRemoteClientUninit`/`SpRemoteCreateClientStreams` (client-side counterparts) that no
managed code in this repo calls yet — presumably used by a remote-debugging/mirroring
tool, not the main Zune shell.
For reviving Zune, **only the `LocalChannel` path matters** — the remote/IPC path is
real functionality Zune shipped with (likely for a remote UI debugger/mirroring tool)
but is not required for a drop-in-replacement desktop client.
### RPC dispatch: how a "method call" becomes a message
`Protocols/Splash/Messaging/RemoteBroker.cs` is a fully-worked example (decompiled
intact, not reimplemented) of the pattern used by every `Remote*` class under
`Protocols/Splash/**` (~60 files: `RemoteContext`, `RemoteDx9Device`, `RemoteSprite`,
`RemoteAnimation`, `RemoteWindow`, etc.):
1. Each wire-visible native C++ class is given a **name string**
(`"Splash::Messaging::Broker"`, `"Splash::Messaging::Context"`,
`"Splash::Rendering::Dx9::Device"`-style, etc. — pattern inferred from
`"Splash::Messaging::Broker"`, not all names confirmed yet) that gets resolved once,
client-side, to a `RENDERHANDLE` via `RenderPort.InitRemoteClass`
`RemoteBroker.SendCreateClass` (message id 2 on the well-known root "Broker" object,
itself always handle `_rootHandle`, allocated at `RenderPort` construction with no
round-trip needed).
2. Each **method** on that remote class is a private nested struct laid out as
`{ uint _priv_size; uint _priv_msgid; RENDERHANDLE _priv_idObjectSubject; <params...>
}` — i.e. exactly the `Message` header followed by fixed-size parameters, with
`_priv_msgid` a small per-class integer (0, 1, 2, ... — `RemoteBroker` has
`DestroyObject=0`, `CreateObject=1`, `CreateClass=2`) and `_priv_idObjectSubject` set
to either the resolved class handle (static/class-level messages, e.g.
`CreateObject`/`CreateClass`/`DestroyObject` all target the Broker class handle) or a
specific instance's `RENDERHANDLE` (instance methods).
3. Variable-length parameters (strings, nested message blobs) go through `BlobInfo`/
`BLOBREF`: `BlobInfo` is constructed with the fixed struct size, `.Add(value)` appends
the variable data after the struct and returns a `BLOBREF` (byte offset) to store in
the fixed struct's field, `.AdjustedTotalSize` gives the real allocation size, and
`.Attach(Message*)` finalizes it once the struct is filled in.
4. The filled struct is allocated from the port's current `MessageHeap`
(`RenderPort.AllocMessageBuffer`), optionally byte-swapped in place for
`ForeignByteOrder` connections (cross-endian remote scenarios — irrelevant for a pure
x86/x64 revival), and handed to `SendRemoteMessage`.
5. Every C# proxy method has a matching **`Build*`** (constructs the message, doesn't
send) and **`Send*`** (`Build*` + `SendRemoteMessage`) pair — `Build*` exists
separately so a caller can batch several `Build*` calls under one
`BeginMessageBatch`/`EndMessageBatch` without each one triggering its own send.
`Cn.CodeGenNameAttribute` (`UIX.RenderApi/Cn/CodeGenNameAttribute.cs`) exists but is
`[Conditional("NEVER")]` (i.e. compiled out of the shipped assembly, metadata-only) and
the extremely uniform `_priv_`-prefixed field/method naming across every `Remote*`/
`Local*` class in `Protocols/Splash/**` strongly suggests this entire proxy layer —
**and its native C++ counterpart inside `UIXrender.dll`** — was originally
machine-generated from some IDL-like class/message schema, not hand-written. This is an
inference from naming-convention evidence, not a directly-stated fact; flagged as such.
Practically: expect the ~60 `Remote*` classes under `Protocols/Splash/**` to be a
reliable, mechanical map of message IDs and payload struct layouts we can read straight
off the managed side for whichever native handler functions we end up decompiling in
Ghidra — we should not need to guess wire formats there.
### Native export catalog (grouped by subsystem; 199 names confirmed called from managed code in this repo)
Not yet decompiled — this is purely a naming/grouping pass from the P/Invoke call sites,
to plan decompilation order for follow-up sessions:
- **Tracing/logging**: `SpInitializeTracing`, `SpUninitializeTracing`,
`SpUpdateTraceSettings`, `SpLogTrace`, `SpRegisterTraceCallback` (export-only).
- **Memory**: `SpMemAlloc`, `SpMemFree`, `SpFreeDib`.
- **Download / HTTP**: `SpFileDownload`, `SpDownloadGetBuffer`, `SpDownloadClose`,
`SpHttpDownload`, `SpHttpStartup`, `SpHttpShutdown`, `SpHttpFlushProxyCache`.
- **Resource / DLL loading (markup assembly loading)**: `SpLoadDll`, `SpFreeDll`,
`SpLoadBinaryResource`, `SpLoadFontResource`, `SpCreateDllLoadResultFactory`,
`SpCreateDllLoadResult`, `SpSendDllSchemaUnloadNotification`.
- **Misc Win32/OS**: `SpGetDpi`, `SpExtractDroppedFileNames`, `SpGetMouseCursorInfo`,
`SpRegNotifyChangeKey`, `SpRegRevokeNotifyChangeKey`, `SpPostDeferredImeMessage`,
`SpRegisterImeCallbacks`, `SpUnregisterImeCallbacks`, `SpCreateNotifyWindow`,
`SpDestroyNotifyWindow`, `SpAttachWndProc`/`SpDetachWndProc` (export-only),
`SpGetMessageA`/`W`, `SpPeekMessageA`/`W` (export-only, distinct from the
`InitArgs`-driven `EngineApi.SpPeekMessage`/`SpWaitMessage`).
- **Engine core / threading / transport** (`EngineApi.cs`, detailed above): `SpInit`,
`SpUninit`, `SpBufferOpen`, `SpWrapBufferProc`, `SpPeekMessage`, `SpWaitMessage`,
`SpInvoke`, `SpRenderThreadInit`, `SpRenderThreadUninit`,
`SpRemoteCreateServerStreams`, `SpRemoteWaitServerStreamsConnected`,
`SpRemoteServerInit`, `SpRemoteServerUninit`, `SpObjectRelease`,
`SpCallDeferredInvokeProc`; export-only: `SpRemoteClientInit`,
`SpRemoteClientUninit`, `SpRemoteCreateClientStreams`, `SpCreateObject`,
`SpDestroyObject`, `SpFindClass` (native side of the class-name→handle resolution
described above).
- **Direct3D 9 / graphics**: `SpDx9CompileEffect`, `SpGdiplusInit`, `SpGdiplusUninit`,
`SpGetStateCache`, `SpSetStateCache`; export-only: `SpDx9SoundDeviceCheckCaps`,
`SpAnimationInputConsumerConnect`, `SpAnimationInputPublisherConnect`,
`SpDynamicSurfaceConnect` (video/live-surface hookup, e.g. `VideoElement`).
- **Native reflection / type-schema system** (large, self-contained subsystem — this is
almost certainly the runtime binding layer that lets markup (`.uix`/`.uib`) reference
native and managed gadget classes/properties/events by name): `SpGetTypeID`,
`SpSetSchemaID`, `SpQueryTypeCount`, `SpQueryTypeName`, `SpGetTypeSchema`,
`SpQueryBaseType`, `SpQueryPropertyCount`, `SpQueryPropertyName`,
`SpQueryPropertyType`, `SpQueryPropertyCanRead`, `SpQueryPropertyCanWrite`,
`SpQueryPropertyIsStatic`, `SpQueryPropertyNotifiesOnChange`, `SpGetPropertySchema`,
`SpGetPropertyValue`, `SpSetPropertyValue`, `SpQueryMethodCount`,
`SpQueryMethodName`, `SpQueryMethodIsStatic`, `SpQueryMethodParameterCount`,
`SpQueryMethodReturnType`, `SpGetMethodParameterTypes`, `SpGetMethodSchema`,
`SpInvokeMethod`, `SpQueryConstructorCount`, `SpQueryConstructorParameterCount`,
`SpGetConstructorParameterTypes`, `SpGetConstructorSchema`, `SpInvokeConstructor`,
`SpQueryEventCount`, `SpQueryEventName`, `SpQueryEventIsStatic`, `SpGetEventSchema`,
`SpQueryEnumCount`, `SpQueryEnumName`, `SpQueryEnumIsFlags`, `SpQueryEnumValueCount`,
`SpGetEnumNameValue`, `SpGetEnumSchema`, `SpInvokeEnumToString`, `SpInvokeToString`,
`SpIsRuntimeImmutable`, `SpGetMarshalAs`, `SpQueryForMarshalAsInterface`,
`SpCreateNativeString`, `SpCreateNativeImage`, `SpConvertStringToManaged`,
`SpConvertImageToManaged`, `SpCopyString`, `SpGetStringHandle`, `SpGetImageHandle`.
- **Rich text editing** (`SpRichText*`, ~25 exports): full native text-box engine —
`SpRichTextBuildObject`/`DestroyObject`, `Measure`, `Rasterize`, `SetContent`,
`GetSimpleContent`/`Length`, `Copy`/`Cut`/`Paste`/`Delete`/`Undo`/`Redo`(`CanUndo`),
`Scroll`/`ScrollToPosition`, `SetSelectionRange`, `SetFocus`, `SetReadOnly`,
`SetWordWrap`, `SetScale`, `SetOversampleMode`, `SetMaximumLength`, `SetDetectUrls`,
`SetScrollbars`, `OnTimerTick`, `ForwardKeyCharacter`/`ForwardKeyState`/
`ForwardMouseInput`/`ForwardImeMessage`, `GetNaturalBounds`,
`DestroyGlyphRunInfo`.
- **Simple text** (measurement/rendering only, no editing): `SpSimpleTextBuildObject`,
`SpSimpleTextDestroyObject`, `SpSimpleTextMeasure`, `SpSimpleTextMeasurePossible`,
`SpSimpleTextIsAvailable`.
- **Virtualized list backing store** (`SpUIXList*`, ~15 exports — native side of a
data-bound/virtualized ListBox): `Add`, `Insert`, `Remove`, `RemoveAt`, `Move`,
`Clear`, `GetItem`, `SetItem`, `GetCount`, `IndexOf`, `IsItemAvailable`,
`FetchSlowData`, `WantSlowDataRequests`, `RegisterCallbacks`,
`UnregisterCallbacks`, `NotifyVisualsCreated`, `NotifyVisualsReleased`.
- **Data binding / query system** (`SpData*` — likely how markup-declared bindings reach
into a query result, plausibly the eventual bridge to `ZuneDBApi`'s query/property-bag
system reimplemented at a different layer of this same revival project; not confirmed,
flagged as a hypothesis to revisit): `SpDataProviderConstructQuery`,
`SpDataProviderReportDataMapping`, `SpDataQueryRefresh`,
`SpDataQueryGetEnabledProperty`/`SetEnabledProperty`,
`SpDataQueryGetResultProperty`/`SetResultProperty`,
`SpDataQueryGetStatusProperty`/`SetStatusProperty`, `SpDataQueryNotifyInitialized`,
`SpDataBaseObjectGetProperty`/`SetProperty`,
`SpDataBaseObjectGetInternalHandle`/`SetInternalHandle`,
`SpDataBaseObjectGetTypeHandle`.
- **XML parsing** (`SpXmlLite*` — thin wrapper around Windows' XmlLite COM API; almost
certainly how `.uix` markup files get parsed): `SpXmlLiteCreateXmlReader`,
`SpXmlLiteDeleteXmlReader`, `SpXmlLiteRead`, `SpXmlLiteGetLocalName`,
`SpXmlLiteGetPrefix`, `SpXmlLiteGetQualifiedName`, `SpXmlLiteGetValue`,
`SpXmlLiteIsEmptyElement`, `SpXmlLiteMoveToFirstAttribute`/`MoveToNextAttribute`,
`SpXmlLiteGetLineNumber`/`GetLinePosition`.
- **Asset loading** (`Extensions/ExtensionsApi.cs`): `SpBitmapLoadFile`,
`SpBitmapLoadRaw`, `SpBitmapLoadResource`, `SpBitmapLoadBuffer`, `SpBitmapDelete`,
`SpSoundLoadBuffer`, `SpSoundDispose`.
- **Debug helpers, separate DLL `UIXsup.dll`** (not `UIXrender.dll` — out of scope here,
noted for later): `DebugDisplayErrorStack`, `DebugSetTimedWriteLines`,
`DebugSetWriteLinePrefix`, `DebugGetCategoryLevel`, `DebugSetCategoryLevel`,
`DebugBreak`.
### Error space
`Protocol/EngineApi.cs`'s `IFC` (If-Failed-Cleanup — same convention name used
elsewhere in this project already) helper hardcodes ~40 specific HRESULT values in the
`-2147221xxx` range with human-readable messages (e.g. `-2147221493` = "object is busy",
`-2147221383` = "message not found during class registration", `-2147221354` = "unable
to connect to remote renderer") — this is effectively a free, already-decompiled HRESULT
error table for this custom facility; no need to reverse it from the native binary.
### Open questions (logged per CLAUDE.md's uncertainty procedure — not guessed at)
1. What native class-name strings exist beyond `"Splash::Messaging::Broker"` /
`"Splash::Messaging::Context"` / `"Splash::Messaging::DataBuffer"` /
`"Splash::Messaging::ContextRelay"` (seen in `ProtocolSplashMessaging.Init`) — the
`Protocols/Splash/Rendering/**` classes almost certainly have their own
`"Splash::Rendering::..."`-style names but none were read this session. Needed before
`SpFindClass`/`SpCreateObject` can be meaningfully decompiled.
2. What `TransportProtocol.VC` (value 1) stands for — not stated anywhere in the
decompiled managed code.
3. Which ~26 exports with no managed caller in this repo are called from (native-internal
use inside `UIXrender.dll`, or a managed consumer not checked into this repo).
4. Full native struct layout / calling convention details (this DLL is native x64, so
this is genuinely Ghidra work for a future session) — everything above is the
managed-visible contract, not the C++ implementation.
### Next steps (not started)
Decompilation via Ghidra of `UIXrender.dll`'s exports, starting with the smallest/most
self-contained subsystems (tracing, memory, DPI — mirroring the "start at the top, work
down" plan discussed with the user) once this session's architecture pass is reviewed.
+305
View File
@@ -0,0 +1,305 @@
# UIXrender.dll engine core — implementation log
Append-only. Do not edit previous entries.
---
## 2026-07-22 — Wired the *remaining* EngineApi.cs methods managed-direct (all 9)
**Task:** the first managed-direct pass (entry below) converted only the 6 transport
methods `LocalChannel` uses. Now that the full surface is implemented, wire the other 9
`UIX.RenderApi/.../Protocol/EngineApi.cs` methods — `SpPeekMessage`, `SpWaitMessage`,
`SpInvoke`, the four `SpRemote*`, `SpDx9CompileEffect`, `SpObjectRelease` — to call
UIXrender's managed implementation instead of `[DllImport]`, keeping the net461 `#else`
branch on the native path exactly as the existing 6 do.
**Single source of truth, established deliberately.** Rather than duplicate logic between
UIX.RenderApi and UIXrender's `[UnmanagedCallersOnly]` shims, I made `EngineService` (the
public facade both sides already share) the one implementation and routed *both* the
native shims (`UIXrender/Interop/EngineApi.cs`) and UIX.RenderApi through it. So each
operation now has exactly one body: `EngineService.{PeekMessage,WaitMessage,Invoke,
RemoteCreateServerStreams,RemoteWaitServerStreamsConnected,RemoteServerInit,
RemoteServerUninit,ReleaseRemoteStream,Dx9CompileEffect}`. `EngineService`'s doc comment
was updated: it's no longer strictly pointer-free (opaque `IntPtr` handles pass through,
and `Invoke` has the one `unsafe` spot to call the function pointer a caller hands it),
and that's called out explicitly.
**Two things that would have been silent bugs, verified against the real call sites rather
than assumed:**
1. **`SpInvoke`'s only in-repo caller is `IRenderEngine.InterThreadWake`, which passes a
*null* function pointer** (`SpInvoke(ctx, IntPtr.Zero, IntPtr.Zero, false)`) purely to
wake a pump. The earlier native shim returned `E_INVALIDARG` for a null pointer — which
would have been wrong for this caller. `EngineService.Invoke` now treats a null pointer
as a well-defined `S_OK` no-op (there is no blocking pump to wake in this
implementation), and both entry paths use that.
2. **The remote path had a handle-model mismatch that would have corrupted memory.**
`RemoteChannel.Connect` calls `SpObjectRelease` on the two stream handles from
`SpRemoteCreateServerStreams`. In managed-direct mode those handles are UIXrender
`GCHandle`s, **not** COM vtable pointers — so the native shim's generic "release through
the vtable slot 2" `SpObjectRelease` would have dereferenced garbage. Fixed by:
- `RemoteServerConnection` now hands out **two distinct** `GCHandle`s (it previously
aliased send == receive) with a **refcount of 2**; `ServerInit` adds a third (the
session). Each release drops one reference and frees *its own* handle; the sockets
are torn down exactly once, when the last reference goes. This matches
`RemoteChannel.Connect`'s exact sequence (create → init → release send → release
receive → later uninit session).
- Managed-direct `SpObjectRelease` routes to `EngineService.ReleaseRemoteStream`
(drop-a-reference), **not** the vtable path. The native shim keeps the vtable-release
behaviour for genuine native COM callers — the two object models are distinct, like
`SpWrapBufferProc` already is. Documented at both sites.
**Callback representation, handled consistently.** `RemoteServerConnection.ServerInit` was
refactored to take a pointer-free `BufferReceivedHandler` (like `RenderThread`), so its
`ReadLoop` no longer assumes a native function pointer. The native shim adapts the raw
`InitArgs.pfnProcessBuffer` function pointer into one; UIX.RenderApi's managed-direct path
resolves the `GCHandle`'d `MessageBufferEventHandler` and reuses the existing
`AdaptCallback` helper. In practice `RemoteChannel` connects with **no** receive callback
(it builds `InitArgs` with the 2-arg ctor → `pfnProcessBuffer == 0`), so the handler is
normally null — but the adaptation is correct for the non-null case too, rather than
mis-reading a `GCHandle` as a function pointer.
**`SpDx9CompileEffect`** routes to `EngineService.Dx9CompileEffect()` (`E_NOTIMPL`, per
FullSurface.md decision 1); the UIX.RenderApi side zeroes its out-params so a caller
ignoring the HRESULT still sees a well-defined "no blob". Note its caller
(`Dx9EffectResource`) wraps it in `IFC`, which throws on failure — same observable outcome
as before on net8.0, where there was no working native `SpDx9CompileEffect` to call
anyway; now it fails deterministically instead of at DLL-resolution time.
**Verified:** `UIX.RenderApi` (net8.0) builds clean (only pre-existing decompiled-code
warnings); whole solution builds except the same pre-existing `SimpleDebugClient`/
`SimpleIrisApp` errors; NativeAOT publish still exports **exactly 193/193** (the wiring
changed no signatures); `Tests/UIXrender.Engine.Tests` extended to **53 checks, all
passing**, including new ones that exercise the managed-direct remote path
(`RemoteCreateServerStreams` VC→`E_NOTIMPL`, TCP→two distinct handles, and release of both
tearing the connection down once without a double-free).
---
## 2026-07-22 — Managed-direct calling: EngineService layer + first modification of UIX.RenderApi
**Task:** per user direction, restructure so `UIX.RenderApi` can call directly into
`UIXrender`'s managed implementation (no P/Invoke, no native marshaling) whenever both
are loaded in the same .NET process, while `Interop/EngineApi.cs`'s
`[UnmanagedCallersOnly]` exports remain for genuine native callers. Logging before/while
writing code, not after.
**Key fact driving the design:** `[UnmanagedCallersOnly]`-attributed methods cannot be
called directly from C# at all (compiler error CS8901) -- they can only be reached via a
function pointer, mimicking how native code calls them. So "call directly" necessarily
means a *separate* idiomatic layer beneath the shim, not calling the shim methods
themselves with fewer steps.
**Design:** split `UIXrender/Engine/` into a public, pointer-free API
(`EngineService`/`IRenderThreadHandle`/`BufferReceivedHandler`, using
`ReadOnlySpan<byte>` and ordinary C# delegates) that `ContextRegistry`/`RenderThread`
are refactored to use internally -- no `IntPtr`/raw function pointers anywhere in
`Engine/` after this change. All pointer/unsafe marshaling concentrates in
`Interop/EngineApi.cs` (adapting native `BufferInfo*`/`void*`/raw function pointers to
and from the idiomatic API), which is exactly the "backwards-compatible entrypoint for
native callers" the user asked to keep.
**Scope of the `UIX.RenderApi` change, decided deliberately, not implicitly:** only the
6 `EngineApi.cs` methods `UIXrender` actually implements today get their `[DllImport]`
bodies replaced with real calls into `EngineService`: `SpInit`, `SpUninit`,
`SpWrapBufferProc`, `SpRenderThreadInit`, `SpRenderThreadUninit`, `SpBufferOpen`. These
happen to be the *entire* set `RenderPort.cs`/`LocalChannel.cs` (the real send/receive
path) use -- confirmed by re-checking call sites, not assumed. Everything else in
`EngineApi.cs` (`SpPeekMessage`, `SpWaitMessage`, `SpInvoke`, the `SpRemote*` family,
`SpDx9CompileEffect`, `SpObjectRelease`) stays untouched `[DllImport]` since `UIXrender`
doesn't implement those yet -- converting them now would break, not improve, anything
that still needs them. **This is the first modification of a previously-decompiled,
until-now-untouched file in this whole effort** (`UIX.RenderApi/Microsoft/Iris/Render/Protocol/EngineApi.cs`),
worth flagging explicitly: only method *bodies* change, not signatures or any other
file's call sites, and it's an internal-only class (nothing outside `UIX.RenderApi.dll`
depends on how it's implemented), so this doesn't touch the drop-in-replacement contract
CLAUDE.md cares about for this project's actual public surface.
**A real, additional simplification found along the way, not just pointer-hiding:**
`SpWrapBufferProc`'s original job was handing a native caller a real function pointer to
invoke later. In the managed-direct path, the delegate the caller passes
(`MessageBufferEventHandler`) doesn't need to become a function pointer at all -- it can
be stored via a `GCHandle` and resolved straight back to the original delegate object,
invoked as a normal C# delegate call. No `delegate* unmanaged<...>`/`calli` anywhere on
this path. A `BufferInfo*`/`void*` pointer still gets reconstructed at the point of
calling the stored `MessageBufferEventHandler` delegate, because that delegate's own
signature (already-decompiled, unchanged) is pointer-shaped -- not something avoidable
without touching `RenderPort.cs`, which is out of scope here.
**Implementation, extending this entry as each piece lands:**
1. `UIXrender/Engine/` refactored to be entirely pointer-free: `BufferReceivedHandler.cs`
(custom delegate, not `Action<>`, specifically so `ReadOnlySpan<byte>` is legal as a
parameter -- `Span<T>`/`ReadOnlySpan<T>` are ref structs and can't be generic type
arguments, but a hand-declared delegate can take one directly), `IRenderThreadHandle.cs`,
`ContextRegistry.cs` (now `ConcurrentDictionary<uint, BufferReceivedHandler>`, no more
`IntPtr`), `RenderThread.cs` (`IDisposable`/`IRenderThreadHandle`, holds the delegate
directly, no function pointer casts), and the new public `EngineService.cs` facade
(`StartRenderThread`, `SendBuffer`) -- the one type meant to be called by anything
that wants managed-direct access.
2. `UIXrender/Interop/EngineApi.cs` rewritten to route through `EngineService` --
`SpRenderThreadInit` builds a `BufferReceivedHandler` closure that reconstructs a
`BufferInfo*`/`void*` call only at the point of invoking the raw native function
pointer it received; `SpBufferOpen` converts its `void*`/`cbSizeBuffer` into a
`ReadOnlySpan<byte>` and calls `EngineService.SendBuffer`. Verified this still builds
clean on Linux.
3. `UIX.RenderApi/Microsoft/Iris/Render/Protocol/EngineApi.cs` -- the 6 in-scope
methods' `[DllImport]` bodies replaced with real implementations calling
`EngineService` directly, converting between `UIX.RenderApi`'s own (unchanged,
pre-existing) `Protocol.ContextID`/`RENDERHANDLE`/`Internal.HRESULT`/nested
`EngineApi.BufferInfo` types and `UIXrender`'s `Interop.*` equivalents at the
boundary via the `ToUInt32`/constructor accessors those types already expose -- kept
as two separate struct sets rather than unifying them, since `UIXrender` must stay
independent of `UIX.RenderApi` (it's also P/Invoked directly by `UIX.dll`'s
`NativeApi.cs`, a different assembly entirely) and a reference in the other direction
would be circular. **`net461` handled explicitly, not overlooked**: `UIX.RenderApi.csproj`
has `EnableNetFXTarget=true` (still targets .NET Framework 4.6.1 on Windows), which
can't reference a net8.0-only project at all -- the new `ProjectReference` to
`UIXrender.csproj` is conditioned out for `net461`
(`Condition="'$(TargetFramework)' != 'net461'"` in the csproj), and all 6 modified
methods are wrapped `#if !NETFRAMEWORK` with the original, unmodified `[DllImport]`
declarations preserved in the `#else` branch -- net461 keeps calling the real native
`UIXRender.dll` exactly as before, nothing regresses for that target.
4. `Tests/UIXrender.Engine.Tests` (new) -- exercises `EngineService` directly via a
`ProjectReference` to `UIXrender.csproj`, no P/Invoke or published DLL involved. Since
`Engine/` is now fully pointer-free, this is the first thing in this whole effort that
could actually be **run and observed**, not just compiled, from this (Linux) machine.
Ran it: `StartRenderThread`'s on-start synthetic invocation fires, `SendBuffer`
correctly routes to the registered handler with the right source context/buffer
handle/payload bytes, and `SendBuffer` to a disposed/unregistered context fails
cleanly (`E_FAIL`) rather than hanging or throwing. All 8 checks passed. Wired into
CI (runs in the same `windows-latest` job today for simplicity, but noted in the
workflow as cross-platform-capable if a Linux job gets added later).
**Verified on Linux:** whole solution builds clean except the same pre-existing,
unrelated `SimpleDebugClient`/`SimpleIrisApp` errors; `Tests/UIXrender.Engine.Tests`
actually **runs** and all 8 checks pass. **Not verified here, needs Windows CI:** the
`#else`/net461 branches in `EngineApi.cs` (Directory.Build.props only adds `net461` when
`IsOsPlatform('Windows')`, so this machine never compiles that branch at all -- a real
gap, flagged rather than assumed fine); the Phase 0 native-ABI spike from the entry
below, still pending its first CI run.
**Follow-up cleanup, prompted by the user asking why `SpRenderThreadInit` had grown so
much logic:** worth recording the answer, not just the fix. Three sources, sorted by
whether they're removable:
- **Not removable** -- inherent to preserving the original API exactly: (1)
`SpWrapBufferProc`/`SpRenderThreadInit` are two separate calls in the original API,
and the only channel between them matching that shape is `InitArgs.pfnProcessBuffer`
as a plain `IntPtr`, which forces the GCHandle-store/GCHandle-resolve round trip
rather than passing the delegate straight through; (2) `MessageBufferEventHandler`'s
signature is still pointer-shaped (`BufferInfo* pBufferInfo, void* pvBufferData`),
unchanged since `RenderPort.cs` still expects to receive calls that way, so
*something* has to reconstruct a pointer-based call no matter how clean
`EngineService`'s own API is; (3) `UIX.RenderApi`'s own `ContextID`/`RENDERHANDLE`/
`BufferInfo` and `UIXrender`'s `Interop.*` equivalents are two genuinely separate type
systems (by design, to avoid the circular-reference problem noted earlier), so
crossing that boundary needs explicit conversion.
- **Removable, and fixed**: the `BufferInfo`-reconstruction closure was inlined directly
inside `SpRenderThreadInit`, making the method read as one undifferentiated block
instead of clear steps -- and it's structurally the *same* pattern (build a
`BufferInfo`, get a pointer to the span, invoke) as `UIXrender/Interop/EngineApi.cs`'s
own `SpRenderThreadInit` uses for native callers, just invoking a stored C# delegate
at the end instead of a raw function pointer. Extracted into a new private
`AdaptCallback(MessageBufferEventHandler, ContextID) : BufferReceivedHandler` helper,
so `SpRenderThreadInit` itself now reads as four named steps (resolve the stored
delegate → adapt it → start the thread → wrap the handle) instead of one block with an
inline lambda in the middle. Same behavior, verified: `UIX.RenderApi.csproj` still
builds clean and `Tests/UIXrender.Engine.Tests` still passes all 8 checks after the
change.
## 2026-07-22 — Phase 0 spike: SpBufferOpen/SpWrapBufferProc/SpRenderThreadInit
**Task:** per the approved plan, implement `UIXrender.dll`'s core transport primitives
well enough to prove the riskiest interop mechanism works: a native OS thread the CLR
didn't create calling back into a managed delegate. Logging as I go, per the user's
explicit reminder not to wait until done.
**Structural decision, before writing code:** creating a new shared project,
`UIXInterop` (`Microsoft.Iris.Interop` namespace), for native-marshaling helpers used by
both `UIXsup` and `UIXrender` (currently just ANSI/Unicode `byte*`/`char*``string`
conversion — `UIXsup/Interop/DebugApi.cs` already has a private `PtrToString` that would
otherwise get copy-pasted here). Per the user's explicit go-ahead to split into
additional projects to cut duplication. `UIXsup` will be refactored to use it too, so
there's a single implementation instead of two copies drifting apart. This is narrower
than fully splitting `UIXrender` into per-subsystem projects (Engine, Graphics, ...) —
that's still a folder-level split inside `UIXrender/`, per the plan's project-structure
section; only genuinely cross-project shared code moves to its own assembly.
**Scoping `SpInit`/`SpUninit` out of this spike, logged rather than guessed:**
`EngineApi.SpUninit()` takes **no parameters at all** — no context id, nothing — which
is hard to reconcile with `SpInit(ref InitArgs args)` registering a specific
`args.idContext`. Checked whether `LocalChannel.Connect()` (the code path real Zune
actually uses) calls `SpInit`/`SpUninit` at all: it doesn't — it only calls
`SpRenderThreadInit`/`SpRenderThreadUninit` (`UIX.RenderApi/Microsoft/Iris/Render/Protocol/LocalChannel.cs`).
`SpInit`/`SpUninit` must serve some other call path not yet located (possibly the
alternate "IGMM_STANDARD messaging model" mentioned in one of `EngineApi.IFC`'s HRESULT
error strings, implying a selectable threading model distinct from the dedicated-thread
one `SpRenderThreadInit` provides) — no confident answer, so no guess: implementing both
as plain `HRESULT.S_OK`-returning stubs for now (the explicitly-allowed stub convention),
real logic deferred until the actual caller/semantics are found. Not blocking this
session's work since the spike's own critical path (`LocalChannel`'s path) never calls
either.
**Design note, not an unknown — a legitimate simplification:** `SpWrapBufferProc`'s
purpose in the original was presumably to hand the CLR-marshaled delegate pointer to
native code in a form the original C++ implementation's internal calling convention
needed (possibly a real trampoline/thunk). Since this reimplementation *is* the native
side now, and controls both ends of the call, there's nothing to adapt — the pointer the
CLR hands us when marshaling `MessageBufferEventHandler` is already directly callable
from a `delegate* unmanaged<...>` field. `SpWrapBufferProc` becomes a real (not fake)
but simple implementation: validate, store/echo the pointer, return `S_OK`. Recording
this so a future reader doesn't wonder why it's "too simple" compared to what the
original probably did internally — the simplification is deliberate, not a missed spot.
**Implementation complete for this phase:**
1. `UIXInterop` project (`UIXInterop/NativeString.cs`) — `AnsiToString`/`UniToString`
pointer conversion, shared by `UIXsup` (refactored to use it, removing its local
copy) and `UIXrender`.
2. `UIXrender/Interop/` wire structs — `HRESULT.cs`, `ContextID.cs`, `RENDERHANDLE.cs`,
`BufferInfo.cs` (+ `BufferFlags`), `InitArgs.cs` — bit-for-bit mirrors of the
decompiled `UIX.RenderApi/Microsoft/Iris/Render/{Internal,Protocol}/*.cs` structs
read in the first architecture session. `InitArgs.pfnTimeout` is `IntPtr` here, not a
delegate type (see file comment: the CLR marshals the managed side's delegate field
to a plain function pointer before the struct crosses over, so that's the actual wire
representation).
3. `UIXrender/Engine/ContextRegistry.cs` — `ContextID -> (callback pointer, callback
data)` registry, real logic (`ConcurrentDictionary`), the smallest coherent slice of
"Engine core" (the plan's subsystem 1) needed for `SpRenderThreadInit`/`SpBufferOpen`
to mean anything together. Callback stored as raw `IntPtr`, cast to a
`delegate* unmanaged<...>` only at the actual invocation site, to avoid putting a
function-pointer-typed field into a generic collection's value type.
4. `UIXrender/Engine/RenderThread.cs` — real (not mocked) thread lifecycle: `Start`
spins up a genuine `System.Threading.Thread`, which invokes the registered callback
once with synthetic `BufferInfo` data then blocks on a `ManualResetEventSlim` until
`Stop` signals it, joins, and unregisters. Not the full message-pump/dispatch loop
(that's later, larger "Engine core" work) — this proves the interop mechanism itself.
5. `UIXrender/Subsystems/Tracing/` — `TracingState.cs` (minimal real init-flag state)
and `TracingApi.cs` (`SpInitializeTracing`/`SpUninitializeTracing` exports). Full
tracing (`SpUpdateTraceSettings`/`SpLogTrace`) deferred to when the Tracing subsystem
gets its own pass per the plan's sequencing.
6. `UIXrender/Interop/EngineApi.cs` — the remaining `[UnmanagedCallersOnly]` exports:
`SpInit`/`SpUninit` (S_OK stubs, per the open question above), `SpWrapBufferProc`
(pass-through, per the design note above), `SpRenderThreadInit`/`SpRenderThreadUninit`
(wrap `RenderThread` behind a `GCHandle`-backed opaque `IntPtr` handle, matching what
the managed side's `out IntPtr pThread` expects to receive and later pass back),
`SpBufferOpen` (looks up the destination context in `ContextRegistry`, invokes its
callback synchronously).
7. `Tests/UIXrender.Interop.Tests` — the Phase 0 exit-criteria harness: a plain console
app (no test framework dependency added) with its own P/Invoke declarations (can't
reference `EngineApi.cs` directly — it's `internal` in `UIX.RenderApi.dll`, and this
harness is meant to exercise the published `UIXrender.dll` the way an arbitrary
external caller would anyway). Registers a callback via `SpWrapBufferProc`, starts a
thread via `SpRenderThreadInit`, waits up to 5s for the callback to fire with the
expected context id, then tears down via `SpRenderThreadUninit`. Explicit pass/fail
per check, non-zero exit code on any failure, wired into
`.github/workflows/uixrender-ci.yml` after the publish steps (locates the published
DLL and the harness's build output via `Get-ChildItem`, copies the DLL alongside the
harness exe, runs it, fails the job on a non-zero exit code).
**Verified on Linux:** the whole solution builds clean except the same pre-existing,
unrelated `Tests/SimpleDebugClient`/`SimpleIrisApp` errors noted in `UIXsup.md` (not
touched by this work). **Not yet verified:** actual execution — the interop spike can
only run on Windows (P/Invoking a real NativeAOT-published `UIXrender.dll`), which
requires the CI workflow to actually run, not checked yet as of this entry.
+354
View File
@@ -0,0 +1,354 @@
# UIXrender.dll — full public surface completion log
Append-only. Do not edit previous entries. Newest entries at the top (matching
`Architecture.md`/`EngineCore.md`'s actual observed order despite the header wording).
---
## 2026-07-22 — Complete: 193/193 exports implemented, verified against the built binary
**Result: the export surface is complete and mechanically verified.** All 193 `Sp*` entry
points that the four managed consumers P/Invoke for are implemented, and the
NativeAOT-published `UIXrender.so` exports **exactly** those 193 — no missing export (which
would be a runtime `EntryPointNotFoundException` waiting to happen) and no stray one.
Verified by diffing `nm -D --defined-only UIXrender.so` against the `extern` declarations
in `NativeApi.cs`/`ExtensionsApi.cs`/`FormApi.cs`/`Protocol/EngineApi.cs`; the diff is
empty. That check is now a CI step (new `linux` job) so it can't silently regress.
**Verification actually performed (not just "it compiles"):**
- `dotnet build MicrosoftIris.sln` — clean apart from the 5 pre-existing
`SimpleDebugClient`/`SimpleIrisApp` errors already documented in `EngineCore.md`
(`IDebuggerClient.InterpreterStep`, `DebugSettings.DebugConnectionUri` — debugger APIs,
untouched by this work).
- `dotnet publish -r linux-x64 -p:PublishAot=true` — succeeds, **zero trim/AOT warnings**
(see the AOT section below; they were fixed, not suppressed wholesale).
- `Tests/UIXrender.Engine.Tests` — extended from 8 checks to **49, all passing**, and they
exercise real behaviour rather than construction: UIXList mutation/move/availability,
`WaveParser` against a hand-built RIFF/WAVE buffer, `XmlLiteReader` walking elements/
attributes/text/EOF, `RichTextObject` copy-paste/undo/redo/read-only/max-length,
`TextMetrics` wrapping, `SchemaRegistration` reflecting over a probe type, and
`UIXVariant` round-trips. Reached via a new `InternalsVisibleTo`, since the exports
themselves are `[UnmanagedCallersOnly]` and uncallable from C#.
**AOT correctness fixes made while publishing (these were real, not cosmetic):**
- `Enum.GetValues(Type)` in `EnumSchema` was an **IL3050** — it constructs a `T[]` of the
enum type at runtime, which can genuinely fail under NativeAOT, and this project
publishes AOT. Replaced with `Enum.GetValuesAsUnderlyingType`, which needs no dynamic
array construction.
- The reflection trim warnings (IL2026/IL2070/IL2075) were resolved by annotating the
schema entry points with `[DynamicallyAccessedMembers(All)]` and, where the types
genuinely come from a host-chosen assembly loaded at runtime (`SpLoadDll`), by
`[UnconditionalSuppressMessage]` **with a written justification** — the trimmer cannot
see into a runtime-chosen assembly by definition, so the host must root markup-visible
assemblies itself. Suppressing with a reason keeps a future *real* warning from being
lost in the noise.
**What is real vs. what is honestly not — the part worth reading before trusting this:**
Real, working logic: memory, tracing, engine transport (local + a genuinely functional
TCP/named-pipe/UDP remote channel), the ~48-export reflection/schema system, UIXList,
data-binding/query, XmlLite (`System.Xml`-backed), image decode (StbImageSharp,
cross-platform), WAV/PCM parsing, module + embedded-resource loading, HTTP/file download,
string & image handle marshaling, the `IRawUIXServices` callback bridge, IME
registration/dispatch, registry-change notification (Windows), and the entire rich-text
**editing** model (content, selection, clipboard, undo/redo, read-only, max length, wrap,
scale, scrollbars, timers, key forwarding) with real `IRichTextCallbacks` notifications.
**Not real, and deliberately failing loudly rather than faking success** — each returns
`E_NOTIMPL` (or `0`/null) with a `// TODO`, so a caller gets an honest error instead of
plausible-looking garbage:
- `SpDx9CompileEffect` — no cross-platform effect-compilation abstraction exists; binding
`d3dcompiler_47.dll` would violate the dependency policy (decision 1 below).
- `SpRichTextRasterize` — needs a glyph rasterizer. Returning an empty bitmap would render
as invisible text and read as a layout bug, so it fails instead.
- `SpBitmapLoadResource`, `SpLoadFontResource` — Win32 resource sections / platform font
registration, no equivalent yet.
- `SpCreateNotifyWindow`, `SpExtractDroppedFileNames` — Win32 message-only window and
shell drag-drop.
- `TransportProtocol.VC` — meaning never recovered (`Architecture.md` open question #2).
**Approximate, and flagged as the single biggest caveat: `TextMetrics`.** Measurement is
derived from font-height ratios typical of Latin UI faces, not from real font tables.
Line height and baseline are close to exact; **per-character advance is an average, so
measured text width is approximate and will not match a real rasterizer**. Word wrapping
itself is real logic — only the width feeding it is estimated. Practical consequence:
layout driven by these numbers will be plausible but not pixel-accurate. This is the top
open question below; it should be replaced wholesale by a font backend, not built upon.
**Documented assumption (per CLAUDE.md's uncertainty procedure):** `SpXmlLiteCreateXmlReader`
receives a byte buffer whose encoding is unstated — `NativeXmlReader` feeds it either a
pinned UTF-16 string (no BOM) *or* a raw file buffer (typically UTF-8). Real XmlLite
sniffs the encoding, so this does the same: BOM first, then the "every second byte is
zero" pattern that identifies BOM-less UTF-16 ASCII, else UTF-8.
**Things read off the decompiled sources rather than guessed** (each would have been a
silent, hard-to-find bug):
- `ListChanged`'s type codes are `Microsoft.Iris.Data.UIListContentsChangeType`, confirmed
from `DllProxyList.ListChanged`'s direct cast — not invented.
- `SpXmlLiteRead`/`MoveToFirstAttribute`/`MoveToNextAttribute` must return a **failing**
HRESULT at end-of-stream, because `NativeXmlReader` gates its loops on `SUCCEEDED(...)`
— returning `S_FALSE` (the natural XmlLite idiom) would have produced an infinite loop.
- `SpXmlLiteCreateXmlReader`'s `length` is a **byte** count (`content.Length * 2` over a
pinned string), not a character count.
- Every `bool`/`out bool` in the original `DllImport`s carries no `[MarshalAs]` override,
so the CLR marshals it as a 4-byte Win32 `BOOL` — these are `int` on the export side,
not `byte`. Getting this wrong would have corrupted the stack on every such call.
- The COM callback interfaces are all `ComInterfaceType.InterfaceIsIUnknown`, so their
vtable slots start at 3; slot assignments follow each interface's declaration order,
read from its own source file.
**Open questions carried forward (not guessed at):**
1. **Text measurement/rasterization needs a real font backend** — the largest remaining
gap by far. Needs a decision on taking a text-shaping dependency (SixLabors.Fonts,
HarfBuzz) versus platform-native text APIs behind an abstraction. Deliberately not
decided unilaterally in this pass.
2. `TransportProtocol.VC` (value 1) — still unidentified.
3. The ~26 "export-only" natives with no managed caller anywhere in this repo
(`SpCreateObject`, `SpFindClass`, `SpAttachWndProc`, `SpGetMessageA/W`, ...) — **not**
implemented, because there is no signature to mirror and CLAUDE.md forbids guessing.
They're absent from the built library; if a consumer surfaces, they can be added then.
4. Whether the schema subsystem's CLR-type projection matches what real `.uix` markup
expects at runtime — it satisfies the *shape* of every accessor, but nothing in this
repo exercises markup against it end-to-end yet.
---
## 2026-07-22 — Landed: shared infrastructure, memory, tracing, transport remainder, schema, assets, lists
Written and building clean (`dotnet build UIXrender -f net8.0`) as of this entry. Logging
mid-flight per the user's reminder, not at the end.
**Shared infrastructure (new, used by every subsystem below):**
- `Engine/HandleTable.cs` — the one place opaque `IntPtr` handles are minted. Every export
that hands back a "pointer" (schema objects, bitmaps, XML readers, rich-text objects,
data queries) returns a `GCHandle`, never a real object address. Safe because *nothing*
outside `UIXrender` is permitted to dereference these — all four managed consumers type
them as opaque `IntPtr`/`HANDLE`. Generalises the convention `SpRenderThreadInit`
already used for its thread handle.
- `UIXInterop/NativeString.cs` extended with `InternUni`/`AllocUni`. Non-obvious detail
worth recording: exports like `SpQueryTypeName`/`SpQueryPropertyName` hand out a `char*`
that **the managed caller never frees**, so the callee must own it for the process
lifetime. Interning by value bounds that — schema names come from a fixed registered set,
so the pool stops growing after each name's first request, instead of leaking one
allocation per call. Per-call values (query results, `ToString` output) use the
non-interned `AllocUni` instead.
- `Interop/Com/ComVtable.cs` — callback interfaces (`IUIXListCallbacks`, `IRawUIXServices`,
`IRichTextCallbacks`, `IImeCallbacks`) can't be received as `[MarshalAs(Interface)]`
params by an `[UnmanagedCallersOnly]` method, so they arrive as `IntPtr` and are called
through their vtable. Slot numbering starts at 3 because all four are
`ComInterfaceType.InterfaceIsIUnknown` — verified from each interface's own attributes,
not assumed.
**Subsystems landed:**
1. **Memory**`SpMemAlloc`/`SpMemFree` real via `NativeMemory`. **Correction made while
writing**: `SpFreeDib` initially routed to GDI's `DeleteObject`; removed. This
implementation never *creates* DIB sections (bitmaps are our own unmanaged allocations),
so calling GDI to free them would have been both wrong and a needless graphics-API
dependency. Now frees through the same allocation path that produced it.
2. **Tracing**`SpUpdateTraceSettings`/`SpLogTrace` now real (prefix/timestamp/category
formatting, debugger + optional file sink), joining the existing init/uninit pair.
3. **Engine transport remainder**`SpInvoke` (real: sync direct call / async via
thread pool, through the supplied function pointer), `SpWaitMessage` (real timeout
wait), `SpPeekMessage` (documented "no message available" — there is no Win32 message
queue behind this yet and `LocalChannel` never calls it), `SpObjectRelease` (real
`IUnknown::Release` through the object's own vtable), and the four `SpRemote*` entry
points.
4. **Remote channel** (`Subsystems/Remote/RemoteServerConnection.cs`) — genuinely working
out-of-process transport over `TcpListener`/`NamedPipeServerStream`/`UdpClient`, with a
length-prefixed `BufferInfo` + payload framing and a reader thread that dispatches into
the *same* `EngineService`/`ContextRegistry` the local channel uses — so a remote
context is indistinguishable from a local one to the rest of the engine.
`TransportProtocol.VC` returns `E_NOTIMPL`: its meaning was never recovered (open
question #2 in `Architecture.md`) and CLAUDE.md forbids guessing.
5. **Native reflection / type-schema** (~48 exports, the single largest group) — real,
projecting **CLR** types through the original export surface via `System.Reflection`.
Recorded as a deliberate substitution, not a stub: the C++ gadget classes the original
reflected over don't exist here and never will, but every accessor keeps its original
signature and semantics, so the managed caller can't tell the difference. Type IDs are
process-wide and stable (the managed side round-trips them across unrelated calls);
IDs 015 are reserved for the primitives `UIXVariant` carries directly.
6. **UIXList** (17 exports) — real store, and it keeps the *virtualization* distinction
real (per-slot availability, `FetchSlowData`, `WantSlowDataRequests`) rather than
pretending every item is resident, because the managed ListBox's scrolling depends on
it. **The `ListChanged` type codes are not guessed**: `DllProxyList.ListChanged`
(`UIX/Microsoft/Iris/CodeModel/Cpp/DllProxyList.cs`) casts the argument straight to
`Microsoft.Iris.Data.UIListContentsChangeType`, so the numbering was read off that
decompiled enum.
7. **Assets**`SpBitmapLoadFile`/`LoadBuffer` real cross-platform decode via
StbImageSharp; `SpBitmapLoadRaw` real (pixel copy, no decoder involved);
`SpSoundLoadBuffer` real (RIFF/WAVE chunk parser — no codec needed since the managed
side's only declared format is `WAVE_FORMAT_PCM`). Non-obvious detail: stb decodes to
RGBA byte order but Iris's `SurfaceFormat.ARGB32` is `0xAARRGGBB` as a little-endian
uint, i.e. BGRA in memory — the channel swap in `BitmapStore` is required, not
incidental. `SpBitmapLoadResource` is `E_NOTIMPL` + TODO: Win32 resource sections have
no cross-platform equivalent and this project has no PE resource reader yet.
**Remaining for this session:** data-binding (`SpData*`), XmlLite, GDI+ init pair,
misc OS (DPI/download/DLL+resource loading/notify window/IME/dropped files/registry
notify/HTTP), native services callbacks + string/image handle marshaling, and rich
text/simple text.
---
## 2026-07-22 — Dependency policy: "Silk.NET abstractions, not specific graphics APIs"
**Trigger:** mid-implementation the user narrowed the earlier "prefer Silk.NET over stubs"
direction to: *use Silk.NET **abstractions** wherever possible, avoid **specific graphics
APIs***. That is a materially different instruction from "bind Silk.NET's D3D packages",
and it changes three decisions already in flight. Recording the reasoning before acting on
it, since two of the three outcomes are "no dependency added", which would otherwise look
like the instruction was ignored.
**Verified first, not assumed:** NuGet is reachable from this machine and
`Silk.NET.Core 2.23.0` restores cleanly (probe project in the scratchpad, not committed).
So "couldn't get the package" is *not* the reason for any decision below — each is a
deliberate fit judgement.
**Decision 1 — `SpDx9CompileEffect`: no `Silk.NET.Direct3D.Compilers`, returns
`E_NOTIMPL` with a `// TODO`.** `Silk.NET.Direct3D.Compilers` is a thin binding over
`d3dcompiler_47.dll` — i.e. exactly "a specific graphics API", Windows-only, and the
narrowest possible thing to hard-code into a project whose stated goal is cross-platform
revival. Silk.NET has **no** cross-platform shader/effect-compilation abstraction to use
instead (checked: `Silk.NET.Core`/`Maths`/`Windowing`/`Input` are the abstraction layer,
and none covers HLSL effect compilation; the D3D/OpenGL/Vulkan packages are all
API-specific bindings). Since there's no abstraction available and the instruction rules
out the specific API, the honest outcome is an explicit `E_NOTIMPL` + `TODO` rather than
a Windows-only binding smuggled in behind `#if WINDOWS`. Logged as an open question
below, **not** silently dropped.
**Decision 2 — no `Silk.NET.Windowing`/`Silk.NET.Input` for `SpCreateNotifyWindow`,
`SpGetMouseCursorInfo`, `SpGetDpi`.** These *look* like the right fit (they're genuine
cross-platform abstractions over exactly these concepts), and I nearly took them. They're
wrong here for a structural reason worth writing down: `Silk.NET.Windowing` owns a
window **and its event loop**, backed by a native GLFW/SDL binary loaded at runtime.
`UIXrender.dll` is a library P/Invoked *into an already-running host process* that already
owns its window and pump (Zune's own shell). A library that spins up a second windowing
backend to answer "what's the system DPI" would be both heavier and less correct than
asking the OS. So these stay platform-gated OS calls (`#if WINDOWS` + `// TODO` for other
platforms), which is what CLAUDE.md's platform rule prescribes anyway. Note these are
**OS** APIs (DPI, registry-change notification, IME), not graphics APIs — the user's
constraint doesn't bite here.
**Decision 3 — image decode uses `StbImageSharp`, not `System.Drawing.Common`.** This one
*does* add a dependency, and it's the case where the instruction changed the answer for
the better. The earlier plan (entry below) had `SpBitmapLoadFile`/`LoadBuffer`/
`LoadResource` decoding via `System.Drawing.Common` under `#if WINDOWS` with
cross-platform decode deferred — i.e. GDI+, a specific graphics API, Windows-only since
.NET 6, and a stub everywhere else. `StbImageSharp` is pure managed (no native
dependency), cross-platform, NativeAOT-safe (matters: this project is
`PublishAot`/`NativeLib=Shared`), and is the decoder Silk.NET's own tutorials pair with
its APIs. Net effect: PNG/JPEG/BMP/TGA decode becomes **real logic on every platform**
instead of Windows-only-plus-a-stub. Strictly better against both the project's
cross-platform goal and CLAUDE.md's "real logic preferred over stubs".
**Unchanged by this:** everything non-graphical (memory, tracing, transport, remote
channel, reflection/schema, UIXList, data-binding, XmlLite, sound) was never going to
touch a graphics API and keeps the real implementations described in the plan entry
below.
---
## 2026-07-22 — Plan: complete the entire native export surface this session
**Task, per explicit user direction:** implement stages 1 & 2 for the *entire* remaining
`UIXrender.dll` public surface (not just the `EngineApi.cs` transport methods worked on
so far), aiming for a complete library by the end of this session, the same bar applied
to `ZuneDBApi`. Real implementations are preferred over stubs (matching CLAUDE.md's stage
2 convention), with Silk.NET preferred over hand-rolled P/Invoke where a fitting binding
package exists (mainly Direct3D9 compile-effect). Existing `UIXsup`/`UIXrender` code may
be rewritten as needed as long as the public surface/signatures stay unchanged. Logging
before/while writing, not after, per this repo's established convention.
**Scope, counted from a full grep of every `[DllImport("UIXRender.dll")]` across the four
managed consumer files** (`UIX/Microsoft/Iris/OS/NativeApi.cs`,
`UIX.RenderApi/Microsoft/Iris/Render/Extensions/ExtensionsApi.cs`,
`UIX.RenderApi/Microsoft/Iris/Render/Internal/FormApi.cs`,
`UIX.RenderApi/Microsoft/Iris/Render/Protocol/EngineApi.cs`): every export that has a
concrete managed-side signature in this repo gets a matching `[UnmanagedCallersOnly]`
counterpart in `UIXrender`, organized into `UIXrender/Subsystems/<Group>/` folders
mirroring `Architecture.md`'s subsystem grouping. Exports flagged in `Architecture.md` as
"export-only" (no managed caller found in this repo, e.g. `SpAttachWndProc`,
`SpCreateObject`, `SpGetMessageA/W`) are **not** implemented here — there is no known
signature to mirror, and CLAUDE.md forbids guessing; logged as still-open, not silently
dropped.
**Subsystem plan and treatment (real logic vs. documented stub), decided before writing
any code:**
1. **Memory** (`SpMemAlloc`/`SpMemFree`/`SpFreeDib`) — real, via `NativeMemory`.
2. **Tracing** (`SpUpdateTraceSettings`/`SpLogTrace`, extending the existing
`SpInitializeTracing`/`SpUninitializeTracing`) — real, `Console`/optional file sink.
3. **Engine transport remainder** (`SpPeekMessage`, `SpWaitMessage`, `SpInvoke`, the 4
`SpRemote*` remote-channel entry points, `SpDx9CompileEffect`, `SpObjectRelease`) —
real where the existing managed-direct `EngineService`/`ContextRegistry` design
extends naturally (peek/wait/invoke against the per-context registry); `SpRemote*`
implemented for real using `System.Net.Sockets`/named-pipe-equivalent streams since
.NET's own stream types are a genuine, working transport, not a stub.
4. **Native reflection / type-schema system** (~50 `SpQuery*`/`SpGet*`/`SpInvoke*`
exports) — real, backed by a new managed `SchemaRegistry` that projects real
`System.Reflection` (`Type`/`PropertyInfo`/`MethodInfo`/`ConstructorInfo`/`EventInfo`)
over registered CLR types through `GCHandle`-wrapped opaque `IntPtr` handles (the same
opaque-handle convention already established for `RenderThread`). This is a genuine
design substitution (real C++ types no longer exist to reflect over), not a stub —
flagged here as a deliberate architecture decision, not an assumption to second-guess
later.
5. **UIXList** (`SpUIXList*`, ~15 exports) — real, a `List<UIXVariant>`-backed store.
6. **Data binding / query** (`SpData*`, ~15 exports) — real, a managed registry mirroring
the query lifecycle (construct/refresh/get-set property/notify).
7. **XmlLite** (`SpXmlLite*`, ~11 exports) — real, backed by `System.Xml.XmlReader`
(cross-platform, not Windows-only XmlLite COM, but drop-in behaviorally).
8. **Asset loading** (`SpBitmap*`/`SpSound*`) — real where feasible: raw/buffer bitmap
paths use direct pixel copy (no decode needed); file/resource decode paths use
`System.Drawing.Common` on Windows (`#if WINDOWS`) since that's an in-box decoder,
cross-platform decode deferred (stub + TODO, logged as an open question — no
cross-platform image codec is referenced anywhere else in this repo yet). Sound
buffer load is real (WAV header parse, no codec needed for PCM).
9. **Graphics** (`SpGdiplusInit`/`Uninit`, `SpGetStateCache`/`SpSetStateCache`,
`SpDx9CompileEffect`) — GDI+ init/uninit are real (documented as legitimate no-ops:
managed GDI+ has no manual startup step). `SpDx9CompileEffect` real via
`Silk.NET.Direct3D.Compilers` (D3DCompiler), Windows-only (`#if WINDOWS`), since D3D9
effect compilation is inherently a Windows/Direct3D concept — cross-platform has no
equivalent, not guessed at.
10. **Misc OS/Win32** (`SpGetDpi`, `SpGetMouseCursorInfo`, `SpExtractDroppedFileNames`,
`SpRegNotifyChangeKey`/`Revoke`, `SpCreateNotifyWindow`/`Destroy`,
`SpRegisterImeCallbacks`/`Unregister`, `SpPostDeferredImeMessage`) — real on Windows
(`#if WINDOWS`, direct Win32 P/Invoke, consistent with CLAUDE.md's platform-gating
rule), stub + `// TODO` elsewhere since these are inherently Win32-shaped concepts.
11. **Rich text / Simple text** (~30 exports) — real editing model (content buffer,
cut/copy/paste/delete/undo/redo, selection, timer/scroll/wrap/readonly/scale state)
backed by an actual `StringBuilder` + undo stack; measurement real via
`System.Drawing.Graphics.MeasureString` on Windows (`#if WINDOWS`), a documented
fixed-width-estimate fallback elsewhere (logged assumption, not silently guessed);
rasterization (`SpRichTextRasterize`) real ARGB bitmap production on Windows via
`System.Drawing`, stub + TODO cross-platform (no font-shaping library referenced
elsewhere in this repo to build on without adding a large new, unrequested
dependency — flagged as an open question rather than guessed).
**Interface-typed callback parameters** (`IUIXListCallbacks`, `IRawUIXServices`,
`IRichTextCallbacks`, `IImeCallbacks`) can't be received as `[MarshalAs(UnmanagedType.
Interface)]` COM parameters in an `[UnmanagedCallersOnly]` method (not blittable) — native
callers pass a raw vtable pointer regardless of what the managed P/Invoke declares, so
`UIXrender`'s side receives `void*`/`IntPtr` and invokes through the vtable directly at
each interface's declared method slots (offset by the standard `IUnknown` 3:
QueryInterface/AddRef/Release), the same "reconstruct the interface, don't invent raw
pointer types" approach CLAUDE.md's *COM objects* section describes.
**Dependent-type mirrors needed** (independent `UIXrender/Interop/*.cs` copies, since
`UIXrender` cannot reference `UIX`/`UIX.RenderApi` — circular dependency, same reasoning
as the existing `ContextID`/`RENDERHANDLE`/`HRESULT`/`BufferInfo`/`InitArgs` mirrors):
`UIXVariant`, `Color`/`Size`/`Rectangle`/`RectangleF`/`ColorF`, `RawImageFormat`,
`SurfaceFormat`, `ImageRequirements`/`ImageInformation`/`ImageHeader`/`ImageData`/
`HSpBitmap`, `TextStyle.MarshalledData`, `TextMeasureParams.MarshalledData` (+
`FormattedRange`), `DataProviderQueryStatus`, `ShutdownReason`, `TransportProtocol`,
`NativeXmlNodeType`, `Win32Api.HANDLE`/`MSG`/`LOGFONTW_STRUCT` (the `Microsoft.Iris.OS`
variant, since that's what `NativeApi.cs` itself resolves to for its own P/Invokes).
Full field layouts confirmed via an Explore pass over the real decompiled sources before
writing any of these — not guessed.
**Not attempted, logged as genuinely out of scope rather than silently skipped:**
`UIXsup.dll`'s own surface beyond what's already there (separate DLL, separate stage-1
target — `logs/UIXsup.md` presumably covers it, not duplicated here); the ~26
"export-only" natives with no managed call site in this repo (no signature to mirror).
Proceeding subsystem-by-subsystem in the order above; each subsystem's own progress gets
logged as it lands, same as `EngineCore.md`'s existing entries.
+146
View File
@@ -0,0 +1,146 @@
# UIXrender.dll — rendering & message pump log
Append-only. Do not edit previous entries. Newest first.
---
## 2026-07-22 — Landed: real StbTrueType text rendering + backend-agnostic message pump
Built, AOT-published, and tested. Summary of what's now real vs. before.
**Text rendering (was: ratio-approximated measurement + `E_NOTIMPL` rasterization):**
- `Subsystems/Text/LoadedFont.cs` — a TrueType face over StbTrueTypeSharp (pure managed,
zero GPU). Pins the font bytes (stb stores the data pointer) and exposes scaled
advance/kerning/ascent/line-height and glyph coverage rasterization.
- `FontStore.cs` — resolves a face name to a font: runtime-registered fonts
(`SpLoadFontResource`, now real — reads the embedded resource and registers it) →
platform system-font discovery (Windows/Linux/macOS dirs) → first-available fallback.
Cached, thread-safe. Returns null when nothing resolves, so callers degrade to the ratio
metrics rather than faking.
- `TextLayout.cs` — real measurement over actual glyph advances + kerning, with the old
ratio `TextMetrics` kept strictly as the no-font fallback. Word wrap is real either way.
- `GlyphRun.cs` — the object `hGlyphRunInfo` points at; `Rasterize(color)` composites stb
coverage bitmaps into a straight-alpha ARGB32 buffer (BGRA-in-memory, matching
`SurfaceFormat.ARGB32`). `TextBitmap` owns that buffer; `SpFreeDib` (repointed to
`HandleTable.Free`) releases it.
- Wired: `SpSimpleTextMeasure` now produces a real glyph run + real geometry;
`SpRichTextRasterize` (was `E_NOTIMPL`) composites it to a bitmap;
`SpRichTextMeasure`/`SpRichTextGetNaturalBounds`/`SpSimpleTextMeasurePossible` use the
real font. Outline/shadow passes and rich-text multi-run `rrcb` remain TODO.
**Message pump (was: `PeekMessage`→0, `WaitMessage`→`Thread.Sleep`, `Invoke(null)`→no-op):**
- `Subsystems/Os/MessagePump.cs` — a `ConcurrentQueue<Action>` + `ManualResetEventSlim`,
with a lost-wakeup guard (reset only when empty, re-check after). No windowing/GPU
dependency. `IWindowMessageSource` is the pluggable seam for a host/optional backend to
inject OS messages during Peek.
- Wired through `EngineService`: `PeekMessage` → drain+run queued work, report
ProcessedMessage(1)/None(0); `WaitMessage` → real bounded wait that returns early on a
post; `Invoke(null ptr)``PostWake` (**the InterThreadWake fix** — a blocked
`WaitMessage` now actually wakes); async `Invoke(real ptr)` → deferred onto the pump
thread; `SetWindowMessageSource` exposes the seam. `SpPostDeferredImeMessage` now posts
its dispatch onto the pump (genuinely deferred). `SpCreateNotifyWindow` now returns
`S_OK` + a token (was `E_NOTIMPL`, which threw inside `UIForm.Initialize`'s `IFC`).
**Verified:** `dotnet build UIXrender -f net8.0` clean; whole solution builds except the
pre-existing `SimpleDebugClient`/`SimpleIrisApp` errors; NativeAOT `linux-x64` publish
succeeds with **zero trim/AOT warnings** (StbTrueTypeSharp is pure-managed, AOT-safe) and
exports **exactly 193/193** (surface unchanged). `Tests/UIXrender.Engine.Tests` extended
to **66 checks, all passing**, exercising real behaviour: proportional metrics ('WWW' >
'iii', which the ratio path could never produce), word-wrap growth, rasterized ink
(non-zero alpha pixels — glyphs actually drawn), deferred-work execution on Peek,
`InterThreadWake` unblocking a blocked `WaitMessage` in ~50ms (not the 5s timeout),
`WaitMessage` honouring its timeout, and `PeekMessage` reporting the right WorkResult.
**Still open / not done (flagged, not hidden):**
- **No on-screen compositing / real window.** By design (backend-agnostic): text
measurement + CPU rasterization is complete, but presenting pixels to a screen needs a
window+GPU backend, which stays out of core. `IWindowMessageSource` is the seam for a
later optional backend package. `SpPeekMessage` therefore never returns
`NewUserMessage(2)` (no HWND to Win32-dispatch).
- Text: DPI scaling (fontHeightPts treated as pixels for now), complex-script shaping
(stb is Latin-oriented), and outline/shadow/underline rendering passes.
- Font family matching is filename-heuristic, not the TTF `name` table / fontconfig — a
bold/italic style request currently resolves the family's plain face.
## 2026-07-22 — Course-correction: no GLFW/SDL; backend-agnostic pump + pluggable window seam
**What happened:** I asked the user (AskUserQuestion) how to back the message pump and
they picked "Silk.NET.Windowing real window". While setting that up the user pushed back:
*"why are you adding a dependency on GLFW? That's not backend-agnostic."* Correct.
**The constraint, stated plainly:** `Silk.NET.Windowing` is an abstraction over `IWindow`,
but there is **no backend-agnostic way to create a real OS window**`Window.Create()`
needs a concrete platform provider (GLFW *or* SDL) registered via a backend package at
runtime, or it throws "no platform registered". So "real window" unavoidably pulls in
GLFW or SDL, which violates the backend-agnostic requirement that has governed this whole
effort (and matches FullSurface.md decision 2: UIXrender is loaded into a host that
already owns its window). Nothing windowing-related had been added to the real project
yet — only a throwaway scratchpad probe — so there was nothing to revert.
**Revised design (what's actually being built):**
- **Message pump = a backend-agnostic message queue** owned by the render-session thread.
This is exactly and only what the render engine's loop uses the pump for, verified from
the call sites (`UIX.RenderApi/.../Internal/RenderEngine.cs`):
`IRenderEngine.WaitForWork``SpWaitMessage(timeout)` (block until work or timeout);
`IRenderEngine.ProcessNativeEvents``SpPeekMessage(drain)` (drain + report a
WorkResult); `IRenderEngine.InterThreadWake``SpInvoke(ctx, NULL, NULL, false)` (wake
a blocked wait from another thread). A `ConcurrentQueue` + a wait handle implements all
three correctly with no windowing/GPU dependency at all.
- **`ProcessNativeEvents`'s `NewUserMessage` (2) path is deliberately never taken.** That
branch forwards a Win32 `MSG` to `RenderWindow.ForwardWindowMessage` then calls
`Win32Api.TranslateMessage`/`DispatchMessage` — genuine Win32/HWND dispatch that only
exists on Windows with a real window. The backend-agnostic pump has no HWND, so
`SpPeekMessage` returns `ProcessedMessage` (1) when it ran queued work, else `0` (none),
and leaves `out msg` default — which routes `ProcessNativeEvents` down its non-Win32
branch (`return TestFlag(nResult, 1)`), exactly right for a windowless pump.
- **Pluggable window seam, no dependency:** a small `IWindowMessageSource` interface plus
`EngineService.SetWindowMessageSource(...)` lets a host that owns a window (or an
*optional, separate* Silk.NET.Windowing backend package that never ships in UIXrender
core) inject OS window/input messages into the pump later. The pump harvests from the
source during Peek if one is registered. This preserves the "windowing can plug in"
intent of the user's original menu choice while keeping UIXrender core free of GLFW/SDL.
**Deferred-invoke semantics fixed at the same time:** `SpInvoke` with a **null** function
pointer is `InterThreadWake` → post a wake to the pump (this is *the* fix the user's "must
implement a pump" requirement was about: a blocked `SpWaitMessage` now actually wakes). A
**non-null** async `SpInvoke` enqueues the callback to run *on the pump thread* during the
next Peek (real "deferred invoke to the render thread" semantics), not on a random
threadpool thread as the interim wiring did; a non-null sync `SpInvoke` runs inline.
---
## 2026-07-22 — Plan: text rendering (StbTrueTypeSharp) + message pump
**Task (user):** implement rendering, specifically text rendering, and — mandatory — a
message pump so `SpPeekMessage`/`SpWaitMessage` work correctly. Backend-agnostic; pause
and ask if no suitable abstraction exists (which is why the font backend was an
AskUserQuestion — Silk.NET has no font/text abstraction, and the community option
SilkyNvg was rejected for being OpenGL-only).
**Font backend chosen (user):** StbTrueTypeSharp — pure-managed stb_truetype port,
zero-GPU, NativeAOT-clean, pairs with the StbImageSharp decoder already referenced. Gives
glyph metrics (real measurement) and CPU glyph rasterization (real `SpRichTextRasterize`).
No complex-script shaping — line/word layout is built here (the ratio-based `TextMetrics`
already did the layout; only the per-glyph *width* becomes real).
**Text subsystem shape (`Subsystems/Rendering/Text/`):**
- `FontStore` — resolves a TextStyle font-face name to a loaded font: registered fonts
(via `SpLoadFontResource`), then platform system-font discovery (Windows `Fonts`, Linux
`/usr/share/fonts` & friends, macOS), then a first-available fallback. Cached,
thread-safe. When nothing resolves, callers fall back to the existing ratio metrics and
rasterization reports `E_NOTIMPL` — honest degradation, not a fake.
- `LoadedFont` — wraps `stbtt_fontinfo` + v-metrics; scaled advance/kerning/ascent/line
height, and glyph coverage-bitmap rasterization.
- `TextLayout` — real measurement (word wrap over real advances) and a measured run.
- `GlyphRun` — a measured run stored behind a `HandleTable` handle (this becomes the
`hGlyphRunInfo` the managed side round-trips from measure to rasterize).
**Rasterize lifecycle (read off the managed consumers, not guessed):**
`SpRichTextRasterize` returns `phTextBitmap` + `ppvBits` + size, which
`Microsoft.Iris.Drawing.RichText.Rasterize` wraps in a `Dib`
(`UIX/.../RenderAPI/Drawing/Dib.cs`) that frees `phTextBitmap` via **`SpFreeDib`**. So the
rasterizer allocates one unmanaged ARGB buffer, returns a `HandleTable` handle as
`phTextBitmap` and the buffer pointer as `ppvBits`, and `SpFreeDib` is repointed to
`HandleTable.Free` (its previous body referenced a since-removed GDI helper and only ever
mattered on Windows; the only producer of "DIBs" in this reimplementation is this
rasterizer). `SpRichTextDestroyGlyphRunInfo` already frees the glyph-run handle.
+99
View File
@@ -0,0 +1,99 @@
# UIXsup.dll — implementation log
Append-only. Do not edit previous entries.
---
## 2026-07-22 — Phase 0 warm-up: project config + export catalog correction
**Task:** implement UIXsup.dll's real export surface as the lowest-risk NativeAOT
rehearsal before tackling UIXrender.dll's harder `SpBufferOpen`/`SpWrapBufferProc`
callback spike (per the approved plan, `/home/yoshiask/.claude/plans/dazzling-dreaming-waffle.md`).
Logging as I go per CLAUDE.md's *Dealing with unknowns and uncertainty* / *Logging*
sections, not after the fact.
**Correction to the plan's export count:** the plan (via a Plan sub-agent's earlier
research) listed 6 exports for `UIXsup.dll` including `DebugBreak`. Re-reading the
actual decompiled call sites (`UIX.RenderApi/Microsoft/Iris/Render/Extensions/eDebugApi.cs`,
`UIX/Microsoft/Iris/Debug/DebugHelpers.cs`) before implementing turned up only **5** real
`[DllImport("UIXsup.dll")]` exports:
- `DebugDisplayErrorStack(string, string, int, string, string) : bool`
- `DebugSetTimedWriteLines(bool)`
- `DebugSetWriteLinePrefix(string)`
- `DebugGetCategoryLevel(DebugCategory) : byte`
- `DebugSetCategoryLevel(DebugCategory, byte)`
`DebugHelpers.Break()` (which the agent's summary conflated with a `UIXsup.dll` export)
actually calls `Win32Api.DebugBreak()` — a *different* function, from `kernel32.dll`,
via `UIX/Microsoft/Iris/OS/Win32Api.cs`, not `UIXsup.dll` at all. Not implemented as
part of this project; it's the real Win32 API, already correctly targeted by the
existing managed code. This matches the original architecture-log entry from the first
research session in this effort, which had the correct 5-export list — the agent's later
research introduced the inaccuracy. Verified directly against source before writing any
code, per CLAUDE.md's requirement not to build on an unverified claim.
**Marshaling facts, read directly from source (not inferred):** none of the 5
`[DllImport("UIXsup.dll")]` declarations specify a `CharSet`. .NET's default
`CharSet` for `[DllImport]` when unspecified is `Ansi` — this is a fact about the
already-compiled, unchanged managed assemblies (baked into their P/Invoke metadata at
Microsoft's original compile time), not a guess, so every `string` parameter here
(`stMessage`, `filename`, `title`, `stackTrace`, `stPrefix`) is ANSI (`LPStr`), not
UTF-16. This matters because `[UnmanagedCallersOnly]` methods must use blittable
parameter types only — `string`/`bool` aren't blittable — so the native side takes
`byte*` (converted via `Marshal.PtrToStringAnsi`) and `int` (matching the default
4-byte Win32 `BOOL` marshaling of C# `bool`) instead.
**csproj/CI config, corrected after user feedback (see thread — recorded here for the
paper trail):**
1. First pass wrongly restricted `UIXrender.csproj`/`UIXsup.csproj` to
`net8.0-windows10.0.22000` only, reasoning from the *original* binary's Windows-only
surface. User corrected this: the whole point of the earlier Silk.NET decision is
staying cross-platform, so platform-specific TFM locking contradicts that — reverted
to plain `net8.0`, with the actual OS/arch chosen only at publish time via `-r <rid>`.
2. This surfaced a real, separate technical conflict: Directory.Build.props' default
Windows-conditional TFM list included `net461`, which cannot use NativeAOT/
`[UnmanagedCallersOnly]` at all (net7.0+ only). Flagged to the user rather than
guessed at silently (two live options: override away from it locally, or fix the
default). **User fixed it at the source**`Directory.Build.props` now gates
`net461` behind an opt-in `$(EnableNetFXTarget)` flag (`UIX`/`UIX.RenderApi` opt in,
`UIXrender`/`UIXsup` don't), rather than each new project needing to route around it.
3. Found and fixed a genuine functional gap, not a style issue: `PublishAot=true` alone
publishes a native **executable**, not an exporting shared library — NativeAOT needs
`<NativeLib>Shared</NativeLib>` to actually emit a DLL with a real export table for
`[UnmanagedCallersOnly]`-attributed methods. Added to both csproj. Without this, every
`[UnmanagedCallersOnly]` method written for either project would silently fail to be
callable via P/Invoke despite compiling cleanly — a bug in Phase 0's exit criteria
that wouldn't have surfaced until CI's Windows publish step, so worth flagging clearly
here even though it's a well-documented NativeAOT requirement (not a reverse-engineered
unknown).
4. `.github/workflows/uixrender-ci.yml` added (`windows-latest`, since NativeAOT can't
cross-compile Linux→Windows and today's managed consumers only run on Windows);
publish steps pass `-f net8.0` explicitly since the project may carry a second TFM
(`net8.0-windows10.0.22000`, from Directory.Build.props' Windows-conditional block)
that isn't the one we ever intend to AOT-publish.
5. Verified locally (Linux): both `UIXrender.csproj` and `UIXsup.csproj` build clean
(0 errors) as plain `net8.0`. Actual `PublishAot`/`NativeLib` publish behavior can
only be verified on Windows (CI) — not checked yet as of this entry.
**Implementation complete for this phase**, all under `UIXsup/`:
- `DebugCategory.cs` — enum mirror, ordinal-exact.
- `DebugState.cs` — shared mutable state (timed-write-lines flag, prefix string,
per-`DebugCategory` byte levels array, bounds-checked on both get/set since the enum
value crosses an unmanaged boundary).
- `Interop/DebugApi.cs` — the 5 `[UnmanagedCallersOnly]` exports, `byte*`/`Marshal.PtrToStringAnsi`
for the ANSI string params, `int` (not `bool`) for `DebugSetTimedWriteLines`'s
parameter and `DebugDisplayErrorStack`'s return (matching the default 4-byte Win32
`BOOL` marshaling `bool` gets on the managed P/Invoke side), operating on `DebugState`.
`dotnet build UIXsup/UIXsup.csproj` verified clean (0 warnings, 0 errors) on Linux —
this only proves compilation, not that NativeAOT publish actually produces a working
export table; that's still unverified until CI runs on Windows (next step).
**Known, deliberate scope gap (not a reverse-engineering unknown — a sequencing
decision):** `DebugDisplayErrorStack` almost certainly showed an interactive Win32
dialog in the original (title/message/file/line/stack, likely Abort/Retry/Ignore-style,
letting a developer break into the debugger) — no dialog UI exists yet in this
reimplementation. Real behavior implemented instead: format and write the message to
`Console.Error` (honoring the `TimedWriteLines`/`WriteLinePrefix` state the other 4
exports configure), always returning "don't break" (`0`/`false`). Marked with a `// TODO`
in the source pointing back to this entry.