- Fix use-after-free in both WaitForAnyTaskCompleted and UE::Tasks::WaitAny
#rnx
#jira UE-208191
#rb kevin.macaulayvacher
#tests Stress FTaskGraphWaitForAnyTask under ASAN/TSAN for both TASKGRAPH_NEW_FRONTEND on/off
[CL 31870683 by danny couture in ue5-main branch]
[FYI] dmytro.vovk
Original CL Desc
-----------------------------------------------------------------
Fixed default initialization of TLS slots to 0 and invalidity check against 0 as 0 is a valid TLS slot index
#rb Francis.Hurteau, Matt.Peters
[CL 31088717 by sean boocock in ue5-main branch]
Significant refactor of RHI command list management and submission, and RHI breadcrumbs / RenderGraph (RDG) scopes, to allow for parallel translation of most RHI command lists.
See individual changelists in //UE5/Dev-ParallelRendering for details. A summary of the changes is as follows:
This work's primary goal was to allow as many RHI command lists as possible to be parallel translated, to make more efficient use of many-core systems. To achieve this:
- The submission code paths for the immediate and parallel RHI command lists have been merged into a single function: FRHICommandListExecutor::Submit().
- A "dispatch thread" (which is simply a series of chained task graph tasks) is used to decide which command lists are batched together in a single parallel translate job.
- Individual command lists can disable parallel translate, which forces them to be executed on the RHI thread. This happens automatically if an RHI command list performs an operation that is not thread safe (e.g. buffer lock, or low-level resource transition).
One of the primary blockers for parallel translation was the RHI breadcrumb system, and the way RDG builds scopes. This was also refactored to remove these limitations:
- RDG could only push/pop events on the immediate command list, which resulted in parallel and immediate work being interleaved, breaking any opportunity for parallelism.
- Platform RHI implementations of breadcrumbs (e.g. in D3D12 RHI) was not correct across multiple RHI contexts. Push/pop operations aren't necessarily balanced within any one RHI context given that RDG builds "parallel pass sets" containing arbitrary ranges of renderer passes.
A summary of the new RHI breadcrumb system is as follows:
- A tree of breadcrumb nodes is built by the render thread and RDG. Each node contains the node name, and pointers to the parent and next nodes. When fully built, the nodes form a depth-first linked list which is used for traversing the tree for GPU crash debugging.
- The memory for breadcrumb nodes is provided by ref-counted allocator objects. These allocators are pipelined through the RHI, allowing the platform RHI implementation to extend their lifetime for GPU crash debugging purposes.
- RHIPushEvent / RHIPopEvent have been removed, replaced with RHIBeginBreadcrumbGPU / RHIEndBreadcrumbGPU. Platform RHIs implement these functions to perform GPU immediate writes using the unique ID of each node, for tracking GPU progress.
- Format string arguments are captured by-value to remove the cost of string formatting while building the breadcrumb tree. String formatting only occurs when the actual formatted string is required (e.g. during GPU crash breadcrumb stack traversal, or when calling platform GPU profiling APIs).
RenderGraph scopes have been simplified:
- The separate scope trees / arrays of ops have been combined. There is now a single tree of RDG scopes containing all types.
- Each RDG pass holds a pointer to the scope it was created under.
- BeginCPU / EndCPU is called on each RDG scope as the various RDG threads enter / exit them. This allows us to mark-up each worker thread with the relevant Unreal Insights scopes.
Other changes include:
- Fixes for bugs uncovered when parallel translate was enabled.
- Adjusted platform affinities necessary due to the new layout of thread tasks in the renderer.
- Refactored RHI draw call stats to better fit the new pipeline design.
#rb jeannoe.morissette, zach.bethel
#jira UE-139543
[CL 30973133 by Luke Thatcher in ue5-main branch]
- Fix new frontend to use ConcurrentLinearAllocator for normal tasks like the old frontend.
- Store subsequents and prerequisites in a more cache-friendly way to significantly improve performance.
#jira UE-117550
#rb Francis.Hurteau
[CL 30478814 by danny couture in ue5-main branch]
- Fix DoNotRunInsideBusyWait tasks that ended up being picked up during busy waiting anyway.
- Change internal API of ExecuteTask to make it evident it can actually return a new task.
- Make sure that busy wait will dequeue tasks from local/global queue in priority order.
- Remove unused local queue functions.
#jira UE-199870
#rb JeanFrancois.Dube
[CL 29908002 by danny couture in ue5-main branch]
- Fix deadlock and indeterminism caused by not always signaling a thread after launching a task
- Fix deadlock that happened when trying to launch blocking tasks on all workers
- Use a PreWait/CancelWait/CommitWait semantic to close the gap where a drowsing thread could pick a task without the signaling thread knowing about it.
- Deprecate TryLaunchAffinity workaround now that the deadlocks above have been fixed
- Get rid of task stealing throttling as stealing to reduce latency as it is only involved when searching work before going to sleep
- Less spurious wake-ups as a worker will only wake-up another one when picking up a task if CancelWait has consumed a signal
- Workers will now honor task priorities between local and global queues
- Add blocked workers benchmark test
#jira UE-199959
#rb JeanFrancois.Dube
#tests TSAN / UnitTests / StressTests / ReplayRuns
[CL 29744016 by danny couture in ue5-main branch]
- Add UE::FTwoBitMutex which takes a pointer to an external-state of which it only uses its two LSBs.
- Adds tests for FMutex and FExternalMutex
#rb devin.doucette
#rb yiliang.siew
[CL 29024883 by jared cotton in ue5-main branch]
FCpuProfilerTrace::OutputEventType() and FCpuProfilerTrace::OutputBeginDynamicEvent() needs to be called only if the CpuChannel is enabled, otherwise the "spec" trace event is not emitted, so it will result in "<unknown>" cpu timers. The issue can be easily reproduced with -trace=tasks (i.e. without enabling cpu channel).
#jira UE-196134
#rb Andriy.Tylychko
[CL 28185939 by ionut matasaru in ue5-main branch]
This is a rather "hacky" fix. The old task implementation (TaskGraph API) is on a life support so there's no point of spending too much time for this
#preflight 644be99f1c2846595c45a4f2
#rb danny.couture
[CL 25235003 by Andriy Tylychko in ue5-main branch]
