Commit Graph

5949 Commits

Author SHA1 Message Date
martin ridgers
dc2a0bfe13 An implementation of LaunchUnrealTrace() for POSIX platforms (currently disabled).
#rb jb
#preflight 6139c15648e495000141820c
#rnx

#ROBOMERGE-AUTHOR: martin.ridgers
#ROBOMERGE-SOURCE: CL 17469929 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17469934 by martin ridgers in ue5-release-engine-test branch]
2021-09-09 05:37:16 -04:00
martin ridgers
9a4471d2d6 Clarified that the shwon return value is in hexadecimal
#rb jb
#preflight 6139c15648e495000141820c
#rnx

#ROBOMERGE-AUTHOR: martin.ridgers
#ROBOMERGE-SOURCE: CL 17469908 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17469914 by martin ridgers in ue5-release-engine-test branch]
2021-09-09 05:36:27 -04:00
martin ridgers
b33b869a1d Swapped to a string-builder and adding missing TEXT() decoration around some literals
#rb jb
#preflight 6139c15648e495000141820c
#rnx

#ROBOMERGE-AUTHOR: martin.ridgers
#ROBOMERGE-SOURCE: CL 17469873 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17469885 by martin ridgers in ue5-release-engine-test branch]
2021-09-09 05:35:19 -04:00
martin ridgers
54d5ea36ea -traceshowstore to pop up the store server in its own console window. Useful for debugging.
#rb jb
#preflight 6139c15648e495000141820c
#rnx

#ROBOMERGE-AUTHOR: martin.ridgers
#ROBOMERGE-SOURCE: CL 17469847 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17469879 by martin ridgers in ue5-release-engine-test branch]
2021-09-09 05:35:03 -04:00
robert millar
f12b8f16a3 Set insights trace memory tag on io requests like we do with LLM tag.
#jira none
#rb none
#[robomerge][bot1] Dev-EngineMerge

#ushell-cherrypick of 17464594 by Robert.Millar

#ROBOMERGE-AUTHOR: robert.millar
#ROBOMERGE-SOURCE: CL 17465963 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17465964 by robert millar in ue5-release-engine-test branch]
2021-09-08 19:53:57 -04:00
charles bloom
4d7bcb6e04 OodleCompressDLL config option needs to be read in CopyBuildToStagingDirectory and passed on the command line
because UnrealPak as a program can't read project config hierarchy

#rb dan.thompson,josh.adams

#ROBOMERGE-AUTHOR: charles.bloom
#ROBOMERGE-SOURCE: CL 17464507 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17464538 by charles bloom in ue5-release-engine-test branch]
2021-09-08 17:53:21 -04:00
geoff evans
b353e75c60 FApp::HasFocus() API to unify checking focus state
This is a defensive change to more uniformly gate conditions of various pathways that want to "nice" CPU consumption when the application is not the foreground application (doesn't "Have Focus"). In benchmarking scenarios we don't want a focus-grabbing application to cause change in behavior while taking performance measurements. This could be in local workstation benchmarking where other foreground apps will naturally come to the front or in data center build where antics that might otherwise be completely unknowable could impact benchmark measurements.

This change was formulated after finding some "nice" CPU conditionals that were instrumented, but not all of them are. This change aims to prevent having to instrument, profile, and mitigate each condition separately.

#rnx
#jira none
#rb francis.hurteau, brandon.schaefer
#preflight 611ec9248ff55400018d5018

#ROBOMERGE-AUTHOR: geoff.evans
#ROBOMERGE-SOURCE: CL 17463345 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17463757 by geoff evans in ue5-release-engine-test branch]
2021-09-08 16:59:42 -04:00
robert millar
eef73ebc8b Insights changes:
Make memory tag specs important so they don't get dropped before a trace connection is made.
Prevent Insights from crashing when opening a trace missing memory scopes or with missing scope specs.
Add a tooltip to symbol resolution noting the environment variable for symbol paths.
Allow Insights to capture full callstacks (including system modules) with the define UE_CALLSTACK_TRACE_FULL_CALLSTACKS or the command line argument -tracefullcallstacks on Windows.

#jira none
#rb ionut.matasaru, martin.ridgers

#ROBOMERGE-AUTHOR: robert.millar
#ROBOMERGE-SOURCE: CL 17461782 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)
#ROBOMERGE[bot1]: Dev-EngineMerge

[CL 17461823 by robert millar in ue5-release-engine-test branch]
2021-09-08 15:02:01 -04:00
aurel cordonnier
b8a2838142 Fix typo in assert text
#rnx
#rb trivial

#ROBOMERGE-AUTHOR: aurel.cordonnier
#ROBOMERGE-SOURCE: CL 17453273 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17453324 by aurel cordonnier in ue5-release-engine-test branch]
2021-09-07 23:34:31 -04:00
zousar shaker
8c80e5a866 Change Compact Binary Package API to represent binary attachments as compressed buffers identified by their raw hash.
Change Compact Binary Package serialization for binary attachments, compressed binary attachments and objects.

#rb devin.doucette

#ROBOMERGE-AUTHOR: zousar.shaker
#ROBOMERGE-SOURCE: CL 17450533 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17450587 by zousar shaker in ue5-release-engine-test branch]
2021-09-07 18:07:00 -04:00
jorgenpt
0ed5ae5123 PR #8376: Fix whitespace issue when updating config sections (Contributed by jorgenpt)
#ROBOMERGE-AUTHOR: josh.adams
#ROBOMERGE-SOURCE: CL 17449932 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17449960 by jorgenpt in ue5-release-engine-test branch]
2021-09-07 17:02:50 -04:00
carlmagnus nordin
2d97f9667e IoStore: Added support for storing compressed chunks in DDC
#rb pj.kack
#rnx

#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 17444267 via CL 17444348 via CL 17444353 via CL 17444359
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v870-17433530)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17444366 by carlmagnus nordin in ue5-release-engine-test branch]
2021-09-07 10:11:43 -04:00
aurel cordonnier
7f517562d5 Merge from Release-Engine-Staging @ 17438845 to Release-Engine-Test
This represents UE4/Main @17430120 and Dev-PerfTest @17437669

[CL 17439044 by aurel cordonnier in ue5-release-engine-test branch]
2021-09-06 12:23:53 -04:00
jason hoerner
110bd98b0a UE5: Optimizations when memory tracing is enabled: 5x perf increase with -llmcsv, 8x perf increase with -trace=memory.
#rb matt.peters johan.berg ionut.matasaru brandon.dawson jason.nadro
#rnx

Over half the perf gain was from reducing critical section locks, with the rest of the gains split between an optimization to call stack traversal (caching return address lookups in a lock free map), and enabling TLS related optimizations for MallocBinned2, which were inadvertently disabled when tracing was enabled (this also saves some locks).  Multiple lock types had to be optimized before measurable perf gains were observed, due to contention on remaining locks increasing as the first few locks were removed.  A total of 6 types of locks were removed or reduced:  AnnounceFNameTag, MallocBinned2, TagDataNameMap, Enum tag data lookup, Windows callstack trace, LLMMap.

* Added hash table utility class TGrowOnlyLockFreeHash which supports lock free reads, writes using a critical section, and no deletion.  This is intended for classes that represent caches of one-time initialized debug data used during memory tracing, which never needs to be freed.
* TGrowOnlyLockFreeHash applied to 4 sets/maps used when memory tracing is enabled, including tag data, announced tag names, call stack functions, and call stack IDs.
* Lock striping added to LLMMap.  This isn't a good candidate for a lock free approach, as it has a heavy amount of modification, and lock free approaches tend to perform worse than lock striping in that case, plus it would be a massive undertaking to make this lock free.
* A future lock to be removed would be the one in "FLowLevelAllocInfo::GetTag", protecting the "FLowLevelMemTracker::TagDatas" array, saving a couple percent more perf.
#preflight 6132a3a6bf137d00019a8c97

#ROBOMERGE-SOURCE: CL 17428762 via CL 17429451
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v865-17346139)

[CL 17429480 by jason hoerner in ue5-release-engine-test branch]
2021-09-03 20:58:47 -04:00
devin doucette
732d680316 StringBuilder: Fixed a static analysis warning, and fixed direct use of the base type
#jira UE-125163
#rb Halfdan.Ingvarsson
#preflight 6132516579ce1700010aae4f

#ROBOMERGE-SOURCE: CL 17425190 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v865-17346139)

[CL 17425211 by devin doucette in ue5-release-engine-test branch]
2021-09-03 14:51:12 -04:00
danny couture
4e6f8f7067 Add Insight tag over each automation tests
#rb trivial

#ROBOMERGE-SOURCE: CL 17418793 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v865-17346139)

[CL 17418828 by danny couture in ue5-release-engine-test branch]
2021-09-03 07:42:37 -04:00
carlmagnus nordin
4d924fbd4a Restore Oodle allocator hooks
#rnx
#jira UE-124678
#rb charles.bloom

#ROBOMERGE-SOURCE: CL 17405626 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v865-17346139)

[CL 17405629 by carlmagnus nordin in ue5-release-engine-test branch]
2021-09-02 13:41:10 -04:00
devin doucette
3d262ca7f0 CompressedBuffer: Removed the FormatName functionality and update its last remaining callers
#rb Paul.Chipchase, Zousar.Shaker
#rnx
#preflight 612feb940380e700016fc8e5

#ROBOMERGE-SOURCE: CL 17403021 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v865-17346139)

[CL 17403035 by devin doucette in ue5-release-engine-test branch]
2021-09-02 10:30:25 -04:00
martin ridgers
9f66dcf36f Be explicit about the "fork" argument when launching the trace store server. Previously it was implicit preventing the addition of useful help text to anyone who ran the store binary explicitly. The process is also launched "detached" so that it can be in the console sub-system without popping up conhost terminals.
#rb johan.berg
#rnx
#preflight 6130519c79ce170001a93958

#ROBOMERGE-SOURCE: CL 17398147 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v865-17346139)

[CL 17398149 by martin ridgers in ue5-release-engine-test branch]
2021-09-02 01:46:57 -04:00
devin doucette
5a57d677bf CompactBinary: Added FIoHashBuilder and switched CompactBinary to use it
#rb Zousar.Shaker
#rnx
#preflight 612fb7306256b100011931e9

#ROBOMERGE-SOURCE: CL 17390911 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v865-17346139)

[CL 17390961 by devin doucette in ue5-release-engine-test branch]
2021-09-01 15:05:03 -04:00
andrew firth
dfa71b7e39 [CSV] - add ExtendedUsedMB to the CSV output as a measure of the amount of extended memory currently in use.
[at]Ben.Woodhouse [at]Dave.Barrett
#rb Ben.Woodhouse
[FYI] nick.bullard
#preflight 612f969f6256b1000112e844

#ROBOMERGE-SOURCE: CL 17388268 via CL 17389243
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v865-17346139)

[CL 17389309 by andrew firth in ue5-release-engine-test branch]
2021-09-01 13:12:36 -04:00
pj kack
e01cc97201 XSXIoDispatcher: Use immediate direct storage hardware decompression for unaligned compression block reads (instead of the cpu decompression fallback).
Fix broken trace counter for IoDispatcherFileBackend/QueuedFilesystemReads.
Use 64K io dispatcher buffers on XSX to match its 64K compression block size.

#FROST-4056
#rb carlmagnus.nordin
#rnx
#preflight 612f86354f89cc00011cddfe

#ROBOMERGE-OWNER: pj.kack
#ROBOMERGE-AUTHOR: pj.kack
#ROBOMERGE-SOURCE: CL 17386851 via CL 17388347
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v865-17346139)

[CL 17388360 by pj kack in ue5-release-engine-test branch]
2021-09-01 12:09:24 -04:00
tiago costa
b009886be2 Local Exposure using Bilaterial Grid based Contrast Reduction.
- Image exposure is locally adjusted by decomposing luminance of the frame into a base layer and a detail layer.
- Base layer contrast can be reduced while preserving detail.
- Support blending with gaussian blurred luminance as proposed in Advances in Real-Time Rendering in Games 2021.
- Controls in PostProcessVolume.
- r.LocalExposure cvar to control whether local exposure is supported.
- Visualization mode in "Show->Visualize->Local Expsoure". Multiple modes controllable using r.LocalExposure.VisualizeDebugMode.
- Bilateral grid generation not yet optimized. Optimization should include increasing depth of grid to match histogram size so that global histogram can be generated from grid.
- Added option to use mirror address mode in PostProcessWeightedSampleSum, since black border adds noticeable vignetting to the blurred luminance.

#preflight 612f65169db3090001ba3eec
#rb Brian.Karis, Guillaume.Abadie, Krzysztof.Narkowicz

#ROBOMERGE-SOURCE: CL 17385317 via CL 17387198
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v865-17346139)

[CL 17387234 by tiago costa in ue5-release-engine-test branch]
2021-09-01 11:22:04 -04:00
carlmagnus nordin
055605dd79 IoStore: Added fallback to normal hashing when failing to generate a perfect hash map within a reasonable amount of time
#jira UE-124678
#rb pj.kack

#ROBOMERGE-SOURCE: CL 17384553 via CL 17384555 via CL 17384557 via CL 17384558
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v865-17346139)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17384561 by carlmagnus nordin in ue5-release-engine-test branch]
2021-09-01 06:14:06 -04:00
andriy tylychko
19daea88b2 removed assert if ticker's delegate is removed multiple times
#jira UE-124560
#rb francis.hurteau vincent.gauthier

#ROBOMERGE-SOURCE: CL 17380188 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v865-17346139)

[CL 17380196 by andriy tylychko in ue5-release-engine-test branch]
2021-08-31 19:46:16 -04:00