233 Commits

Author SHA1 Message Date
devin doucette
99545b03c0 DDC: Added FValueId::MakeIndexed to more easily support arrays of values
#rb Dan.Elksnitis, Matt.Peters

[CL 35881628 by devin doucette in ue5-main branch]
2024-08-28 17:35:13 -04:00
devin doucette
ef41453691 DDC: Fixed cache policy merging for chunk requests in HttpCacheStore
#jira UE-218960
#rb Matt.Peters

[CL 34715315 by devin doucette in ue5-main branch]
2024-06-27 13:28:09 -04:00
devin doucette
01cb852a23 Deprecated the old names and paths for TSharedString
#rb Zousar.Shaker
#rnx

[CL 34567121 by devin doucette in ue5-main branch]
2024-06-21 12:52:32 -04:00
devin doucette
f5edfaec39 DDC: Added FCacheKeyBuilder because it has been requested several times
#rb Zousar.Shaker

[CL 34437142 by devin doucette in ue5-main branch]
2024-06-17 15:16:01 -04:00
devin doucette
36f5a24ddd Moved TSharedString from DDC into Core
#rb Steve.Robb

[CL 34340152 by devin doucette in ue5-main branch]
2024-06-13 10:40:59 -04:00
charles bloom
5e932ff8b5 TextureBuildFunction : methodology to validate RequiredMemoryEstimate vs. actual mem use using standalone texture build workers and process memory counters
#rb Dan.Thompson

[CL 34109385 by charles bloom in ue5-main branch]
2024-06-04 18:12:55 -04:00
devin doucette
6f9e8a3995 DDC: Added detection and reporting of Cancel() and Wait() deadlocks in FRequestOwner
#rb Zousar.Shaker

[CL 33982474 by devin doucette in ue5-main branch]
2024-05-29 14:46:55 -04:00
devin doucette
6fccd4be26 DDC: Eliminated blocking tasks used by the legacy cache API
#rb Zousar.Shaker

[CL 33794360 by devin doucette in ue5-main branch]
2024-05-21 10:20:40 -04:00
mark lintott
0db277eb21 Changed Derived Data Cache Usage UI to use the same Resource Stats as Studio Telemetry
The total is already served up as an Asset Type so had to and some logic to separate it in the view.
Changed sorting from Size to Count. I felt that sorting by highest count is more useful as is it clearly indicates the less efficient DDC work.
Added Hit Efficiency to Asset Stats and added this as an extra column to the UI
#rb Devin.Doucette

[CL 31009963 by mark lintott in ue5-main branch]
2024-01-30 12:02:31 -05:00
devin doucette
e675fcb5b5 DDC: Exposed metadata more extensively in the build API
#rb Zousar.Shaker

[CL 30691096 by devin doucette in ue5-main branch]
2024-01-18 14:06:50 -05:00
dan engelbrecht
ba66b561de For local Zen DDC connections show the path where data is store in the Cache Statistics tab for DDC instead of local ip address
For Zen DDC connection show the storage size
#jira UE-199929
#rb Devin.Doucette

[CL 30501101 by dan engelbrecht in ue5-main branch]
2024-01-09 06:08:29 -05:00
mark lintott
b99d530ce0 Moved EpicStudioAnalytics plugin to StudioTelemetry which is now public facing.
The intention is to provide a set of "in the box" telemetry hooks to our most common developer workflows for use internal Epic use and licensees
Moved startup of StudioTelemetry plugin much earlier to PostConfigInit stage so that sessions can start much earlier in the workflow
Added WIP Client support to telemetry plugin
Added data driven provider support for FAnlayiticProvidersET interfaces via BaseEngine.ini for common games and sample projects. Settings and URLs for EPic are only avaliable in Resttricted/NotForLicensee config folders for Lyra and Shooter game.
Added support for Horde telemetry and fixed up various problems with duplicate attributes being sent.
Added Core.VirtualAssets telemetry event
Added Core.Zen telemetry event
Added Core.IAS telemetry event
Added check for IOStoreOnDemand IsEnabled to avoid sending empty IAS events
Added FAnalyticsMulticastProvider to forward telemetry events to multiple providers contained within
Removed deprecated Fire_LoadingEvent from StudioAnalytics
[FYI] paul.chipchase
#rb Wes.Hunt

[CL 29908039 by mark lintott in ue5-main branch]
2023-11-23 07:06:10 -05:00
danny couture
5c1c6cd4b4 Fix TSAN warning about potential race when releasing shared string
#rnx
#rb Devin.Doucette

[CL 26735342 by danny couture in ue5-main branch]
2023-08-01 07:49:14 -04:00
devin doucette
3df0bf7f14 DDC: Gather bucket stats from the new ICacheStats
#jira UE-133395
#rb Matt.Peters
#rnx

[CL 26555188 by devin doucette in ue5-main branch]
2023-07-24 16:52:05 -04:00
mark lintott
e63c334048 EpicStudioAnalytics plugin rewrite
High Level Vision:
https://docs.google.com/document/d/13R4nJ2UNMJZlf6JCjYssFMSY-5CrURazE4tqxW77yBw/edit#heading=h.bpycydduomgi

Details of changes:
Removed "most" references to FStudioAnalytics from Engine code.
Deprecated use of FStudioAnalytics::FireEvent_Loading(), will deprecate FStudioAnalytics in due course.
Added FAnalyticsFlowTracker to the engine code base based on the flow system used in UEFN.
Added explicit DDC Resource and DDC Summary events to track DDC usage by resource type.
Moved common Editor analytics callbacks into EpicStudioAnalytics plugin
Added a variety of delegates to the engine code to so that analytics capture can be defined in the plugin code or within the game code instead of the engine code ( eg. load map, editor initialization, cooking, slow task )
Added JSON serializer for analytics events to provide facility to capture analytics in restriced environments ( eg. UEFN CookWorker ). Though this fetaure is not enabled yet.
This will temporarily break Loading and Cooking analytics for Lyra, ShooterGame, CitySample but not Fortnite or UEFN. The remainder will need to be "fixed up" in UE5/Main once the changes have propogated (stakeholders have been notified)

[FYI] zousar.shaker, devin.doucette, eric.knapik, francis.hurteau
#rb wes.hunt
#preflight 643e45150a5a4b944e603be8

[CL 25083739 by mark lintott in ue5-main branch]
2023-04-18 07:16:36 -04:00
Devin Doucette
32c049c3f5 Removed some includes of Function.h in Core
#preflight 6435cf83127980724b0f16b9
#rnx

[CL 25006462 by Devin Doucette in ue5-main branch]
2023-04-12 10:05:50 -04:00
devin doucette
592c55ccc8 DDC: Removed deprecated code
#preflight 6410d4770e1f02786b69ac5a
#rb Zousar.Shaker
#rnx

[CL 24654027 by devin doucette in ue5-main branch]
2023-03-15 10:15:30 -04:00
charles bloom
5e488c3f89 LaunchTaskInThreadPool API change needs to be properly deprecated
add pass-through for old args

#rb devin.doucette
#preflight none

[CL 24263337 by charles bloom in ue5-main branch]
2023-02-16 13:36:05 -05:00
charles bloom
048e8bc0fd AssetCompilingManager track down and log about resources that don't set RequiredMemory
mark places that need fixing with [at]todo RequiredMemory
add DebugName to Task and Work to track down sources of the work items

#preflight https://horde.devtools.epicgames.com/job/63eabe61b96a6b2718336f8f
#rb danny.couture

[CL 24201094 by charles bloom in ue5-main branch]
2023-02-13 22:01:48 -05:00
Devin Doucette
82621298bf DDC: Added -ExportBuildOutput to include outputs in exported builds
#preflight 63d804445c69f453c111cfd0
#rb Zousar.Shaker

[CL 23917590 by Devin Doucette in ue5-main branch]
2023-01-30 17:44:07 -05:00
mark lintott
8bad2c0365 Removed the callback system from core Analytics system. Fixed up analytics gathering in DerivedDataCache, Virtualization and HttpCacheStore
#rb wes.hunt
#preflight 63be95bd68068a8bd6a4438e

[CL 23651011 by mark lintott in ue5-main branch]
2023-01-11 14:28:01 -05:00
Devin Doucette
9b5b232cca Texture: Integrated the derived data build jobs with the asset compiling manager in a two-phase approach
The texture build task queues with the texture compiling manager to block on concurrency limits, or when texture builds are paused. The build scheduler will queue with the asset compiling manager to reserve memory when the build needs to execute locally. With this approach, resuming one texture will complete one texture, even though multiple separate build jobs may be executed.

#preflight 6381820933774509009e6e7d
#rb Danny.Couture, Zousar.Shaker

[CL 23271848 by Devin Doucette in ue5-main branch]
2022-11-25 22:18:09 -05:00
Devin Doucette
110afa128b DDC: Added new build scheduler modular features to queue by type and reserve memory
#preflight 638128f3cc307d6fa5b4ee80
#rb Zousar.Shaker

[CL 23271781 by Devin Doucette in ue5-main branch]
2022-11-25 21:38:59 -05:00
Devin Doucette
e2483014be DDC: Added LaunchTaskInThreadPool as a replacement for the two existing thread pool task requests
#preflight 6380f7c4170bc34a93b9104b
#rb Zousar.Shaker

[CL 23267593 by Devin Doucette in ue5-main branch]
2022-11-25 12:35:29 -05:00
Devin Doucette
165d24ede5 DDC: Made the cache optional for the build system
#preflight 636c045ef56cab38c78f8b47
#rb Tim.Smith
#rnx

[CL 23062223 by Devin Doucette in ue5-main branch]
2022-11-09 15:53:21 -05:00