Commit Graph

785 Commits

Author SHA1 Message Date
devin doucette
7bcc3722c1 DDC: Fixed merging and filtering of partial records in the cache hierarchy
#rb Zousar.Shaker

[CL 25310446 by devin doucette in ue5-main branch]
2023-05-02 19:07:42 -04:00
devin doucette
8ff0cd05ca DDC: Added EHttpVersion to HttpClient
#rb Zousar.Shaker
#rnx

[CL 25184555 by devin doucette in ue5-main branch]
2023-04-25 13:54:42 -04:00
mark lintott
e63c334048 EpicStudioAnalytics plugin rewrite
High Level Vision:
https://docs.google.com/document/d/13R4nJ2UNMJZlf6JCjYssFMSY-5CrURazE4tqxW77yBw/edit#heading=h.bpycydduomgi

Details of changes:
Removed "most" references to FStudioAnalytics from Engine code.
Deprecated use of FStudioAnalytics::FireEvent_Loading(), will deprecate FStudioAnalytics in due course.
Added FAnalyticsFlowTracker to the engine code base based on the flow system used in UEFN.
Added explicit DDC Resource and DDC Summary events to track DDC usage by resource type.
Moved common Editor analytics callbacks into EpicStudioAnalytics plugin
Added a variety of delegates to the engine code to so that analytics capture can be defined in the plugin code or within the game code instead of the engine code ( eg. load map, editor initialization, cooking, slow task )
Added JSON serializer for analytics events to provide facility to capture analytics in restriced environments ( eg. UEFN CookWorker ). Though this fetaure is not enabled yet.
This will temporarily break Loading and Cooking analytics for Lyra, ShooterGame, CitySample but not Fortnite or UEFN. The remainder will need to be "fixed up" in UE5/Main once the changes have propogated (stakeholders have been notified)

[FYI] zousar.shaker, devin.doucette, eric.knapik, francis.hurteau
#rb wes.hunt
#preflight 643e45150a5a4b944e603be8

[CL 25083739 by mark lintott in ue5-main branch]
2023-04-18 07:16:36 -04:00
devin doucette
90200ff6d0 DDC: Fixes related to the use of OIDC functionality from DesktopPlatform
- Fixed a crash when HttpCacheStore is used without DesktopPlatform.
- Fixed GetOidcAccessToken and GetOidcTokenStatus failing for targets with no Intermediate directory.
- Added DesktopPlatform to DerivedDataTool to allow it to use HttpCacheStore with OIDC.

#rb Zousar.Shaker
[FYI] Dan.Engelbrecht

[CL 25072527 by devin doucette in ue5-main branch]
2023-04-17 15:23:51 -04:00
zousar shaker
3bd034edf7 Add commandline option to enable the "repair" mode for UE Cloud DDC client so that blobs that are missing get re-populated.
#rb devin.doucette

[CL 25027230 by zousar shaker in ue5-main branch]
2023-04-13 12:15:21 -04:00
Devin Doucette
32c049c3f5 Removed some includes of Function.h in Core
#preflight 6435cf83127980724b0f16b9
#rnx

[CL 25006462 by Devin Doucette in ue5-main branch]
2023-04-12 10:05:50 -04:00
zousar shaker
3d140963d1 When a deactivation latency limit is defined, make the initial speed test executed by filesystem DDC time limited so we don't spend an inordinate amount of time speed testing if we're just going to deactivate it due to high latency.
#rb devin.doucette

[CL 24831864 by zousar shaker in ue5-main branch]
2023-03-29 02:28:27 -04:00
zousar shaker
2a4b4cdaf5 Change Zen DDC cache store to allow:
- Local storage flag to be inferred by whether the service is run locally or set explicitly in config.
- Remote storage flag to be inferred by whether the service is NOT run locally or set explicitly in config.
- ReadOnly storage flag to be set explicitly in config (defaults to true).

#rb devin.doucette

[CL 24822351 by zousar shaker in ue5-main branch]
2023-03-28 13:43:50 -04:00
devin doucette
72f0249b95 Replaced uses of FLazyEvent with FManualResetEvent
#rb none
#rnx

[CL 24810985 by devin doucette in ue5-main branch]
2023-03-27 18:26:53 -04:00
devin doucette
085d8cd57b Changed complex uses of events to use FEventCount
#rb Zousar.Shaker
#rnx

[CL 24810984 by devin doucette in ue5-main branch]
2023-03-27 18:26:44 -04:00
mark lintott
071272eb3f Formatting Changes to Cook Profiling Telemetry for Grafana. Replaced "." with "_". Bumped SchemaVersion of Cook Telemetry to 2.
Format changes will not have any impact on existing telemetry or analytics dashboards
#rb trivial
[FYI] devin.doucette, zousar.shaker
#preflight 64218253973e6096708e6714

[CL 24802042 by mark lintott in ue5-main branch]
2023-03-27 09:49:14 -04:00
devin doucette
9b540c4c54 Reverted use of FManualResetEvent while investigating an issue
#jira UE-181229
#rb none
#rnx

[CL 24781860 by devin doucette in ue5-main branch]
2023-03-24 12:55:38 -04:00
bob tellez
c248e6e469 [Backout] - CLs 24735827, 24746530, 24750104, 24751543, 24752787, 24753773
[FYI] Devin.Doucette
Original CL Desc
-----------------------------------------------------------------
Added PlatformManualResetEvent

This private abstraction is used to signal a single waiting thread to wake.
This first implementation is only specialized for Microsoft platforms. Other platforms will be specialized later.

#preflight 6418bc6a5d3e25354f1a2a5b
#rb Andriy.Tylychko, Zousar.Shaker

[CL 24773885 by bob tellez in ue5-main branch]
2023-03-23 23:28:45 -04:00
zousar shaker
90749b3b1a DDC: Restored the use of ManualResetEvent by HttpClient
#rb Zousar.Shaker
#rnx

[CL 24771744 by zousar shaker in ue5-main branch]
2023-03-23 20:59:56 -04:00
mark lintott
ad70d0d087 Fix up of Event Attribute names to support Grafana in StudioAnalytics, DerivedData, Zen, Virtualization and CookProfiling. Grafana does not like the . separator so these have been replaced by _. I have checked this with the DPE team ( and verified ) this is still compatible with Snowflake/Tableau telemetry and such will not requires any changes to Snowflake or exisitng Tableau dashboards.
[FYI] wes.hunt, eric.knapik, paul.chipchase, devin.doucette
#rb devin.doucette
#preflight 641c6e31c44ce895fc452a84

[CL 24764989 by mark lintott in ue5-main branch]
2023-03-23 13:43:00 -04:00
zousar shaker
886b2fe9cd Change HttpCacheStoreTest to pass access token rather than OAuth details to Zen intermediary instances. Also ensure that we don't overlap keys for Meta and NoMeta test data.
#rb joakim.lindqvist

[CL 24764335 by zousar shaker in ue5-main branch]
2023-03-23 13:05:19 -04:00
devin doucette
f52cece901 Reverted the conversion of FLazyEvent to FManualResetEvent while investigating an issue
#rb none
#rnx

[CL 24763649 by devin doucette in ue5-main branch]
2023-03-23 12:35:09 -04:00
nick edwards
723c5d5999 Fix C++20 compiler error. Add new equality operator for FCacheBucketOwner to avoid ambiguity when reversing arguments.
bool FCacheBucketOwner::operator==(const FCacheBucketOwner& Other) is new
bool FCacheBucketOwner::operator==(const FCacheBucket& Other) is what it would find ambiguous

#preflight 6419f58b25389270b78540dd
#jira none
#rb none

[CL 24759829 by nick edwards in ue5-main branch]
2023-03-23 06:59:24 -04:00
dan engelbrecht
85b808b153 Accept duplicated file handles in Zen RPC responses
#preflight 641c11bf76461c460b8eabb4
#rb stefan.boberg per.larsson

[CL 24759274 by dan engelbrecht in ue5-main branch]
2023-03-23 05:15:22 -04:00
devin doucette
a37ae60c3e Renamed Signal to Notify in new synchronization primitives
#rb Zousar.Shaker
#rnx

[CL 24756519 by devin doucette in ue5-main branch]
2023-03-22 19:47:20 -04:00
devin doucette
18dfce3eca Added FManualResetEvent
This is a one-byte manual-reset event that can replace that use case for the experimental FLazyEvent.

#rb Zousar.Shaker

[CL 24756514 by devin doucette in ue5-main branch]
2023-03-22 19:47:06 -04:00
devin doucette
b8e64bec92 Added EventCount
This is an event that provides a mechanism for avoiding a race between waiting and signaling through the use of a signal count token.

#preflight 641b44cabb1320be41aad497
#rb Zousar.Shaker

[CL 24756487 by devin doucette in ue5-main branch]
2023-03-22 19:46:13 -04:00
devin doucette
592c55ccc8 DDC: Removed deprecated code
#preflight 6410d4770e1f02786b69ac5a
#rb Zousar.Shaker
#rnx

[CL 24654027 by devin doucette in ue5-main branch]
2023-03-15 10:15:30 -04:00
devin doucette
1a688b31e2 DDC: Fixed Clang static analysis warnings
#jira none
#preflight 640f96f3482188d710688621
#rb Zousar.Shaker
#rnx
#lockdown zousar.shaker

[CL 24643936 by devin doucette in ue5-main branch]
2023-03-14 16:48:35 -04:00
devin doucette
672818b928 StringBuilder: Converted WriteToString from a type to a function
This was not possible on C++14 and is possible now due to guaranteed copy elision in C++17.
This will break backward compatibility when WriteToString was used as a type or in a context where it would be copied.

#preflight 640a0460af3fc3529234c216
#rb Matt.Peters

[CL 24583299 by devin doucette in ue5-main branch]
2023-03-09 17:06:55 -05:00