Commit Graph

660 Commits

Author SHA1 Message Date
Devin Doucette
7962247e4e DDC: Added a DeleteOnly mode to FileSystemCacheStore
This allows a cache to be kept in the hierarchy solely to ensure that it is cleaned up as its contents expire. Maintenance has been improved to delete more empty folders and to count every scanned folder.

#preflight 6363370154471d10be395619
#rb Zousar.Shaker

[CL 22953493 by Devin Doucette in ue5-main branch]
2022-11-03 11:01:04 -04:00
mark lintott
5ae432fc55 Addeed specific Unreal Cloud DDC analytics for core.cooking and core.loading events ( Domain, Effective Domain, Cache Hit/MIss for Cache Puts & Gets, FailedOIDC login attempts )
#rb joakim.lindqvist
#jira none
#preflight 6363c123d308b70e66f4cf33

[CL 22951911 by mark lintott in ue5-main branch]
2022-11-03 09:39:24 -04:00
Devin Doucette
7945169d6c DDC: Fixed the async cache replay to be tracked as async work by the cache
#preflight 6362d7f69160784fc32ea869
#rb Zousar.Shaker
#rnx

[CL 22926967 by Devin Doucette in ue5-main branch]
2022-11-02 17:13:24 -04:00
Devin Doucette
f1d54c3a16 DDC: Sorted and removed duplicate includes in DerivedDataCache.cpp
#preflight 63627f0254471d10bef32ed7
#rb none
#rnx

[CL 22918352 by Devin Doucette in ue5-main branch]
2022-11-02 11:58:13 -04:00
mark lintott
490070be3c Decoupled Zen, DerivedDataCache and Virtualization dependency from Studio Analytics. Systems can now register callbacks to append or modify Event payloads to analytics events by name via the FAnalytics interface.
#rb Nick.Darnell
#fyi Paul.Chipchase,Zousar.Shaker
#preflight 636248d054471d10bedc3749

[CL 22909100 by mark lintott in ue5-main branch]
2022-11-02 06:56:10 -04:00
devin doucette
271b9555dc Added Unreal Cloud DDC to the base cache graphs
It is disabled by default outside of Epic, and can be enabled by configuring the Default key in the StorageServers section of the Engine config.

#rb Zousar.Shaker

[CL 22836839 by devin doucette in ue5-main branch]
2022-10-28 16:07:04 -04:00
zousar shaker
11736e9e50 If multiple active FileSystemCacheStores in DDC are configured to have the same path, ignore all but the first one to become active.
#rb devin.doucette

[CL 22660595 by zousar shaker in ue5-main branch]
2022-10-20 12:54:34 -04:00
Zousar Shaker
fee1c826d6 Update references to "Horde Storage" to "Unreal Cloud DDC" or appropriate variations.
Specifically:
HordeStorageServers-> StorageServers
HordeStorage  -> UnrealCloudDDC
Horde Storage -> Unreal Cloud DDC

#rb matt.peters
#fyi devin.doucette
#fyi joakim.lindqvist
#jira UE-167262
#preflight 6349da9bad0f7e2f206125e3

[CL 22538004 by Zousar Shaker in ue5-main branch]
2022-10-14 18:39:39 -04:00
dan engelbrecht
b0b3345d11 Add "Accept" field in Zen RPC request to enable new style RPC responses.
See https://github.com/EpicGames/zen/pull/180 for Zen side change.
#rb stefan.boberg zousar.shaker
#preflight 6347c2568a0a7b2adcbe4d71

[CL 22505182 by dan engelbrecht in ue5-main branch]
2022-10-13 11:17:53 -04:00
Devin Doucette
1db832860f DDC: Fixed the default cache verify mode to verify every access
#preflight 63458481e76c1171e7f453a9
#rb Zousar.Shaker
#rnx

[CL 22459545 by Devin Doucette in ue5-main branch]
2022-10-11 12:10:29 -04:00
Devin Doucette
e0a0961a5d DDC: Used InheritedContext to implement the reverted inheritance of memory tags
#preflight 633c89b70e66c104f1cdc505
#rb Andriy.Tylychko, Zousar.Shaker
#rnx

[CL 22353683 by Devin Doucette in ue5-main branch]
2022-10-05 09:23:53 -04:00
Andriy Tylychko
ab5a1abe8d InheritedContext
grouped inherited LLM tag, mem tag and trace metadata into FInheritedContext to simplify their usage, avoid forgetting using all of them together, and to hide the ugly conditional compilation. Added FInheritedContext to higher level tasks instead of the low-level tasks to allow batch task launch to capture the inherited context once per batch

#rb ionut.matasaru
#preflight 633c47792f4bf32d31687192

[CL 22331272 by Andriy Tylychko in ue5-main branch]
2022-10-04 11:22:08 -04:00
Devin Doucette
78b4dfd59b DDC: Fixed untagged allocations by capturing tags in tasks
#jira UE-154013
#preflight 6335f0ffc37844870abec3f3
#rb Matt.Peters

[CL 22254430 by Devin Doucette in ue5-main branch]
2022-09-29 16:11:35 -04:00
dan engelbrecht
0f1143130a Use Http::TryLoadCbPackage as first option to parse packages from Zen.
This will allow UE to talk to both old and new zen servers as we move to use the new package wire format in Zen responses.
#rb devin.doucette
#preflight 6334bf81691c0168b71bc0a9

[CL 22233160 by dan engelbrecht in ue5-main branch]
2022-09-28 18:22:23 -04:00
dan engelbrecht
c58d25f1f4 Make ZenCacheStore::FGetChunksOp asynchronous using EnqueueAsyncRpc
#preflight 63341e5a10030508062960fa
#rb devin.doucette zousar.shaker

[CL 22230515 by dan engelbrecht in ue5-main branch]
2022-09-28 16:39:24 -04:00
Matt Peters
a46d4a439a CacheStoreTest - automated test spurious failure fix: Record whether the first local put and remote put succeed, and use that to decide bExpectHit for each key that get what we put to local/remote.
#jira UE-157577
#rb Devin.Doucette
#rnx
#preflight 63336ff37b582f58abd78581

[CL 22211844 by Matt Peters in ue5-main branch]
2022-09-27 18:10:32 -04:00
Devin Doucette
6c0a4f9cc8 DDC: Added a check for non-zero size in the legacy cache
#jira UE-160573
#preflight 632dd4517b582f58abb80f63
#rb Zousar.Shaker
#rnx

[CL 22157364 by Devin Doucette in ue5-main branch]
2022-09-23 13:12:14 -04:00
Devin Doucette
c9aa2452df DDC: Allow both OAuthProvider and OAuthProviderIdentifier to be used in the same configuration for HttpCacheStore
- At least one of OAuthProvider, OAuthProviderIdentifier, or OAuthAccessTokenEnvOverride must be set.
- If only OAuthAccessTokenEnvOverride is set, then the environment variable must be set too.
- If only OAuthProvider is set, then OAuthSecret must be set too.
- If OAuthProvider is set, then OAuthClientId must be set.

#preflight 632d2eab10030508069b36c8
#rb Zousar.Shaker

[CL 22155720 by Devin Doucette in ue5-main branch]
2022-09-23 11:32:36 -04:00
Devin Doucette
7f98c7fe6c DDC: Added functions to convert between derived data priority and queued work priority
#preflight 632b42d4b4515b7e226ffae8
#rb Zousar.Shaker
#rnx

[CL 22118431 by Devin Doucette in ue5-main branch]
2022-09-21 14:25:32 -04:00
devin doucette
6c0954fb0d DDC: Added to the summary when file system cache store maintenance completes
#rb Zousar.Shaker
#rnx

[CL 22105271 by devin doucette in ue5-main branch]
2022-09-20 19:32:44 -04:00
henrik karlsson
6739fe3d60 [Engine]
* Added UE_SCOPED_ENGINE_ACTIVITY in a couple places that takes noticable amount of time when starting monolith editor

#preflight skip
#rb self

[CL 22030271 by henrik karlsson in ue5-main branch]
2022-09-15 11:47:18 -04:00
devin doucette
1239a75353 DDC: Removed obsolete unstructured namespace parameter from the clients for Horde Storage and Zen
#rb Zousar.Shaker

[CL 22018759 by devin doucette in ue5-main branch]
2022-09-14 18:25:46 -04:00
bryan sefcik
892629fe83 Fixed some paths that got updated to include the full "ThirdParty" path.
#jira
#preflight 6320877bbc40358fa2995069

[CL 21997782 by bryan sefcik in ue5-main branch]
2022-09-13 21:48:33 -04:00
devin doucette
abd22903b7 Fixed forwarding of -DDC=<Graph> to avoid initializing DDC and only forward non-default graph names
#preflight 631f926c1e6887c4a6d4c30d
#rb Zousar.Shaker

[CL 21984731 by devin doucette in ue5-main branch]
2022-09-13 12:12:58 -04:00
bryan sefcik
a3dddc6630 Pass 1 on Developer include fixes:
Removed redundant private include paths from build.cs files.
Fixed include paths to be relative to the private or public folders.
Hid or removed includes that reached into other private module folders.
Updated PublicInclude paths when necessary.

#jira
#preflight 631e281694758d0bf2ea1399

[CL 21960082 by bryan sefcik in ue5-main branch]
2022-09-11 18:32:18 -04:00