Commit Graph

195 Commits

Author SHA1 Message Date
steve robb
0edc9ec830 Some missing TEXT macros added.
#rb trivial
#jira UE-132142
#preflight none

#ROBOMERGE-AUTHOR: steve.robb
#ROBOMERGE-SOURCE: CL 20671620 via CL 20671646 via CL 20671659
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v955-20579017)

[CL 20675152 by steve robb in ue5-main branch]
2022-06-15 15:09:53 -04:00
devin doucette
3e2538a313 DDC: Removed the legacy cache store implementations and limit cache bucket names to 63 characters
The cache bucket name was already limited to 63 characters in practice by Horde Storage. This makes it consistent across every cache store.

#preflight 629e6f0a1c7e8f9a180e192a
#rb Zousar.Shaker

#ROBOMERGE-OWNER: Devin.Doucette
#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-COMMAND: _robomerge[bot4] UE5-MAIN
#ROBOMERGE-SOURCE: CL 20536454 via CL 20536805 via CL 20536993
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v954-20466795)

[CL 20538633 by devin doucette in ue5-main branch]
2022-06-07 11:17:10 -04:00
Devin Doucette
633521c203 DDC: Exposed IsCanceled and LaunchTask through FRequestOwner
#preflight 62958df65370042eb050c1c3
#rb Zousar.Shaker
#rnx

[CL 20436079 by Devin Doucette in ue5-main branch]
2022-05-31 09:01:19 -04:00
mark lintott
da46ade47c Added Cache Speed Stats to Analytics
Added Zen Server Stats to Analytics
#rb zousar.shaker, devin.doucette
#jira none
#preflight 6295d84bd57da28cc8ee9eab

[CL 20434901 by mark lintott in ue5-main branch]
2022-05-31 05:11:05 -04:00
Dan Thompson
7f5df95274 no functionality change, misc comments
#rb none
#preflight 62869724f15d967ff4d1e712

[CL 20286015 by Dan Thompson in ue5-main branch]
2022-05-19 15:44:07 -04:00
Devin Doucette
92a888c1f6 Replaced uses of _ASV with ANSITEXTVIEW
#preflight 6266acf50634d0904cdfe91c
#rb Steve.Robb
#rnx

[CL 19903251 by Devin Doucette in ue5-main branch]
2022-04-25 11:31:36 -04:00
Devin Doucette
ceabbe51e8 DDC: Added more documentation to the cache policy
#preflight 626183407d162829ae811783
#rb Zousar.Shaker
#rnx

[CL 19851224 by Devin Doucette in ue5-main branch]
2022-04-21 12:40:49 -04:00
Devin Doucette
8c27714a84 DDC: Added ConvertLegacyCacheKey to construct new cache keys from legacy cache keys
The function is shared with the legacy cache store to allow callers of the legacy API to migrate to the value API using this conversion function without invalidating existing cache keys.

#preflight 624c9ab68d1db441a913137e
#rb Zousar.Shaker

[CL 19630681 by Devin Doucette in ue5-main branch]
2022-04-05 16:27:17 -04:00
Devin Doucette
436b6a6461 DDC: Added IRequestOwner::LaunchTask
- Added IRequestOwner::LaunchTask to launch a task as a request in the request owner.
- Renamed ExecuteInCacheThreadPool to LaunchTaskInCacheThreadPool.
- Made GCacheThreadPool private to one source file.
- Changed task scheduling throughout DDC to use these functions where appropriate.

#preflight 623b846d8900c14eecd4daa4
#rb Zousar.Shaker

[CL 19485686 by Devin Doucette in ue5-main branch]
2022-03-23 16:56:39 -04:00
Johan Torp
175fbdc61b Optimized BuildCacheKey using FAsciiSet for ~4x speedup
Estimated to save 45s in a 1h50m cook of a larger internal project

#rb devin.doucette
#preflight 6238648a10251d53d55d3d12

[CL 19450563 by Johan Torp in ue5-main branch]
2022-03-21 07:56:39 -04:00
Devin Doucette
d8c983bbe6 DDC: Optimized legacy cache key validation and shortening overhead
- Validate and shorten legacy keys lazily.
- Change to ValueOnly mode for PakFile and Zen.

#preflight 623235998a6037989395baf2
#rb Zousar.Shaker
#rnx

[CL 19410386 by Devin Doucette in ue5-main branch]
2022-03-16 15:28:29 -04:00
devin doucette
70d0aea9a6 DDC: Added support for 64-bit sizes in the legacy cache
Includes an optimization to avoid an extra copy of data loaded from the cache.

#jira UE-103039
#preflight 6205f379b84973a2bb76d942
#lockdown Aurel.Cordonnier
#rb Zousar.Shaker

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18954059 in //UE5/Release-5.0/... via CL 18954686 via CL 18955089
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v917-18934589)

[CL 18955195 by devin doucette in ue5-main branch]
2022-02-11 12:56:47 -05:00
devin doucette
c3443800ec DDC: Converted modular feature names to constant lazy names
#rb Zousar.Shaker
#rnx
#preflight 61f97050fa6554dff3d6fa76

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18812871 in //UE5/Release-5.0/... via CL 18812927 via CL 18822743
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v909-18823584)

[CL 18823987 by devin doucette in ue5-main branch]
2022-02-02 06:26:22 -05:00
devin doucette
3cb1585ced DDC: Changed "Type&& Get() &&" accessors to "Type Get() &&"
#rb Zousar.Shaker
#rnx
#preflight 61f80d4868795b2f45779b4b

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18792049 in //UE5/Release-5.0/... via CL 18792224 via CL 18792941
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v908-18788545)

[CL 18793084 by devin doucette in ue5-main branch]
2022-01-31 13:23:12 -05:00
devin doucette
169318850a DDC: Masked record-only cache policy flags out of the value policy
Fixing backends to respect the masked flags includes a few fixes to handling of ~Query by Put and PutValue.

#rb Matt.Peters
#rnx
#preflight 61f7f7ede55232619f887977

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18790481 in //UE5/Release-5.0/... via CL 18790769 via CL 18791304
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v908-18788545)

[CL 18791515 by devin doucette in ue5-main branch]
2022-01-31 11:52:13 -05:00
devin doucette
517947112b DDC: Reorganized cache policy code into its own files
- FCacheRecordPolicyBuilder now disallows adding value policies with flags that only make sense for records.
- FCacheRecordPolicyBuilder now disallows adding multiple value policies for the same value.
- FCacheRecordPolicyBuilder now skips addition of value policies that match the default policy.

#rb Matt.Peters
#rnx
#preflight 61f471cc6b5aea38e5c64cb6

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18782116 in //UE5/Release-5.0/... via CL 18782123 via CL 18782168
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18782178 by devin doucette in ue5-main branch]
2022-01-28 20:34:31 -05:00
devin doucette
b6bc462f04 DDC: Made ICacheStore private and removed ICache::CancelAll
#rb Zousar.Shaker
#rnx
#preflight 61f2da78801201ab3862d5ac

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18756063 in //UE5/Release-5.0/... via CL 18756070 via CL 18758707
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18760714 by devin doucette in ue5-main branch]
2022-01-27 15:43:52 -05:00
devin doucette
5b730ad9d0 DDC: Added LegacyDebugOptions and LegacyStats to ILegacyCacheStore
#rb Zousar.Shaker
#rnx
#preflight 61f1bcafbe0f0e0a623e8a15

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18745532 in //UE5/Release-5.0/... via CL 18745716 via CL 18746304
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18746526 by devin doucette in ue5-main branch]
2022-01-26 17:32:11 -05:00
devin doucette
b412a532ed DDC: Added MakeResponse(EStatus) to the cache request types
#rb Zousar.Shaker
#rnx
#preflight 61f1b88fc0033b39f89c8d84

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18744557 in //UE5/Release-5.0/... via CL 18744970 via CL 18745683
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18745958 by devin doucette in ue5-main branch]
2022-01-26 17:07:56 -05:00
matt peters
c19bac1163 FCacheKey: Add operator<< to write to an FCbWriter.
FCacheRecord: Add Save function that allows appending to an existing package.
#preflight 61f174dcf8088a3d298c6e48

#ROBOMERGE-AUTHOR: matt.peters
#ROBOMERGE-SOURCE: CL 18736675 in //UE5/Release-5.0/... via CL 18736693 via CL 18737351
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18737679 by matt peters in ue5-main branch]
2022-01-26 12:31:58 -05:00
matt peters
464d8e698f CachePolicy: Add Save/Load to StringBuilder.
CacheRecordPolicy: Add Save/Load to CompactBinary
#rb Devin.Doucette
#rnx
#preflight 61f005391f5c4c6f64941707

#ROBOMERGE-AUTHOR: matt.peters
#ROBOMERGE-SOURCE: CL 18722255 in //UE5/Release-5.0/... via CL 18722342 via CL 18722401
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18722411 by matt peters in ue5-main branch]
2022-01-25 09:50:39 -05:00
devin doucette
cdedb37d66 DDC: Converted the Build API to UTF-8 where appropriate
#jira UE-133382
#rb Zousar.Shaker
#rnx
#preflight 61e798ee3f00a0a23ef78c49

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18655307 in //UE5/Release-5.0/... via CL 18655320 via CL 18655322
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v900-18638592)

[CL 18655349 by devin doucette in ue5-main branch]
2022-01-19 00:27:48 -05:00
devin doucette
cd23a81f6c DDC: Added hashing and comparison to FSharedString and updated natvis
#rb Zousar.Shaker
#rnx
#preflight 61e5ff073778a195de94ce9c

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18636544 in //UE5/Release-5.0/... via CL 18636545 via CL 18636553
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v899-18417669)

[CL 18637608 by devin doucette in ue5-main branch]
2022-01-18 04:50:13 -05:00
devin doucette
6ab080890c DDC: Renamed cache request and response types from Chunk to GetChunk
#rb Zousar.Shaker
#rnx
#preflight 61e5ec410705ece855b8da74

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18635634 in //UE5/Release-5.0/... via CL 18635654 via CL 18635655
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v899-18417669)

[CL 18637582 by devin doucette in ue5-main branch]
2022-01-18 04:47:59 -05:00
devin doucette
23f9d839c7 DDC: Renamed ECachePolicy::PartialOnError to PartialRecord
The name more accurately reflects the purpose of the flag, which is to allow a partial record to be fetched or stored in the absence of data for otherwise-required values.

#rb Zousar.Shaker
#rnx
#preflight 61e5ae87a2616066f68abc35

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18634974 in //UE5/Release-5.0/... via CL 18634979 via CL 18634988
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v899-18417669)

[CL 18637552 by devin doucette in ue5-main branch]
2022-01-18 04:45:13 -05:00