Commit Graph

473 Commits

Author SHA1 Message Date
devin doucette
b46e70ea4e DDC: Rename CompleteParams to Response and use FSharedString with no alias
#rb Zousar.Shaker
#rnx
#preflight 61ddf9e0f10add4574e6cf59

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18577467 in //UE5/Release-5.0/... via CL 18577471
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18577481 by devin doucette in ue5-release-engine-test branch]
2022-01-11 17:09:20 -05:00
devin doucette
0f3498e71c DDC: Moved the pak file backends out of their header
This hides implementation details and reduces the number of files that need to change when refactoring backends.

#rb Zousar.Shaker
#rnx
#preflight 61ddd0a16925dc1f2c9bd3a6

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18575491 in //UE5/Release-5.0/... via CL 18575502
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18575511 by devin doucette in ue5-release-engine-test branch]
2022-01-11 14:41:53 -05:00
devin doucette
ed0c7a9b68 DDC: Replaced BuildValue(ValueKey) with Build(Key)
The policy allows filtering the result to an arbitrary set of values, which makes the new function both more powerful and better aligned with the rest of the API.

#jira UE-133389
#rb Zousar.Shaker
#rnx
#preflight 61ddb5fa4533f92a4e2c48d0

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18574442 in //UE5/Release-5.0/... via CL 18574451
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18574456 by devin doucette in ue5-release-engine-test branch]
2022-01-11 13:18:16 -05:00
devin doucette
23ddaa8bab DDC: Moved cache backends out of headers
This hides implementation details and reduces the number of files that need to change when refactoring backends.

Backends moved in this change: AsyncPut, Http, Memory, S3, Zen.

#rb Zousar.Shaker
#rnx
#preflight 61dd0d758d72a407aab89074

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18573340 in //UE5/Release-5.0/... via CL 18573347
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18573359 by devin doucette in ue5-release-engine-test branch]
2022-01-11 11:57:38 -05:00
zousar shaker
87fc31409c Switch to using HEAD when doing existance checks against the structured cache in Jupiter. Was not possible before due to a bug in Jupiter, but that hs now been fixed.
#rb devin.doucette
#preflight 61dda4790134bb26f3bbd33b

#ROBOMERGE-AUTHOR: zousar.shaker
#ROBOMERGE-SOURCE: CL 18572483 in //UE5/Release-5.0/... via CL 18572521
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18572545 by zousar shaker in ue5-release-engine-test branch]
2022-01-11 10:53:46 -05:00
devin doucette
32844891bd DDC: Do not pass null completion callbacks from ICache into ICacheStore
#rb Zousar.Shaker
#rnx
#preflight 61dc80c4892dee612b348b71

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18565644 in //UE5/Release-5.0/... via CL 18565663
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18565672 by devin doucette in ue5-release-engine-test branch]
2022-01-10 16:54:04 -05:00
zousar shaker
831400efc2 Add support for structured cache operations through the Jupiter DDC backend.
#rb devin.doucette
#preflight 61dc76dc2d5fd64ff7d4d4e8

#ROBOMERGE-AUTHOR: zousar.shaker
#ROBOMERGE-SOURCE: CL 18562740 in //UE5/Release-5.0/... via CL 18562761
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18562791 by zousar shaker in ue5-release-engine-test branch]
2022-01-10 13:43:40 -05:00
devin doucette
5c5ec11c61 DDC: Added a value API to ICacheStore
The new API is only implemented for the file system cache store for now. Support for the other cache stores will follow.

#rb Zousar.Shaker
#rnx
#preflight 61d93dd9411c1403a3ee8758

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18561274 in //UE5/Release-5.0/... via CL 18561311
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18561332 by devin doucette in ue5-release-engine-test branch]
2022-01-10 12:13:18 -05:00
devin doucette
bf8230db59 DDC: Fixes to the cache record and cache value refactor
#rb Zousar.Shaker
#rnx
#preflight 61d93c241a3fd09dcbefcfb8

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18560428 in //UE5/Release-5.0/... via CL 18560448
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18560457 by devin doucette in ue5-release-engine-test branch]
2022-01-10 11:20:18 -05:00
devin doucette
b4e6fce55b DDC: Changed FCacheRecord to be a collection of values
The structure with a value and attachments is an artifact of an early design where the record *was* a compact binary package. That design changed months ago and it now simplifies the cache record significantly to make it a collection of values without specific "value" and "attachment" types.

#rb Zousar.Shaker
#rnx
#preflight 61d8b94ed17842e547def37a

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18553971 in //UE5/Release-5.0/... via CL 18553976
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18553977 by devin doucette in ue5-release-engine-test branch]
2022-01-07 23:22:29 -05:00
devin doucette
35393bbb2b DDC: Split FPayload into separate FValue and FValueId types
A payload was conceptually a value with an ID. That has been formalized by removing the ID from the payload and having separate FValue and FValueId types. This separation cleans up the API in a few areas, and provides a more natural path to providing a basic key/value API.

#rb Zousar.Shaker
#rnx
#preflight 61d704c04c252480ca284d61

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18531844 in //UE5/Release-5.0/... via CL 18531856
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18531864 by devin doucette in ue5-release-engine-test branch]
2022-01-06 11:05:57 -05:00
devin doucette
48f68493ef DDC: Removed the Put/Get compatibility functions from ICache
#rb Zousar.Shaker
#rnx
#preflight 61d49a971a3fd09dcbc87978

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18511490 in //UE5/Release-5.0/... via CL 18511542
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18511622 by devin doucette in ue5-release-engine-test branch]
2022-01-04 14:38:48 -05:00
devin doucette
f0dd93d7cf DDC: Changed FCacheRecord::Load to take the record as an object instead of a view
#rb Zousar.Shaker
#rnx
#preflight 61d489feec35d1b940d21bbf

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18510417 in //UE5/Release-5.0/... via CL 18510464
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18510496 by devin doucette in ue5-release-engine-test branch]
2022-01-04 13:28:36 -05:00
devin doucette
f8fbec038e DDC: Added FSharedString to minimize copies of cache request names
#rb Zousar.Shaker
#rnx
#preflight 61d4831f932a02483cbba5c1

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18510411 in //UE5/Release-5.0/... via CL 18510430
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18510447 by devin doucette in ue5-release-engine-test branch]
2022-01-04 13:25:54 -05:00
devin doucette
8e3ba57e91 DDC: Moved the ICacheStore Context params to be a Name member of the request types and return them in the completion params
#rb Zousar.Shaker
#rnx
#preflight 61d3729eaf44b43628f4bdc9

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18507435 in //UE5/Release-5.0/... via CL 18507440
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18507451 by devin doucette in ue5-release-engine-test branch]
2022-01-04 09:26:33 -05:00
devin doucette
0f14d13c88 DDC: Fixed the logged cache summary stats
Stats from the root node must be a superset of stats from the rest of the nodes. The key length limiter is always the root node at present, and has been updated to include stats for the structured cache.

#rb Zousar.Shaker
#rnx
#preflight 61d318cc932a02483cb29797

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18501391 in //UE5/Release-5.0/... via CL 18501397
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18501400 by devin doucette in ue5-release-engine-test branch]
2022-01-03 11:24:10 -05:00
devin doucette
ffe8432dc4 DDC: Fixed static analysis warnings about null checks
#rb trivial
#rnx
#preflight 61d30e4d1f62d3ad4d46cb23

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18500919 in //UE5/Release-5.0/... via CL 18500934
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18500948 by devin doucette in ue5-release-engine-test branch]
2022-01-03 10:04:48 -05:00
devin doucette
c6113ec521 DDC: Fixed file system cache store put stats
#rb Zousar.Shaker
#rnx
#preflight 61bbb694c32d8309558303b9

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18480250 in //UE5/Release-5.0/... via CL 18481558
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18481828 by devin doucette in ue5-release-engine-test branch]
2021-12-16 19:57:53 -05:00
devin doucette
54235555db DDC: Moved FDerivedDataBackendInterface from Public to Private
The interface was in the public folder before 5.0, but was not usable from there in a meaningful way.

#rb Zousar.Shaker
#rnx
#preflight 61bbac388a62de8385c132c7

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18479761 in //UE5/Release-5.0/... via CL 18481541
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18481817 by devin doucette in ue5-release-engine-test branch]
2021-12-16 19:56:56 -05:00
devin doucette
14bcbbe4c6 DDC: Updated the stat dialog layout and split Zen to include its local stats and upstream stats
#rb Zousar.Shaker
#rnx
#preflight 61bba91ac32d830955820a47

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18479360 in //UE5/Release-5.0/... via CL 18481538
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18481812 by devin doucette in ue5-release-engine-test branch]
2021-12-16 19:56:35 -05:00
devin doucette
68dcdf74a1 DDC: Folded the corruption wrapper into the S3 cache
#rb Zousar.Shaker
#rnx
#preflight 61badf093961edbe76d77bf1

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18475378 in //UE5/Release-5.0/... via CL 18481136
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18481156 by devin doucette in ue5-release-engine-test branch]
2021-12-16 19:04:45 -05:00
devin doucette
73b904baed DDC: Extended the S3 cache with the ability to do local generation and decompression of bundles
#rb Zousar.Shaker
#preflight 61ba653d055f301345a1457a

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18471703 in //UE5/Release-5.0/... via CL 18471718
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18471721 by devin doucette in ue5-release-engine-test branch]
2021-12-15 17:23:24 -05:00
devin doucette
f7ff6e2de1 DDC: Fixed the file access log to write paths relative to the cache directory
#rb Bryan.Johnson
#rnx
#preflight 61ba63d7055f301345a0f56b

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18471469 in //UE5/Release-5.0/... via CL 18471470
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18471478 by devin doucette in ue5-release-engine-test branch]
2021-12-15 17:03:40 -05:00
devin doucette
844d30fa29 DDC: Fold the corruption wrapper into the file system cache store
#rb Zousar.Shaker
#rnx
#preflight 61b98c448f57c5c0c234fb17

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18466778 in //UE5/Release-5.0/... via CL 18466797
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18466801 by devin doucette in ue5-release-engine-test branch]
2021-12-15 09:30:12 -05:00
charles bloom
d6c8d22b81 Fix DDC should not sanitize $ characters because it makes $ characters
SanitizeCacheKey and ValidateCacheKey should check the same condition

#rb devin.doucette, danny.couture

#ROBOMERGE-AUTHOR: charles.bloom
#ROBOMERGE-SOURCE: CL 18455963 in //UE5/Release-5.0/... via CL 18455975
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v898-18417669)

[CL 18455987 by charles bloom in ue5-release-engine-test branch]
2021-12-14 12:35:14 -05:00