izzy/UnrealEngineUWP
0
0
Fork 0
mirror of https://github.com/izzy2lost/UnrealEngineUWP.git synced 2026-03-26 18:15:20 -07:00
Code Issues Packages Projects Releases Wiki Activity
Files
5.5
UnrealEngineUWP/Engine/Source/Developer/DerivedDataCache/Private/PakFileCacheStore.h

25 lines
594 B
C
Raw Permalink Normal View History

Updating copyright for Engine Developer. #rnx #rb none #ROBOMERGE-SOURCE: CL 10869240 via CL 10869516 via CL 10869902 #ROBOMERGE-BOT: (v613-10869866) [CL 10870584 by ryan durand in Main branch]
2019-12-26 15:32:37 -05:00
// Copyright Epic Games, Inc. All Rights Reserved.
Updating CIS Counter [CL 2649301 by buildmachine in Main branch]
2015-08-10 08:14:45 -04:00
#pragma once
DDC: Added structured cache support to the pak file cache store #rb Zousar.Shaker #rnx #preflight 61942ee0cc0baff9b08f7659 #ROBOMERGE-AUTHOR: devin.doucette #ROBOMERGE-SOURCE: CL 18218377 in //UE5/Release-5.0/... via CL 18218399 #ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v895-18170469) #ROBOMERGE[STARSHIP]: UE5-Main [CL 18218408 by devin doucette in ue5-release-engine-test branch]
2021-11-16 21:12:12 -05:00
#include "CoreTypes.h"
DDC: Removed the legacy cache store implementations and limit cache bucket names to 63 characters The cache bucket name was already limited to 63 characters in practice by Horde Storage. This makes it consistent across every cache store. #preflight 629e6f0a1c7e8f9a180e192a #rb Zousar.Shaker #ROBOMERGE-OWNER: Devin.Doucette #ROBOMERGE-AUTHOR: devin.doucette #ROBOMERGE-COMMAND: _robomerge[bot4] UE5-MAIN #ROBOMERGE-SOURCE: CL 20536454 via CL 20536805 via CL 20536993 #ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v954-20466795) [CL 20538633 by devin doucette in ue5-main branch]
2022-06-07 11:17:10 -04:00
#include "DerivedDataLegacyCacheStore.h"
DDC: Added structured cache support to the pak file cache store #rb Zousar.Shaker #rnx #preflight 61942ee0cc0baff9b08f7659 #ROBOMERGE-AUTHOR: devin.doucette #ROBOMERGE-SOURCE: CL 18218377 in //UE5/Release-5.0/... via CL 18218399 #ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v895-18170469) #ROBOMERGE[STARSHIP]: UE5-Main [CL 18218408 by devin doucette in ue5-release-engine-test branch]
2021-11-16 21:12:12 -05:00
DDC: Enabled compression of legacy cache data - FileSystem, Http, Pak, S3 use the ValueWithLegacyFallback mode by default, which cause them to fall back to searching for uncompressed data if compressed data is not found. - FileSystem has been fixed to store up to 1 MiB of compressed data inline with the value package rather than separately in content-addressable storage. - Pak has been optimized to have GetChunks only load the required region of the requested value, rather than the whole value. - Pak has been changed to stop storing data inline in the record package, since it will end up in the same file anyway when stored separately. - Pak will upgrade the compressor and compression level when a compressed pak file is requested. Default cache compression uses Oodle Mermaid VeryFast and will upgrade to Oodle Kraken Optimal2. - Zen does not have compression enabled by default, pending deployment of a new version that stores compressed values to Horde Storage. - Added a missing request barrier when saving uncompressed data as compressed. Example reduction in file system cache size when cooking for Windows: - CitySample dropped from 66.5 GiB to 21.8 GiB. - Lyra dropped from 2.54 GiB to 672 MiB. - ShooterGame dropped from 1.21 GiB to 380 MiB. Example reduction in compressed pak file cache size when cooking for Windows: - CitySample dropped from 22.3 GiB to 18.5 GiB. - Lyra dropped from 691 MiB to 543 MiB. - ShooterGame dropped from 387 MiB to 313 MiB. #jira UE-134381 #preflight 620a703f583261b0a658e043, 620a6fb2803d9066e6805310, 620a733117632e948459b6af #lockdown Aurel.Cordonnier #rb Zousar.Shaker #ROBOMERGE-AUTHOR: devin.doucette #ROBOMERGE-SOURCE: CL 18983671 in //UE5/Release-5.0/... via CL 18983890 via CL 18984096 #ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v917-18934589) [CL 18984126 by devin doucette in ue5-main branch]
2022-02-14 14:43:39 -05:00
namespace UE::DerivedData
DDC: Implement ICacheStore on FDerivedDataBackendInterface This allows the new cache to be implemented more efficiently using the old cache backends because functionality like the corruption wrapper and key length limiter can be bypassed and the individual backends can store cache records in whatever way is most efficient for them. The hierarchical backend may request payloads when they are not required due to incomplete tracking of backend state, and GetPayload will never fill other backends due to the inefficiency of the existing backend framework. The filesystem backend does not cache any state in memory, which makes requests for individual payloads less efficient than if it cached a mapping of cache payload key to raw hash after the first request for a cache key. The HTTP, S3, and pak file backends are not implemented for the new interface. The backends do not implement ICacheStore::CancelAll() because the existing backend framework provides WaitForQuiescence to wait for completion of async requests, and the implementation of ICacheStore by those backends works with that mechanism. The non-leaf backends (hierarchical, async put, etc.) do not update stats from the ICacheStore functions. #rb Zousar.Shaker #rnx #preflight 60899f35d324590001b47517 [CL 16148296 by Devin Doucette in ue5-main branch]
2021-04-28 16:22:18 -04:00
{
DDC: Removed the legacy cache store implementations and limit cache bucket names to 63 characters The cache bucket name was already limited to 63 characters in practice by Horde Storage. This makes it consistent across every cache store. #preflight 629e6f0a1c7e8f9a180e192a #rb Zousar.Shaker #ROBOMERGE-OWNER: Devin.Doucette #ROBOMERGE-AUTHOR: devin.doucette #ROBOMERGE-COMMAND: _robomerge[bot4] UE5-MAIN #ROBOMERGE-SOURCE: CL 20536454 via CL 20536805 via CL 20536993 #ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v954-20466795) [CL 20538633 by devin doucette in ue5-main branch]
2022-06-07 11:17:10 -04:00
class IPakFileCacheStore : public ILegacyCacheStore
Updating CIS Counter [CL 2649301 by buildmachine in Main branch]
2015-08-10 08:14:45 -04:00
{
public:
DDC: Moved the pak file backends out of their header This hides implementation details and reduces the number of files that need to change when refactoring backends. #rb Zousar.Shaker #rnx #preflight 61ddd0a16925dc1f2c9bd3a6 #ROBOMERGE-AUTHOR: devin.doucette #ROBOMERGE-SOURCE: CL 18575491 in //UE5/Release-5.0/... via CL 18575502 #ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669) [CL 18575511 by devin doucette in ue5-release-engine-test branch]
2022-01-11 14:41:53 -05:00
virtual void Close() = 0;
DDC: Removed the legacy cache store implementations and limit cache bucket names to 63 characters The cache bucket name was already limited to 63 characters in practice by Horde Storage. This makes it consistent across every cache store. #preflight 629e6f0a1c7e8f9a180e192a #rb Zousar.Shaker #ROBOMERGE-OWNER: Devin.Doucette #ROBOMERGE-AUTHOR: devin.doucette #ROBOMERGE-COMMAND: _robomerge[bot4] UE5-MAIN #ROBOMERGE-SOURCE: CL 20536454 via CL 20536805 via CL 20536993 #ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v954-20466795) [CL 20538633 by devin doucette in ue5-main branch]
2022-06-07 11:17:10 -04:00
virtual bool IsWritable() const = 0;
DDC: Moved the pak file backends out of their header This hides implementation details and reduces the number of files that need to change when refactoring backends. #rb Zousar.Shaker #rnx #preflight 61ddd0a16925dc1f2c9bd3a6 #ROBOMERGE-AUTHOR: devin.doucette #ROBOMERGE-SOURCE: CL 18575491 in //UE5/Release-5.0/... via CL 18575502 #ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669) [CL 18575511 by devin doucette in ue5-release-engine-test branch]
2022-01-11 14:41:53 -05:00
virtual bool SaveCache() = 0;
virtual bool LoadCache(const TCHAR* InFilename) = 0;
DDC: Enabled compression of legacy cache data - FileSystem, Http, Pak, S3 use the ValueWithLegacyFallback mode by default, which cause them to fall back to searching for uncompressed data if compressed data is not found. - FileSystem has been fixed to store up to 1 MiB of compressed data inline with the value package rather than separately in content-addressable storage. - Pak has been optimized to have GetChunks only load the required region of the requested value, rather than the whole value. - Pak has been changed to stop storing data inline in the record package, since it will end up in the same file anyway when stored separately. - Pak will upgrade the compressor and compression level when a compressed pak file is requested. Default cache compression uses Oodle Mermaid VeryFast and will upgrade to Oodle Kraken Optimal2. - Zen does not have compression enabled by default, pending deployment of a new version that stores compressed values to Horde Storage. - Added a missing request barrier when saving uncompressed data as compressed. Example reduction in file system cache size when cooking for Windows: - CitySample dropped from 66.5 GiB to 21.8 GiB. - Lyra dropped from 2.54 GiB to 672 MiB. - ShooterGame dropped from 1.21 GiB to 380 MiB. Example reduction in compressed pak file cache size when cooking for Windows: - CitySample dropped from 22.3 GiB to 18.5 GiB. - Lyra dropped from 691 MiB to 543 MiB. - ShooterGame dropped from 387 MiB to 313 MiB. #jira UE-134381 #preflight 620a703f583261b0a658e043, 620a6fb2803d9066e6805310, 620a733117632e948459b6af #lockdown Aurel.Cordonnier #rb Zousar.Shaker #ROBOMERGE-AUTHOR: devin.doucette #ROBOMERGE-SOURCE: CL 18983671 in //UE5/Release-5.0/... via CL 18983890 via CL 18984096 #ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v917-18934589) [CL 18984126 by devin doucette in ue5-main branch]
2022-02-14 14:43:39 -05:00
virtual void MergeCache(IPakFileCacheStore* OtherPak) = 0;
DDC: Moved the pak file backends out of their header This hides implementation details and reduces the number of files that need to change when refactoring backends. #rb Zousar.Shaker #rnx #preflight 61ddd0a16925dc1f2c9bd3a6 #ROBOMERGE-AUTHOR: devin.doucette #ROBOMERGE-SOURCE: CL 18575491 in //UE5/Release-5.0/... via CL 18575502 #ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669) [CL 18575511 by devin doucette in ue5-release-engine-test branch]
2022-01-11 14:41:53 -05:00
virtual const FString& GetFilename() const = 0;
Updating CIS Counter [CL 2649301 by buildmachine in Main branch]
2015-08-10 08:14:45 -04:00
DDC: Moved the pak file backends out of their header This hides implementation details and reduces the number of files that need to change when refactoring backends. #rb Zousar.Shaker #rnx #preflight 61ddd0a16925dc1f2c9bd3a6 #ROBOMERGE-AUTHOR: devin.doucette #ROBOMERGE-SOURCE: CL 18575491 in //UE5/Release-5.0/... via CL 18575502 #ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669) [CL 18575511 by devin doucette in ue5-release-engine-test branch]
2022-01-11 14:41:53 -05:00
static bool SortAndCopy(const FString& InputFilename, const FString& OutputFilename);
Updating CIS Counter [CL 2649301 by buildmachine in Main branch]
2015-08-10 08:14:45 -04:00
};
DDC: Implement ICacheStore on FDerivedDataBackendInterface This allows the new cache to be implemented more efficiently using the old cache backends because functionality like the corruption wrapper and key length limiter can be bypassed and the individual backends can store cache records in whatever way is most efficient for them. The hierarchical backend may request payloads when they are not required due to incomplete tracking of backend state, and GetPayload will never fill other backends due to the inefficiency of the existing backend framework. The filesystem backend does not cache any state in memory, which makes requests for individual payloads less efficient than if it cached a mapping of cache payload key to raw hash after the first request for a cache key. The HTTP, S3, and pak file backends are not implemented for the new interface. The backends do not implement ICacheStore::CancelAll() because the existing backend framework provides WaitForQuiescence to wait for completion of async requests, and the implementation of ICacheStore by those backends works with that mechanism. The non-leaf backends (hierarchical, async put, etc.) do not update stats from the ICacheStore functions. #rb Zousar.Shaker #rnx #preflight 60899f35d324590001b47517 [CL 16148296 by Devin Doucette in ue5-main branch]
2021-04-28 16:22:18 -04:00
DDC: Enabled compression of legacy cache data - FileSystem, Http, Pak, S3 use the ValueWithLegacyFallback mode by default, which cause them to fall back to searching for uncompressed data if compressed data is not found. - FileSystem has been fixed to store up to 1 MiB of compressed data inline with the value package rather than separately in content-addressable storage. - Pak has been optimized to have GetChunks only load the required region of the requested value, rather than the whole value. - Pak has been changed to stop storing data inline in the record package, since it will end up in the same file anyway when stored separately. - Pak will upgrade the compressor and compression level when a compressed pak file is requested. Default cache compression uses Oodle Mermaid VeryFast and will upgrade to Oodle Kraken Optimal2. - Zen does not have compression enabled by default, pending deployment of a new version that stores compressed values to Horde Storage. - Added a missing request barrier when saving uncompressed data as compressed. Example reduction in file system cache size when cooking for Windows: - CitySample dropped from 66.5 GiB to 21.8 GiB. - Lyra dropped from 2.54 GiB to 672 MiB. - ShooterGame dropped from 1.21 GiB to 380 MiB. Example reduction in compressed pak file cache size when cooking for Windows: - CitySample dropped from 22.3 GiB to 18.5 GiB. - Lyra dropped from 691 MiB to 543 MiB. - ShooterGame dropped from 387 MiB to 313 MiB. #jira UE-134381 #preflight 620a703f583261b0a658e043, 620a6fb2803d9066e6805310, 620a733117632e948459b6af #lockdown Aurel.Cordonnier #rb Zousar.Shaker #ROBOMERGE-AUTHOR: devin.doucette #ROBOMERGE-SOURCE: CL 18983671 in //UE5/Release-5.0/... via CL 18983890 via CL 18984096 #ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v917-18934589) [CL 18984126 by devin doucette in ue5-main branch]
2022-02-14 14:43:39 -05:00
} // UE::DerivedData
Reference in New Issue Copy Permalink