Commit Graph

28 Commits

Author SHA1 Message Date
devin doucette
74dce2d631 CompressedBuffer: Added FCompressedBufferReader::HasSource()
#rb Zousar.Shaker
#rnx
#preflight 61b11fd75238845473be0c30

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18413426 in //UE5/Release-5.0/... via CL 18413465
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v897-18405271)

[CL 18413480 by devin doucette in ue5-release-engine-test branch]
2021-12-08 17:15:25 -05:00
dan thompson
89f184e47f string helper functions for oodle enums.
#rb Fabien.Giesen
#preflight 61ae76ab2b3384289ad5be91

#ROBOMERGE-AUTHOR: dan.thompson
#ROBOMERGE-SOURCE: CL 18387813 in //UE5/Release-5.0/... via CL 18387899
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v896-18170469)

[CL 18387935 by dan thompson in ue5-release-engine-test branch]
2021-12-06 16:26:55 -05:00
devin doucette
27c1393427 CompressedBuffer: Removed partial decompression from FCompressedBuffer now that FCompressedBufferReader is available
Requiring the use of a separate reader type makes it more likely that readers will be reused, and makes it easier to audit reader usage going forward. Reusing readers is desirable to reduce the number of large temporary allocations made during partial decompression of a buffer.

- Added FCompressedBuffer::Save(FArchive&) and renamed FromCompressed(FArchive&) to Load(FArchive&).
- Added FCompressedBufferReaderSourceScope to set a buffer source within a scope.
- Added proper bounds checks to FNoneDecoder.
- Store the header checksum on the decoder context to allow raw blocks to be reused across sources.
- Decode the header on the fly to avoid a temporary header allocation when the header is in contiguous memory.

#rb Zousar.Shaker
#rnx
#preflight 61a98d53800738dbfbc84c73

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18382211 in //UE5/Release-5.0/... via CL 18382310
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v896-18170469)

[CL 18382377 by devin doucette in ue5-release-engine-test branch]
2021-12-06 10:16:05 -05:00
devin doucette
c26bedd2fa CompressedBuffer: Added FCompressedBufferReader to store decoder buffers and allow streaming from seekable archives
#rb Zousar.Shaker
#rnx
#preflight 61a5acc4ee100b68faabeecc

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18325071 in //UE5/Release-5.0/... via CL 18325078
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v895-18170469)

[CL 18325081 by devin doucette in ue5-release-engine-test branch]
2021-11-30 10:34:57 -05:00
aurel cordonnier
a6e741e007 Merge from Release-Engine-Staging @ 17915896 to Release-Engine-Test
This represents UE4/Main @17911760, Release-5.0 @17915875 and Dev-PerfTest @17914035

[CL 17918595 by aurel cordonnier in ue5-release-engine-test branch]
2021-10-25 20:05:28 -04:00
per larsson
59ff132987 CompressedBuffer - decompress at offset
#rb Devin.Doucette, Paul.Chipchase
#rnx

#ROBOMERGE-OWNER: Devin.Doucette
#ROBOMERGE-AUTHOR: per.larsson
#ROBOMERGE-COMMAND: _robomerge UE5-Main
#ROBOMERGE-SOURCE: CL 17797770 in //UE5/Release-5.0/... via CL 17797780
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v883-17842818)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17859847 by per larsson in ue5-release-engine-test branch]
2021-10-19 09:38:07 -04:00
devin doucette
3d262ca7f0 CompressedBuffer: Removed the FormatName functionality and update its last remaining callers
#rb Paul.Chipchase, Zousar.Shaker
#rnx
#preflight 612feb940380e700016fc8e5

#ROBOMERGE-SOURCE: CL 17403021 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v865-17346139)

[CL 17403035 by devin doucette in ue5-release-engine-test branch]
2021-09-02 10:30:25 -04:00
dan thompson
5074a8efce Move replay compression functions only used in fortnite back to fortnite
Clarify interface to Oodle compressed array functions
Add support for >2gb buffers
#preflight 6109e3ff4cd793000191470d
#rb charles.bloom

#ROBOMERGE-SOURCE: CL 17075612 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17075735 by dan thompson in ue5-release-engine-test branch]
2021-08-05 16:59:34 -04:00
charles bloom
1ecb02670e Archive support CompressionFormat other than NAME_Zlib
v2 header tag to indicate format is serialized in data
writes new data, continues to load old data

#rb devin.doucette

#ROBOMERGE-SOURCE: CL 17017219 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v839-17012307)

[CL 17017233 by charles bloom in ue5-release-engine-test branch]
2021-08-02 12:28:00 -04:00
dan thompson
7884ae0b8a Convenience class for compressing/decompressing with Oodle #rb charles.bloom #pf 60f9ddad391d460001d39660
#ROBOMERGE-SOURCE: CL 16932166 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v838-16927207)

[CL 16932178 by dan thompson in ue5-release-engine-test branch]
2021-07-22 18:37:49 -04:00
devin doucette
a149477819 CompressedBuffer: Added Oodle as a compression method
#rb Charles.Bloom, Zousar.Shaker
#rnx
#preflight 60f85a18a6959a000176686e

#ROBOMERGE-SOURCE: CL 16913054 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)

[CL 16913063 by devin doucette in ue5-release-engine-test branch]
2021-07-21 14:55:45 -04:00
charles bloom
df3497d089 OodleDataCompression fixing namespace convention
#rb none

#ROBOMERGE-SOURCE: CL 16912164 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)

[CL 16912187 by charles bloom in ue5-release-engine-test branch]
2021-07-21 14:05:39 -04:00
charles bloom
e0987b11a6 Compression Flags add ForPackaging to differentiate packaging compression usage
add OodleDataCompression CommonUsage to translate Flags

#rb dan.thompson

#ROBOMERGE-SOURCE: CL 16908872 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)

[CL 16908880 by charles bloom in ue5-release-engine-test branch]
2021-07-21 09:52:44 -04:00
charles bloom
0af8bed7db OodleDataCompression.h function param docs
#rb none

#ROBOMERGE-SOURCE: CL 16896523 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)

[CL 16896542 by charles bloom in ue5-release-engine-test branch]
2021-07-20 10:25:49 -04:00
charles bloom
645446f96c OodleDataCompression in Core
All games now use Engine Oodle for runtime decompression
Oodle in Engine can load an older Oodle DLL for encoding packages to keep making the same binary data

#rb dan.thompson,devin.doucette

#ROBOMERGE-SOURCE: CL 16879404 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)

[CL 16879428 by charles bloom in ue5-release-engine-test branch]
2021-07-18 13:31:39 -04:00
Devin Doucette
0250ac38d2 Added static Null members to FCompressedBuffer and FCompositeBuffer
#rb Tim.Smith
#rnx

[CL 16405471 by Devin Doucette in ue5-main branch]
2021-05-20 12:37:18 -04:00
Devin Doucette
942b5601db CompressedBuffer: Added a serialization operator for FArchive
#rb Zousar.Shaker
#rnx

[CL 16209561 by Devin Doucette in ue5-main branch]
2021-05-05 13:12:17 -04:00
Devin Doucette
bd8b0cce81 CompressedBuffer: Added GetFormatName()
#rb Tim.Smith
#rnx
#preflight 607718121fa9990001c43176

[CL 16007986 by Devin Doucette in ue5-main branch]
2021-04-14 13:14:20 -04:00
Devin Doucette
150cebf43f CompressedBuffer: Added the raw data hash to the header
#rb Zousar.Shaker
#rnx

[CL 15983819 by Devin Doucette in ue5-main branch]
2021-04-12 17:22:20 -04:00
Devin Doucette
5d93c9c31e CompressedBuffer: Added FHeader::TotalCompressedSize and FCompressedBuffer::FromCompressed(FArchive&)
#rb Tim.Smith
#rnx

[CL 15946194 by Devin Doucette in ue5-main branch]
2021-04-07 17:13:31 -04:00
Devin Doucette
ab4a43d5db CompressedBuffer: Implemented a stable format for compressed buffers with support for no compression and LZ4
#rb Tim.Smith

[CL 15895927 by Devin Doucette in ue5-main branch]
2021-04-01 16:31:55 -04:00
Devin Doucette
7e2e44ebf4 DDC: Refactored the Cache API to prepare for the Build API
- Removed the "Cache" prefix from ECachePriority, FCacheRequest, ICacheRequest, FCachePayload, FCachePayloadId to allow reuse in the Build API.
- Added FCompressedBuffer as the starting point for the API for creating and accessing compressed buffers.

#rb Zousar.Shaker
#rnx

[CL 15511723 by Devin Doucette in ue5-main branch]
2021-02-24 01:37:45 -04:00
Josh Adams
eafc89607e - Yet more lz4 fixes, for NonUnity, this time
#rb none
#jira UE-88502, UE-88501
#rnx

[CL 11341032 by Josh Adams in Main branch]
2020-02-11 10:59:23 -05:00
Josh Adams
fcd5eb79ae - Made sure that lz4.h works if CoreMinimal.h isn't included in any Unity files, but using the basic types (unsigned int) instead of types like uint32
#rb trivial

[CL 11315177 by Josh Adams in Main branch]
2020-02-10 18:41:35 -05:00
Josh Adams
75ef276574 - Final fix for lz4 IncludeTool warnings
#rb none

[CL 11309470 by Josh Adams in Main branch]
2020-02-10 15:04:04 -05:00