Commit Graph

78 Commits

Author SHA1 Message Date
dan thompson
57e66552cc Add missing Oodle define.
#rb fabian.giesen
#jira UE-220836
#rnx

[CL 35410732 by dan thompson in ue5-main branch]
2024-08-08 18:19:30 -04:00
charles bloom
7da13e27d0 CopyBuildToStagingDirectory : bForceUseLatestOodle option to let you disable OodleCompressDLL per-target-platform
#rb fabian.giesen

[CL 35086563 by charles bloom in ue5-main branch]
2024-07-25 13:00:18 -04:00
steve robb
bc28537104 Fixed some Printf specifiers.
#rb brandon.schaefer

[CL 33211124 by steve robb in ue5-main branch]
2024-04-24 15:59:50 -04:00
devin doucette
01cab540f9 CompressedBuffer: Added an assert to prevent saving a null buffer to an archive
Added tests of serialization to and from an archive.

#jira UE-211764
#rb Zousar.Shaker

[CL 33166518 by devin doucette in ue5-main branch]
2024-04-23 09:03:01 -04:00
devin doucette
fffc34d127 Updated FCompressedBufferReader to support a RawHash of zero
This situation arises with the Zen project store. This is a temporary fix to avoid unintentional reuse of decompressed data between from different compressed buffers.

#jira UE-207220
#rb daniele.pieroni
#rnx

[CL 32131510 by devin doucette in ue5-main branch]
2024-03-08 16:22:50 -05:00
charles bloom
60c1e005ab nop comments only about slow hash functions being texture import bottlenecks
[CL 31803982 by charles bloom in ue5-main branch]
2024-02-26 10:31:16 -05:00
devin doucette
ad1fb5d83a CompressedBuffer: Added an optional flag to decompress blocks to an intermediate buffer
#rb Daniele.Pieroni

[CL 31730467 by devin doucette in ue5-main branch]
2024-02-22 14:37:43 -05:00
dan thompson
2b3a8d681c FCompression interface update to 64 bit.
-> Done because UE disables 32 bit down cast - silently could overflow.
-> General use should automatically work.
-> Deprecate some old 32 bit functions not generally used.
-> Deprecate/rename some ancient flags for future removal.
-> Update CompressorData to uint ptr for future use
-> Remove some stuff deprecated in 4.21 (2018)
#rb fabian.giesen
#jira UE-173911

[CL 31688300 by dan thompson in ue5-main branch]
2024-02-21 12:49:23 -05:00
steve robb
66266c6a11 Fixed up DerivedDataCache, DesktopPlatform, ApplicationCore, AssetRegistry, Core, CoreUObject, Projects, Sockets code to use EAllowShrinking instead of bools.
[CL 30676428 by steve robb in ue5-main branch]
2024-01-17 19:51:06 -05:00
matt peters
0e8980ebcc Cooker: DiffOnly in ZenStore: Fix UE::DiffWriter to not break at offsets in the header when the requested offset happens to be within (OriginalHeaderSize - CompressedHeaderSize) of the beginning of the .uexp file.
#rb Zousar.Shaker
#rnx

[CL 27113802 by matt peters in ue5-main branch]
2023-08-15 14:46:47 -04:00
Matt Peters
da707613b5 Cooker IterativeValidate: Fix diff output from DiffWriterArchive that is being logged directly rather than being sent to the MessageCallback.
#rb Zousar.Shaker
#rnx

[CL 26356735 by Matt Peters in ue5-main branch]
2023-06-30 17:32:21 -04:00
fabian giesen
bbb9d0ff10 Change Oodle encoder scratch allocation to accomodate other codecs.
Currently sized for Mermaid VeryFast only, but we use Kraken Fast in certain places,
and bumping it up all the way to Kraken Normal is less than a 2x increase in memory
use (~1.2MB/scratch buffer to ~2.2MB in the editor; not used in game)

#rb dan.thompson

[CL 26138693 by fabian giesen in ue5-main branch]
2023-06-20 19:15:34 -04:00
charles bloom
d8257dfbb6 OodleDataCompression : make CompressParallel output not be machine dependent
#preflight none
#rb none
#rnx

[CL 25727111 by charles bloom in ue5-main branch]
2023-06-01 11:01:36 -04:00
devin doucette
cd050ee69a CompressedBuffer: Added parallel decoding for targets with editor-only data
#jira UE-183996
#preflight 645eaf63cf788a25587d9c0c
#rb Zousar.Shaker, Dan.Thompson

[CL 25474141 by devin doucette in ue5-main branch]
2023-05-15 11:07:39 -04:00
dan thompson
28970d0e6f Adding printf handler to oodle data.
#rb fabian.giesen
#preflight 64553e7a023fe5d3ad9cbf98

[CL 25363351 by dan thompson in ue5-main branch]
2023-05-05 21:38:08 -04:00
keaton stewart
304ee14f1b [Backout] - CL25309015 CIS AutoTest Cook Error
[FYI] Dan.Thompson
Original CL Desc
-----------------------------------------------------------------
Parallelizing FBlockDecoder::TryDecompressTo
#rb charles.bloom
#preflight 645164720e33f2d51d6465dc

[CL 25311278 by keaton stewart in ue5-main branch]
2023-05-02 20:08:37 -04:00
dan thompson
152b670945 Parallelizing FBlockDecoder::TryDecompressTo
#rb charles.bloom
#preflight 645164720e33f2d51d6465dc

[CL 25310355 by dan thompson in ue5-main branch]
2023-05-02 19:02:40 -04:00
charles bloom
42d21f0267 CompressedBuffer : profile scope and note that decoder is serial and should be parallelized
#preflight none
#rb none
#rnx

[CL 25140355 by charles bloom in ue5-main branch]
2023-04-20 19:29:00 -04:00
charles bloom
8057d877d4 OodleDataCompression just comments behavior nop
#rb none
#preflight none

[CL 24604196 by charles bloom in ue5-main branch]
2023-03-11 13:20:53 -05:00
charles bloom
0a198a3d1f OodleDataCompression parallel encode and decode helpers
can be swapped in with existing OodleDataCompression calls
does not add any extra header

#preflight https://horde.devtools.epicgames.com/job/640b9be070639dfc94013653
#rb fabian.giesen

[CL 24601830 by charles bloom in ue5-main branch]
2023-03-10 20:20:03 -05:00
dan thompson
4a6b8c8b00 Removal of the r.Shaders.CompressionFormat CVar, and shader group compression during UnrealPak now inherits the packaging compresison level.
Due to the fact that UnrealPak doesn't load INIs, the cvar was always its default value. This, combined with the fact that shaders don't store what codec they are compressed with means that setting this cvar to anything other than default (oodle) causes a crash in UnrealPak as DecompressShader will try to decompress with Oodle independent of the actual codec. Rather than plumb this information down through ProjectParams, I felt it prudent to just always use Oodle.

Additionally, in order to facilitate faster iteration times for development build farm jobs, the shader compression level used during UnrealPak will inherit from the overall package compression level set via Project Settings (PackageCompressionLevel_*).

This tacitly avoids patch issues as the default shader compression level happens to be the default package compression level, except now it inherits any faster levels for iteration time.

Note that we still use Mermaid as the codec as it's faster for decompressing than Kraken, which we care about for shaders.

#rb Fabian.Giesen
#rb Arciel.Rekman
#preflight 63e2f94c786751d1e0ffb290

[CL 24092953 by dan thompson in ue5-main branch]
2023-02-09 00:35:55 -05:00
martins mozeiko
6a8a5a4a75 Fix static analysis warnings in updated lz4 library code.
#preflight 63e3004c5c0ce8f11beccee5

[CL 24064469 by martins mozeiko in ue5-main branch]
2023-02-07 21:09:07 -05:00
martins mozeiko
a8c418acd4 Upgrade lz4 to 1.9.4 version.
#preflight 63e17d459740e583f439d183
#rb devin.doucette

[CL 24058819 by martins mozeiko in ue5-main branch]
2023-02-07 16:03:20 -05:00
devin doucette
461379fbf0 CompressedBuffer: Added a check to avoid a buffer overflow
#rb Zousar.Shaker
#rnx

[CL 23798852 by devin doucette in ue5-main branch]
2023-01-20 18:05:15 -05:00
charles bloom
fe229cf1e3 OodleDataCompressionUtil : take dest buff size to ensure no overrun
#jira https://jira.it.epicgames.com/browse/FORT-546392
#preflight https://horde.devtools.epicgames.com/job/63c899cfb065224750e38076
#rb none

[CL 23768764 by charles bloom in ue5-main branch]
2023-01-18 20:24:32 -05:00