6 Commits

Author SHA1 Message Date
paul chipchase
1bea3bb207 Add a function to the virtualization utilities that will expand environment variables found in a file path.
#rb trivial
#jira UE-175157
#rnx
#preflight 63f8b276ae54ee4ce9dcb620

[CL 24399368 by paul chipchase in ue5-main branch]
2023-02-24 07:58:31 -05:00
paul chipchase
f4645786f9 Add a warning when virtualizing packages if we find any that predate when package trailers were enabled.
#rb Per.Larsson
#jira UE-176605
#preflight 63ea68d9b91ae11c1cbab311

- A common problem that has been reported is that the user has tried to virtualize packages with the virtualization tool but not seen any virtualization occur. When debugged the issue turns out to be that the packages are too old and need to be re-saved.
- If the user submits via the editor then the package will be auto reverted if it has not yet been re-saved  so this problem is not seen in that flow.
- Although this is a problem that will solve itself over time, it would be helpful to the person testing the system if we explicitly log that X number of packages were too old.
- Given how annoying this has proven to people the log message is currently set to warning.
- We don't actually return why a package trailer fails to load from FPackageTrailer::LoadTrailer, as working that out requires additional file reading and most the time we don't really care.
- Instead I added a new utility to the virtualization module so that we can opt into checking the reason.
- At the moment we only report if the package is too old and not for every possible reason.

[CL 24206813 by paul chipchase in ue5-main branch]
2023-02-14 04:12:26 -05:00
paul chipchase
f8c5ad0a71 Payload files now use the extension .upayload rather than .payload.
#rb Mark.Lintott
#jira UE-143823
#preflight 6218ce5c8849a7424f822b4b
#lockdown Mark.Lintott


#ROBOMERGE-AUTHOR: paul.chipchase
#ROBOMERGE-SOURCE: CL 19145824 via CL 19146094 via CL 19146118 via CL 19146141 via CL 19147003
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v921-19075845)

[CL 19156252 by paul chipchase in ue5-main branch]
2022-02-25 16:06:09 -05:00
paul chipchase
05d81ee7f3 EditorBulkData and the virtualization system now use FIoHash directly and FPayloadId is removed
#rb Per.Larsson, Devin.Doucette
#jira UE-133497
#rnx
#preflight 61f835491c5ac5523462810a

- A lot of files have been touched but most of this is changing FPayload::IsValid to !FIoHash::IsZero, the main changes are in EditorBulkData/EditorBulkDataTests
- The only difference between FPayloadId and FIoHash is that the former considered the hash of an invalid or empty buffer to be 'invalid' too where as FIoHash would return a valid hash
-- To keep behaviour the same, we only hash payloads in EditorBulkData using a utility method ::HashBuffer which will not hash empty or invalid payloads and return a default FIoHash instead.
-- Unit tests have been extended to prove that the behaviour has not changed.

#ROBOMERGE-AUTHOR: paul.chipchase
#ROBOMERGE-SOURCE: CL 18806362 in //UE5/Release-5.0/... via CL 18808527 via CL 18821790
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v908-18788545)

[CL 18822154 by paul chipchase in ue5-main branch]
2022-02-02 02:21:24 -05:00
aurel cordonnier
a12d56ff31 Merge from Release-Engine-Staging @ 17791557 to Release-Engine-Test
This represents UE4/Main @17774255, Release-5.0 @17791557 and Dev-PerfTest @17789485

[CL 17794212 by aurel cordonnier in ue5-release-engine-test branch]
2021-10-12 21:21:22 -04:00
paul chipchase
d1060bb972 Change the file path to be three directories deep from the root, with each directory being named 00->ff, we then attempt to place each file in a directory based on a hash of the PayloadId, ideally evenly distributing between all available leaf directories.
- Cleaned up some allocations caused by string manipulation.

*Add a place for shared utility code that could have use in multiple places in the virtualization system.
- First utility is a function to generate a file path based off a FPayloadId.
- The path produced is similar to how we organize files in the DDC but instead of 0->9 we use the range 0->ff in hex.
- To generate the path we take the string form of FPayloadId and then use the first 6 characters to create the directories (each directory using 2 characters) and then the remaining 34 characters becomes the filename.

#rb Stefan.Boberg
#rnx

#ushell-cherrypick of 16167644 by paul.chipchase
#preflight 608f9330f5b27a0001b2d5ab

[CL 16182581 by paul chipchase in ue5-main branch]
2021-05-03 02:47:29 -04:00