#rb Per.Larsson
- Provides a good way to see the total size of structured data vs bulkdata, virtualized vs non virtualized, how much data could still be virtualized and how much is currently rejected by the projects virtualization filter options.
- Providing '-DetailedFilterReasons' on the cmdline will cause the commandlet to generate more detailed statistics on the payload filtering.
- The eventual plan for this commandlet is to provide an easy way to track disk usage for a project via our analytics systems. Especially for larger projects that are split up into many game feature plugins.
[CL 35487315 by paul chipchase in ue5-main branch]
#rb Per.Larsson
#rnx
- Payload filter reasons generated during the virtualization process, such as min size and package path reasons will now be correctly recorded.
- The commandlet will now cleanup ALL files from the previous manifest rather than only replacing the files that it needs to write out. This should avoid confusion where files from different manifests were left in the same directory.
- Added new cmdline switches 'PendingOnly', 'FilteredOnly' and 'VirtualizedOnly' in addition to the preexisting 'LocalOnly' allowing finer control over what type of payloads are actually included in the manifest.
-- These options are now containers in a single enum rather than each as their own bool.
[CL 35366756 by paul chipchase in ue5-main branch]
#rb PJ.Kack
#rnx
- If the virtualization system is not enabled for the current project then we should display a message informing the user of this rather than just displaying the normal statistics grid without any backends.
- This code was accidentally removed from the last refactor of this panel in CL 33601457, note that we also removed the lazyinit code path, this is fine as that code path is effectively deprecated anyway.
[CL 34090342 by paul chipchase in ue5-main branch]
#rb PJ.Kack
#jira UE-147384
#rnx
- The previous functor based system was difficult to update as changes to the data being passed to the functor could break existing code, so ::GetAccumualtedPayloadActivityInfo has been replaced with ::GetBackendStatistics, a version that returns a struct for each backend in the system which contains the relevant statistics for that backend. Adding new data to a struct is much easier.
-- The new data added per backend is the storage type so that the caller can tell if the backend was for cached storage or persistent storage.
- To fit into this naming scheme ::GetAccumualtedPayloadActivityInfo has been deprecated in favor of ::GetSystemStatistics which is simply a rename.
- The deprecated methods should continue to work as before.
- Existing engine code had been updated to use the new methods.
- Removed a typo from the copyright boilerplate at the begining of VirtualizationSystem.h
[CL 33626815 by paul chipchase in ue5-main branch]
#rb PJ.Kack
#jira UE-147384
#rnx
- Added boxes around each set of categories (pull, push, cache) to make it easier to see the differences between them.
- Moved the "Push" statistics to be last in order as it is the least interest of the statistics, infact when an external submit tool is being used with the editor it won't display any useful info at all (we should look into removing that set of stats if we can detect such tool use)
- Changed the total time spent in request to an average time per request
- Refactored the code to reduce code duplication.
[CL 33601468 by paul chipchase in ue5-main branch]
#rb Martin.Ridgers
#rnx
- Allow more than one path to be specified at a time. Additional directories can be chained with either ',' or '+'
- Log each root directory searched or that the project itself is being searched.
[CL 31044961 by paul chipchase in ue5-main branch]
#rb none
#jira none
#rnx
#preflight 646492502d446eac96a6fff9
- Each payload in the package is loaded or pulled and checked to make sure that it:
-- Could be accessed.
-- The hash stored in the FCompressedBuffer is the same as the original requested hash.
-- That the payload can be decompressed.
-- That the decompressed payload when hashed has the same hash as the original request.
- This should catch cases where the payload cannot be found or is corrupted in some way.
[CL 25503830 by paul chipchase in ue5-main branch]
#rb trivial
#jira none
#preflight 645cc3cc2965f6ea8e81cfab
- If a project contained too many packages the final .csv output file might be too large to import into some spread sheet systems, in these cases we need to split the output into multiple smaller .csv files.
- The output is now sent to its own subdirectory under saved, ''/payloadmanifest/"
- When done the commandlet will report the file path of each .csv written to disk
- Add the number of payloads discovered to the final output summary.
[CL 25424197 by paul chipchase in ue5-main branch]
#rb none
#jira UE-178523
#preflight 6437a727d03b1c87ddf5ca15
- The commandlets now accept '-PackageDir=' and '-PackageFolder=' commnad line args.
[CL 25021806 by paul chipchase in ue5-main branch]
#rb none
#jira UE-182196
#rnx
#preflight 642d2b1389339023eb66b663
- The new cmdline option will download each payload and make sure that the data is correct in addition to checking that the payload exists.
- Note: There is no way to control which backend returns the payload so the VA graph, or cmdline parameters would need to be changed be the caller to make sure they are checking the backend that they are interested in.
-- This new option is fairly slow and is intended to be used as part of an automated checking system rather than by users themselves, so needing to change the graph and running the commandlet once per backend that needs testing shouldn't be too hard to do.
[CL 24924696 by paul chipchase in ue5-main branch]
#rb none
#rnx
#preflight 641c01e2a6092dfb8dfa1478
- The previous default iteration mode could saturdate the worker threads, which would prevent the Zen DDC backend from running clean up code resulting in very large memory spikes.
- We now launch a limited number of tasks that process a limited number of payloads before completing. Once a task is finsihed we issue another until there is no more work to do.
- This seems to strike a good balance of keeping the network transfer at peak speed but allowing other systems time on the cpu.
- Removed all the experimental iteration modes and going all in on this solution, which makes the code easier to parse.
[CL 24759860 by paul chipchase in ue5-main branch]
#rb trivial
#jira none
#rnx
#preflight 640adf1eaf3fc352929322a9
- When a tester tried '-run=VirtualizationEditor.???' via powershell a spece was added so the editor got '-run=VirtualizationEditor .???' which doesn't parse correctly.
- Change the documentation to suggest using '-run="VirtualizationEditor.???"' so that the original text is preserved.
[CL 24588709 by paul chipchase in ue5-main branch]
