Commit Graph

29 Commits

Author SHA1 Message Date
per larsson
00573797c3 I/O - simple file based FIFO cache
#rb none
#preflight 644b6f70401d328ad90fa1c5

[CL 25238719 by per larsson in ue5-main branch]
2023-04-28 15:55:46 -04:00
per larsson
5d85c42089 I/O - parallel HTTP requests
- removed background thread in favor using Tasks
- using fixed set of task pipes to throttle HTTP requests
- updated chunk encoding API to use FIoStatus and FOffsetAndLength

#rb none
#preflight 644a50851150e908d00b36f3

[CL 25214018 by per larsson in ue5-main branch]
2023-04-27 10:06:24 -04:00
per larsson
d57a460945 Added missing export
#rb trivial
#preflight none

[CL 25174981 by per larsson in ue5-main branch]
2023-04-24 22:59:26 -04:00
per larsson
ab30bdc35b I/O - added caching to HTTP I/O backend
-  added simple in memory LRU cache for testing purposes, enabled with -HttpBackendMemoryCache=<size>

#rb none
#preflight 6446408e0171aae0b1e8855e

[CL 25162523 by per larsson in ue5-main branch]
2023-04-24 08:27:25 -04:00
per larsson
0804ba324f I/O - move FIoOffsetAndLength to separate header
#rb Paul.Chipchase
#preflight 64412146a35280ed4f2500ab

[CL 25142898 by per larsson in ue5-main branch]
2023-04-21 04:08:48 -04:00
danny couture
1b51796326 Properly handle query to GPlayInEditorID while packages are being loaded
#rnx
#jira UE-175454
#rb Francis.Hurteau, Ben.Zeigler
#preflight 64399e149566b61450059959

[CL 25066973 by danny couture in ue5-main branch]
2023-04-17 09:17:06 -04:00
per larsson
7b3c2ac4ed I/O - added support for loading ondemand utoc files from CbObject
#rb trivial
#preflight 643ceb978901e5ef98c7b9e3

[CL 25066201 by per larsson in ue5-main branch]
2023-04-17 07:29:44 -04:00
per larsson
7b3a9ed322 I/O - extended ondemand I/O store writer to output compressed and encrypted chunks
- renabled the ondemand PAK rule that was previously removed
- enabled handling more than one ondemand containe
- added ondemand I/O store TOC
- added support for serializing ondemand TOC's to CbObjet

#rb Martin.Ridgers
#preflight 6436cbf80c19befb6c8ee4cc

[CL 25025449 by per larsson in ue5-main branch]
2023-04-13 10:51:22 -04:00
henrik karlsson
c8c3d0b8b2 [Core]
* Made code compatible with modules/header units. This includes following things:
   * Removed static in front of global functions/variables (static indicates it is hidden outside of module/header unit which is not what many places want...)
   * Moved dllexport from type to methods in order to get dllexport of static constexpr fields. This is supposed to work according to microsoft but since it is a good change I don't mind doing it anyway)
   * static constexpr -> static constexpr inline for member variables and methods (this makes them available outside module/headerunit)
   * Added HEADER_UNIT_IGNORE markup to includes that are circular
   * Added HEADER_UNIT_SKIP to files that can't compile by themselves

#preflight 63e2aeba3c44c83044bfcc75
#rb steve.robb

[CL 24066986 by henrik karlsson in ue5-main branch]
2023-02-08 00:25:15 -05:00
martins mozeiko
993681afd5 Fixes non-unity build, previous change requires FStringView type visible in header
[CL 23924046 by martins mozeiko in ue5-main branch]
2023-01-31 01:28:16 -05:00
martins mozeiko
dddf036247 Reduce amount of malloc calls during pak creation
1) Move few containers outside of inner loop to reuse same memory across iterations in AssignPackagesDiskOrder
2) Change FString to FStringView when list of filenames is prepared during FIoStoreWriter::Finalize

#rb fabian.giesen
#preflight 63d826653656ea96dc1fabfe

[CL 23923937 by martins mozeiko in ue5-main branch]
2023-01-31 01:26:03 -05:00
henrik karlsson
0f2ff3a256 Fixed non-unity (with orphaned headers compiled independently) compile errors
#preflight 639278e767018b14b5b68597
#rb none

[CL 23460523 by henrik karlsson in ue5-main branch]
2022-12-08 21:34:12 -05:00
graeme thornton
faf6311ef1 IoStore: Add functionality to sign a container as a separate step
[FYI] carlmagnus.nordin

[CL 23312469 by graeme thornton in ue5-main branch]
2022-11-29 12:35:12 -05:00
carlmagnus nordin
8af3f70844 ZenLoader: Implemented priority support for the loader event queue instead of just relying on io request priorities
#preflight 63847b497b4bd3f057f36e53
#rb pj.kack
#jira UE-162812

[CL 23286836 by carlmagnus nordin in ue5-main branch]
2022-11-28 05:57:01 -05:00
bryan sefcik
746b7df074 Same as CL 20991367:
Updated Core code to not use CoreMinimal.h when possible.
Core public headers still will contain CoreMinimal.h if any included it before my IWYU changes.
#preflight 62c74ed0324817d81488b7ee

[CL 20993420 by bryan sefcik in ue5-main branch]
2022-07-07 17:41:31 -04:00
bryan sefcik
a9cc65bca4 Updated Core code to not use CoreMinimal.h when possible.
Core public headers still will contain CoreMinimal.h if any included it before my IWYU changes.
#preflight 62c73594756222ced4e48707

[CL 20991367 by bryan sefcik in ue5-main branch]
2022-07-07 16:02:19 -04:00
bryan sefcik
13f4c9daac Ran IWYU again after fixing some IWYU mapping issues.
#preflight 62c71b3ab3789302b1a0818b

[CL 20987904 by bryan sefcik in ue5-main branch]
2022-07-07 14:01:46 -04:00
bryan sefcik
62489fe2b3 Ran a pass of IWYU on Core.
Things to note regarding this pass:
* No includes are currently being removed from public headers.
* Any private file that has a #if is currently not being updated.

#rb Devin Doucette and Marc Audy
#preflight 62c62086756222ced497ff95
#preflight 62c626d12f2d04691814d331
#preflight 62c630107e606620fdd64e31
#preflight 62c635c6756222ced49a13cf

[CL 20979027 by bryan sefcik in ue5-main branch]
2022-07-06 21:44:18 -04:00
Dan Thompson
809c0f9062 Adding support for writing back metadata to an asset registry during iostore staging. Initially this is size information about the chunks.
#rb matt.peters
#rb carl-magnus.nordin
#preflight 62200571943787f1d2199924

[CL 19234043 by Dan Thompson in ue5-main branch]
2022-03-02 19:22:28 -05:00
carlmagnus nordin
fdc3ad3cb7 Move IoDispatcherFileBackend from Core to PakFile
#rnx
#preflight 61b89ea22e65a1df0459b68a
#rb pj.kack

#ROBOMERGE-OWNER: carlmagnus.nordin
#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 18465244 via CL 18465257 via CL 18465387 via CL 18465391 via CL 18465458 via CL 18465466
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18465469 by carlmagnus nordin in ue5-release-engine-test branch]
2021-12-15 03:24:48 -05:00
carlmagnus nordin
7070b5d25f IoStore:
Log toc signature hash when mounting containers
Add guard against loading toc files with a later version than we know about
#rnx
#rb per.larsson
#preflight 61a5d3ddae3418c7dfc944a1

#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 18322682 via CL 18322689 via CL 18323461 via CL 18323527 via CL 18323901 via CL 18323964
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v895-18170469)

[CL 18323991 by carlmagnus nordin in ue5-release-engine-test branch]
2021-11-30 09:00:54 -05:00
carlmagnus nordin
4a7d7d277c Implemented -checkpak support for iostore containers
#rnx
#rb pj.kack
#preflight 61a5dd60be6d0d4512d6cd04

#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 18322656 via CL 18322665 via CL 18323459 via CL 18323526 via CL 18323898 via CL 18323959
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v895-18170469)

[CL 18323988 by carlmagnus nordin in ue5-release-engine-test branch]
2021-11-30 09:00:43 -05:00
carlmagnus nordin
195c0d0907 IoDispatcher: Trace counters cleanup pass, fixed double counting of bytes scattered and removed duplicate counters.
#rb pj.kack
#rnx
#preflight 61968cbce86c9c10f4536a42

#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 18256237 via CL 18256241 via CL 18256243 via CL 18263290 via CL 18263345
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v895-18170469)

[CL 18263518 by carlmagnus nordin in ue5-release-engine-test branch]
2021-11-22 16:44:27 -05:00
Marc Audy
0c3be2b6ad Merge Release-Engine-Staging to Test @ CL# 18240298
[CL 18241953 by Marc Audy in ue5-release-engine-test branch]
2021-11-18 14:37:34 -05:00
aurel cordonnier
fc542f6cfd Merge from Release-Engine-Staging @ 18081189 to Release-Engine-Test
This represents UE4/Main @18073326, Release-5.0 @18081140 and Dev-PerfTest @18045971

[CL 18081471 by aurel cordonnier in ue5-release-engine-test branch]
2021-11-07 23:43:01 -05:00