Commit Graph

1194 Commits

Author SHA1 Message Date
francis hurteau
414d985756 Downgrade of error to warning until properly investigated with additional diagnostic
#rb danny.couture

[CL 29762187 by francis hurteau in ue5-main branch]
2023-11-15 17:38:24 -05:00
joe pribele
1e6438eb61 combining FObjectPropertyPtr and FObjectProperty and FClassPropertyPtr and FClassProperty together
simplifies usage of Object Properties and fixes a few edge cases

see zousar.shaker for issues

#rb zousar.shaker

[CL 29761675 by joe pribele in ue5-main branch]
2023-11-15 17:27:09 -05:00
francis hurteau
2009e4a2ef Implemented the LoadPackage function with optional param struct for the cooked editor legacy loader
#jira UE-200291
#rb danny.couture

[CL 29745750 by francis hurteau in ue5-main branch]
2023-11-15 10:01:34 -05:00
francis hurteau
06d9e4ccf6 Disable s.AsyncLoadingAlwaysProcessPackages by default for the legacy loader as it is a workaround for undiagnosed editor sync load deadlock
It however can cause issues when loading a mix of cooked and uncooked content.

#rb Danny.Couture

[CL 29661095 by francis hurteau in ue5-main branch]
2023-11-10 18:26:26 -05:00
francis hurteau
a871a842ee Implemented the legacy loader LoadPackage with a param struct
Reintroduce the usage in ConditionalFlushAsyncLoadingForLinkers

#rb Danny.Couture

[CL 29596393 by francis hurteau in ue5-main branch]
2023-11-09 10:39:23 -05:00
francis hurteau
d62eb15eb7 When using zenloader, mark gc'ed object as invalid in the linker
either the entire asset is getting gc'ed and they will be recreated on reload
or they are truly invalid and shouldn't be recreated once gc'ed now that the package is marked fully loaded

#rb Danny.Couture, Dave.Jones2

[CL 29523265 by francis hurteau in ue5-main branch]
2023-11-07 12:11:35 -05:00
danny couture
13f277d6f4 [ZenLoader] - Add PackageId reverse lookup in editor so warnings can contain useful information instead of just hashes
#rnx
#jira UE-191241
#rb Francis.Hurteau

[CL 29521062 by danny couture in ue5-main branch]
2023-11-07 10:41:30 -05:00
danny couture
50e404c5dc [ZenLoader] - Fix crash when trying to mix cooked and non-cooked content
#rnx
#rb Francis.Hurteau

[CL 29518881 by danny couture in ue5-main branch]
2023-11-07 09:01:53 -05:00
danny couture
37ae8de1ec [ZenLoader] - Fix crash when reloading a world twice in PIE due to missing fixup in GlobalImportStore
Simple repro
  - PIE in EngineTest
  - 2x Open /Engine/Maps/Templates/OpenWorld

#rnx
#rb Francis.Hurteau

[CL 29269279 by danny couture in ue5-main branch]
2023-10-31 05:27:11 -04:00
danny couture
98c425668a [ALT] - Fix TSAN warning on non-atomic package loading state
#rnx
#jira UE-180244
#rb Francis.Hurteau

[CL 29046743 by danny couture in ue5-main branch]
2023-10-24 11:18:01 -04:00
francis hurteau
71572f1b07 [ZenLoader] Do not propagate an instancing context to our imports when those aren't also instanced
#rb Danny.Couture

[CL 29045841 by francis hurteau in ue5-main branch]
2023-10-24 10:31:28 -04:00
francis hurteau
dd952d468f Make sure objects preloaded during the async deferred post load step of the legacy loader adds them to the list of objects it tracked
#rb Danny.Couture

[CL 28937362 by francis hurteau in ue5-main branch]
2023-10-19 18:03:02 -04:00
francis hurteau
1a8d7c8b49 [AsyncLoading] - Fix to avoid potential deadlock during loading
#rnx
#rb Francis.Hurteau
submitted on behalf of Danny.Couture

[CL 28936876 by francis hurteau in ue5-main branch]
2023-10-19 17:55:15 -04:00
paul chipchase
4e2debb49b Disable an error message checking the input of FChunkReadFileHandle::ReadRequest
#rb Martin.Ridgers
[FYI] Per.Larsson
#jira UE-193940
#rnx

- In CL 27337756  we added a log error if a FChunkReadFileHandle::ReadRequest attempted to read outside of the bulkdata payload, however this error is being triggered by FFileCache which is still using the handle to represent the entire .ubulk file.
- Although this is something we want to move away from in the future the read is currently valid so for now we are disabling the log message via a new define UE_ENABLE_BULKDATA_RANGE_TEST

[CL 28921144 by paul chipchase in ue5-main branch]
2023-10-19 11:53:44 -04:00
bhavan vaishnav
05b85f3266 This change adds the requested priority to the package summary so we can assess the order in which packages are loaded and if AsyncLoading is respecting package priorities as expected.
Discussions suggest we don't have enough data points to debug/ensure priorities are being respected, and this would be good to have in the engine in any case.

#rb simon.orr ionut.matasaru

[CL 28885835 by bhavan vaishnav in ue5-main branch]
2023-10-18 15:04:49 -04:00
danny couture
9deed45ff0 [ZenLoader]
- Simplify partial loading logic which is now only applied during flush if required
  - During partial load, only remove the exact RequestID being flushed so that other requestid can be flushed independently
  - Add missing timeout handling for postloads
  - Add missing support to run linkerload thread-safe postloads on async loading thread
  - Add unittest to make sure timeout is respected for postloads
  - Add unittest to make sure behavior of IsReadyForAsyncPostLoad is respected until a flush occurs
  - Add unittest for syncloads coming from thread-safe postloads and validate partial loading behavior
  - Make unittest run in editor only as they rely on saving package before reloading them
  - Refactor unittest for better reuse

#rb Francis.Hurteau

[CL 28799225 by danny couture in ue5-main branch]
2023-10-16 11:26:11 -04:00
KristofMorva
a75f380b2a Fix warning "Failed to find object 'Object None.None'" when deserializing FObjectAndNameAsStringProxyArchive.
Should also speed up loading a buffer with a lot of NULL objects considerably (see PR #6875).

#jira UE-92396
#PR 6875
#rb robert.millar

#ushell-cherrypick of 28497640 by kristof.morva1

[CL 28502883 by KristofMorva in ue5-main branch]
2023-10-05 12:09:52 -04:00
danny couture
fa346f675c [ZenLoader] Make static analyzer happy
#rb Francis.Hurteau

[CL 28463907 by danny couture in ue5-main branch]
2023-10-04 13:29:23 -04:00
graeme thornton
4547c6ceef Downgrading a warning to an error to unblock development while a proper fix is investigated
#rnx
[FYI] danny.couture

[CL 28433079 by graeme thornton in ue5-main branch]
2023-10-03 17:16:40 -04:00
danny couture
d40076e978 [ZenLoader]
- Add partial flush support from loading thread
 - Fix the ConvertFromType allowing to load the softobjectptr during its conversion to hardref even when called from ALT and zenloader is active
 - Add unittest to exercise softobjectptr to hardref conversion in ConvertFromType
 - Move asserts preventing flush from loading thread directly into legacy loader
 - Added a few veryverbose logs that were missing for linker load
 - Fix partial loading being applied to sync calls coming from a postload, preventing the object to be fully postloaded before finishing the sync load

#rb Francis.Hurteau

[CL 28373045 by danny couture in ue5-main branch]
2023-10-02 07:51:24 -04:00
andriy tylychko
d938d346bd fixed crash in AsyncFileHandle with unimplemented EnsureCompletion()
#jira UE-195894
#rb danny.couture, francis.hurteau

[CL 28313586 by andriy tylychko in ue5-main branch]
2023-09-28 07:09:26 -04:00
danny couture
fd1c551986 [ZenLoader]
- Further improve the partial load logic by avoiding some operations where possible
 - Also add some very verbose logging that can help identifying load ordering issues

#rnx
#rb Francis.Hurteau

[CL 28312996 by danny couture in ue5-main branch]
2023-09-28 06:41:49 -04:00
danny couture
68be86ed63 [ZenLoader] Fix export bundle being picked up recursively during a flush caused by a synchronous load invoked while running a previous export bundle of the same package
#rnx
#rb Francis.Hurteau

[CL 28274954 by danny couture in ue5-main branch]
2023-09-27 11:16:23 -04:00
danny couture
98cf4f14e5 [ZenLoader] Fix recursive loading behavior in zenloader by following more closely how legacy loading works
- Add a new partial loading mode where flushing the requestID will exit after serialization is done (after exportdone but before postload)
  - Automatically opt-in partial loading when an explicit load is called recursively from another load (treated as an import)
  - During recursive loading, attach postload of the recursive load to the caller's postload
  - Add loading unit tests to validate the new behavior for zenloader
  - Add a way to detect which loader is currently active to run unit tests on zenloader only

#rnx
#jira UE-190649
#rb Francis.Hurteau

[CL 28227792 by danny couture in ue5-main branch]
2023-09-26 10:26:04 -04:00
bob tellez
c00095c628 [Backout] - CL28184258
[FYI] danny.couture
Original CL Desc
-----------------------------------------------------------------
[ZenLoader] Fix recursive loading behavior in zenloader by following more closely how legacy loading works
  - Add a new partial loading mode where flushing the requestID will exit after serialization is done (after exportdone but before postload)
  - Automatically opt-in partial loading when an explicit load is called recursively from another load (treated as an import)
  - During recursive loading, attach postload of the recursive load to the caller's postload
  - Add loading unit tests to validate the new behavior for zenloader
  - Add a way to detect which loader is currently active to run unit tests on zenloader only

#rnx
#jira UE-190649
#rb Francis.Hurteau

[CL 28214800 by bob tellez in ue5-main branch]
2023-09-25 22:04:05 -04:00