danny couture f138d26887 Optimize Asset Registry Scanning ...

- Parallelize package reading in AssetGatherer
  - Implement scheduling ParallelFor as background prio
  - Saving the cache is now based on the amount of time it would take to redo the work
    so it scales better on many cores systems

  - Tested on AMD TR 3970X 128GB RAM with Crucial MP600 2TB
    - 33m43s to 3m45s (Cold Cache + No AV Exclusions)
    - 6m25s to 38.9s (Cold Cache + AV Exclusion on .uasset)
    - 1m25s to 20.8s (Warm Cache + No AV Exclusions)

    - All AssetCache .bin files deleted between runs
    - Cold cache = Cleared standby pages, mostly equivalent to a reboot
    - Warm cache = 2nd run of 2 consecutive runs

#rnx
#rb Jamie.Dale, Chris.Gagnon, Sebastien.Lussier, Steve.Robb

#ROBOMERGE-SOURCE: CL 12620239 via CL 12620242 via CL 12620244
#ROBOMERGE-BOT: RELEASE (Release-Engine-Staging -> Main) (v675-12543919)

[CL 12620246 by danny couture in Main branch]

2020-04-06 07:56:57 -04:00

..

Async.h

…

AsyncFileHandle.h

…

AsyncResult.h

…

AsyncWork.h

…

Future.h

…

IAsyncProgress.h

…

IAsyncTask.h

…

MappedFileHandle.h

…

ParallelFor.h

Optimize Asset Registry Scanning

2020-04-06 07:56:57 -04:00

TaskGraphInterfaces.h

TaskGraph can now multithreaded on a forked server with -PostForkThreading

2020-03-27 11:21:21 -04:00