Commit Graph

4 Commits

Author SHA1 Message Date
Unrealbot
3f463d80d4 Branch snapshot for CL 2959226
[CL 2959226 in Main branch]
2016-04-28 13:50:05 -04:00
Robert Manuszewski
1ff335d148 Copying //UE4/Dev-Core to //UE4/Main
==========================
MAJOR FEATURES + CHANGES
==========================

Change 2877442 on 2016/02/23 by Graeme.Thornton@GThornton_DevCore

	Allow 'dumpnonframe' stat command to also take an optional group name which will filter the output

Change 2879289 on 2016/02/24 by Graeme.Thornton@GThornton_DevCore

	Removed "-" in switch string for load time tracker. Makes -NoLoadTrackClear cmd line param actually work

Change 2879323 on 2016/02/24 by Graeme.Thornton@GThornton_DevCore

	PC builds now log engine initialization time with the load time tracker

Change 2879418 on 2016/02/24 by Graeme.Thornton@GThornton_DevCore

	Optimization for FSignedArchiveReader
	 - Compute incoming data hash in parallel on another thread while we serialize and decrypt the signature on the chunk cache worker thread

Change 2879419 on 2016/02/24 by Graeme.Thornton@GThornton_DevCore

	Extra stats for PakFile group
	 - Time reading from pak handles
	 - Number of open pak handles

Change 2881199 on 2016/02/25 by Graeme.Thornton@GThornton_DevCore

	Use 32 bit CRCs to hash chunk contents rather than SHA1. Much faster to compute than SHA1, and the decrypting 4 bytes rather than 20 is also a lot faster

Change 2885687 on 2016/02/29 by Robert.Manuszewski@Robert.Manuszewski_NCL_Stream1

	Close disregard for GC pool when disabling it to leave it in a good state.

Change 2887358 on 2016/03/01 by Steve.Robb@Dev-Core

	FlushCurrentThreadCache calls forwarded on from FMalloc wrappers.

Change 2887398 on 2016/03/01 by Steve.Robb@Dev-Core

	Stats for MallocBinned2.

Change 2887682 on 2016/03/01 by Robert.Manuszewski@Robert.Manuszewski_NCL_Stream1

	Added code to catch game hangs. If game thread is stalled for more than 5 seconds we will catch that, report its callstack and crash.

Change 2889573 on 2016/03/02 by Robert.Manuszewski@Robert.Manuszewski_NCL_Stream1

	Renamed the new StackWalkAndDump to ThreadStackWalkAndDump to avoid name collisions and build errors.

Change 2889687 on 2016/03/02 by Robert.Manuszewski@Robert.Manuszewski_NCL_Stream1

	Send thread heartbeats during potentially slow tasks.

Change 2889963 on 2016/03/02 by Robert.Manuszewski@Robert.Manuszewski_NCL_Stream1

	Added the ability to query the name of a thread given its TLS id.

Change 2890439 on 2016/03/02 by Steve.Robb@Dev-Core

	MallocBinned redirection header, to allow easy switching between MallocBinned and MallocBinned2.
	Switch back to MallocBinned by default.

Change 2891799 on 2016/03/03 by Steve.Robb@Dev-Core

	Shrunken GMallocBinned2MemSizeToIndex array.
	Improvement of some shared templated alignment-related functions.

Change 2891812 on 2016/03/03 by Graeme.Thornton@GThornton_DevCore

	Refactoring of pak signing to be more compatible with patching
	 - Store signatures in a block at the end of the pak file
	 - Optimize decryption of signatures by serializing them all when the pak is opened, and decrypting on the chunk cache worker thread between read requests.

Change 2891813 on 2016/03/03 by Robert.Manuszewski@Robert.Manuszewski_NCL_Stream2

	Don't compile UnrealPak with ICU

Change 2892386 on 2016/03/03 by Steve.Robb@Dev-Core

	GitHub #2070 - Fix incorrect FMallocDebug::Malloc alignment checking

Change 2894189 on 2016/03/04 by Steve.Robb@Dev-Core

	Memswap fixes for sizes of 3, 5, 6 and 7, and early out on 0.

Change 2894427 on 2016/03/04 by Robert.Manuszewski@Robert.Manuszewski_NCL_Stream1

	Don't display message boxes in unattended mode when D3D device is removed.

Change 2896912 on 2016/03/07 by Graeme.Thornton@GThornton_DevCore

	Disable reading of anything outside the pak file when in shipping builds

Change 2897053 on 2016/03/07 by Robert.Manuszewski@Robert.Manuszewski_NCL_Stream1

	A few improvements to thread heartbeat thread:
	- Don't check for heartbeat in debug builds
	- Don't spawn the thread heartbeat thread for programs
	- Ignore stalls when exiting

Change 2898752 on 2016/03/08 by Robert.Manuszewski@Robert.Manuszewski_NCL_Stream1

	Fixing race conditions when using malloc verify

Change 2899075 on 2016/03/08 by Robert.Manuszewski@Robert.Manuszewski_NCL_Stream1

	UnrealPak will now accept hex values for p and q. Added the command line param to trust p and q.

Change 2899091 on 2016/03/08 by Steve.Robb@Dev-Core

	Fix for TQueue::Peek const-correctness.

Change 2902775 on 2016/03/10 by Robert.Manuszewski@Robert.Manuszewski_NCL_Stream1

	Adding code to test signature keys.

Change 2905040 on 2016/03/11 by Robert.Manuszewski@Robert.Manuszewski_NCL_Stream1

	TBigInt Multiply is now 30x faster

Change 2905218 on 2016/03/11 by Steve.Robb@Dev-Core

	New FCString::Strspn and FCString::Strcspn functions.

[CL 2921423 by Robert Manuszewski in Main branch]
2016-03-24 13:53:55 -04:00
Robert Manuszewski
18e2561ceb Copying //UE4/Dev-Core to //UE4/Main
==========================
MAJOR FEATURES + CHANGES
==========================

Change 2836261 on 2016/01/20 by Robert.Manuszewski@Robert.Manuszewski_NCL_Stream1

	Flush FAsyncPackage cache after pre-load to reduce peak memory usage when async loading (5.5-10x).

Change 2828630 on 2016/01/14 by Robert.Manuszewski@Robert.Manuszewski_NCL_Stream1

	Adding debug code to catch memory stomps in the async loading buffers that's independent from mallocstomp allocator. Changed the signature of PageProtect functions to be able to read-only protect memory.

Change 2816129 on 2016/01/05 by Steve.Robb@Dev-Core

	Fixes for Realloc and alignment logic which caused redundant reallocations and incorrect binning.

Change 2821054 on 2016/01/08 by Steve.Robb@Dev-Core

	Further Realloc savings when realigning within a block.

Change 2806820 on 2015/12/17 by Steve.Robb@Dev-Core

	New AlignDown function, like Align, but which rounds a value/pointer down to the next alignment instead of up.

Change 2806816 on 2015/12/17 by Steve.Robb@Dev-Core

	Sort UHT modules by type to improve iteration times in conjunction with makefiles.

Change 2823235 on 2016/01/11 by Steve.Robb@Dev-Core

	UHT error messages about missing GENERATED_BODY() macros updated to represent intended use.

Change 2806815 on 2015/12/17 by Steve.Robb@Dev-Core

	Module types split into Game and Engine runtime versions.

Change 2833809 on 2016/01/19 by Robert.Manuszewski@Robert.Manuszewski_NCL_Stream1

	StaticLoadObject will now respect LOAD_NoRedirects flag.

Change 2811194 on 2015/12/22 by Bob.Tellez@Z2434_DevCore

	#UE4 Prevent loading packages that have a newer LegacyFileVersion since serialization for FCustomVersion is not forward compatible. UE-24443

Change 2806818 on 2015/12/17 by Steve.Robb@Dev-Core

	Removal of stats from MallocBinned2, to be readded later.

Change 2807069 on 2015/12/17 by Steve.Robb@Dev-Core

	Clarification of some bucket hashing terminology.

Change 2815117 on 2016/01/04 by Steve.Robb@Dev-Core

	Fix for a missing root build path on game modules.

Change 2815673 on 2016/01/05 by Steve.Robb@Dev-Core

	Move FMalloc verification into a proxy object.

Change 2822873 on 2016/01/11 by Steve.Robb@Dev-Core

	Fixes to off-by-one errors and removal of BinnedSizeLimit (assumed to be the same as MAX_POOLED_ALLOCATION_SIZE after OBO fix).

Change 2822923 on 2016/01/11 by Steve.Robb@Dev-Core

	Simplification of MemSizeToPoolTable indexing.

Change 2824974 on 2016/01/12 by Steve.Robb@Dev-Core

	Assert fixed.
	AllocateBlockFromPool's return value made debuggable.

Change 2825241 on 2016/01/12 by Steve.Robb@Dev-Core

	UHT now returns an error code on a warning when -warningsaserrors is specified.

Change 2825291 on 2016/01/12 by Steve.Robb@Dev-Core

	WarningsAsErrors enabled on UHT, after disabling the hardcoded behavior in CL# 2825241.

Change 2829846 on 2016/01/15 by Steve.Robb@Dev-Core

	GitHub #1938 - wrong Max value of enum is used during net serialization

Change 2829914 on 2016/01/15 by Robert.Manuszewski@Robert.Manuszewski_NCL_Stream1

	Reduce the amount of memory allocated for async cache buffers when guarding against memory stomps.

Change 2829988 on 2016/01/15 by Steve.Robb@Dev-Core

	Generalized large pool allocations.
	More redundancy removed.

Change 2831935 on 2016/01/18 by Chris.Wood@Chris.Wood.StreamB

	Added UserActivity property to crash description in CRP and CR website.
	[OR-12043] - Phone Home where crashes occur - pass context info to Crash Reporter

	DB column added to db-09 by ColinR matching this change.
	Published to server on Jan 18th 2016

Change 2834003 on 2016/01/19 by Chris.Wood@Chris.Wood.StreamB

	Added Linux to normal callstack parsing code on CR website
	[UE-25527] - Linux CrashReporter is missing information

	Published to server on Jan 19th 2016

Change 2835466 on 2016/01/20 by Joe.Conley@Joe.Conley_EGJWD5708_Dev-Core-Minimal

	Fix issue for cancelling package loads when there are still packages queued.  Call their PackageLoadedDelegate with a "Cancelled" result.

	Should solve remaining issue with UE-24062 - "Calling CancelAsyncLoading triggers an assert in FAsyncPackage::DetachLinker()"
	- (ULevelStreaming::AsyncLevelLoadComplete was not being called if packages were still queued when cancel was issued)

Change 2836803 on 2016/01/20 by Chris.Wood@Chris.Wood.StreamB

	CrashReportWebsite - fix exception thrown when parsing certain callstack formats

Change 2837952 on 2016/01/21 by Robert.Manuszewski@Robert.Manuszewski_NCL_Stream1

	Changing FAsyncIORequest to be stored as reference when cancelling IO requests to improve performance.

Change 2838289 on 2016/01/21 by Robert.Manuszewski@Robert.Manuszewski_NCL_Stream1

[CL 2845588 by Robert Manuszewski in Main branch]
2016-01-27 12:09:53 -05:00
Robert Manuszewski
dfc39a6ca5 Copying //UE4/Dev-Core to //UE4/Main
==========================
MAJOR FEATURES + CHANGES
==========================

Change 2783106 on 2015/11/30 by Robert.Manuszewski@Robert.Manuszewski_NCL_Stream2

	Introduced GC UObject clusters. GC clusters provide means to create disregard for GC subsets at load time (e.g. Materials with material expressions and their textures).

	- Saves about 25ms in reachability analysis (58ms -> 33ms)
	- UObject classes/instances can now be marked as cluster root objects with CanBeClusterRoot() function override.
	- Cluster creation is automatic. Clusters don't require any manual handling for GC to collect them when nothing is referencing them.
	- Moved token stream processing to a new class FFastReferenceFinder to make it more generic and re-usable by other code
	- Removed REFERENCE_INFO macro from GC code and replaced it with a local variable (saves about ~1.9ms: 33.2ms -> 31.3ms)

Change 2773094 on 2015/11/19 by Steve.Robb@Dev-Core

	Multicast script delegate check for existing bindings replaced with ensure.
	Multicast native delegate no longer checks for existing bindings.
	Removal of old delegate code.
	Some FORCEINLINEing to improve debugging experience of stepping into delegate code.

Change 2782180 on 2015/11/27 by Graeme.Thornton@GThornton_DesktopMaster

	Make scoped seconds timer class available outside of stats build. Normal usage macros still remain guarded
	Added SCOPE_SECONDS_COUNTER_RECURSION_SAFE which only times during the outmost instance of a recursive function
	Added SCOPE_SECONDS_COUNTER_RECURSION_SAFE_BASE and SCOPE_SECONDS_COUNTER_BASE which are defined in all build types, for easy temporary timing in Test/Shipping builds.
	Added a boolean parameter to the timer class which can be used to disable it without having to mess around with scoping the calling code

Change 2782635 on 2015/11/30 by Graeme.Thornton@GThornton_DesktopMaster

	Added GetTimeStampPair() to the filemanager and platformfile interfaces. Requests timestamps for a pair of files where we assume that both files would always exist at the same wrapper level. Allows us to skip file system queries for localization package lookups where the native file is in a pak but the localized file doesn't exist.

Change 2775153 on 2015/11/20 by Jaroslaw.Surowiec@Stream.1.JarekSurowiec

	CrashReportServer moved out of the not for licencees, a few fixes, removed RegisterPII

Change 2775560 on 2015/11/20 by Steve.Robb@Dev-Core

	FDelegateBase::GetDelegateInstance deprecated and replaced with FDelegateBase::GetDelegateInstanceProtected.

Change 2781138 on 2015/11/25 by Jaroslaw.Surowiec@Stream.1.JarekSurowiec

	Stats - Converted is using a new stats reader, a few more optimizations, should be 10x times faster

Change 2772990 on 2015/11/19 by Robert.Manuszewski@Robert.Manuszewski_NCL_Stream2

	Fixing potential dead lock when suspending and resuming async loading multiple times

Change 2773023 on 2015/11/19 by Robert.Manuszewski@Robert.Manuszewski_NCL_Stream2

	Support for references added through AddReferencedObjects in FArchiveReplaceObjectRef

Change 2781055 on 2015/11/25 by Steve.Robb@Dev-Core

	Changes to IDelegateInstance reverted to allow licensees an easier time when upgrading their use of the now-deprecated GetDelegateInstance() code path.
	New TryGetBoundFunctionName() to aid the debugging of delegate bindings in non-shipping configs.

Change 2773114 on 2015/11/19 by Steve.Robb@Dev-Core

	FMath::IsPowerOfTwo is now templated to take any type.

Change 2773643 on 2015/11/19 by Steve.Robb@Dev-Core

	GetDelegateInstance() calls replaced - delegate instances never compare equal unless you are comparing two unbound delegates (both null) or comparing a delegate with itself.

Change 2777686 on 2015/11/23 by Steve.Robb@Dev-Core

	GitHub #1793 - File write flags argument

Change 2780590 on 2015/11/25 by Steve.Robb@Dev-Core

	Fix for FArchiveProxy::operator<< overloads.

Change 2780845 on 2015/11/25 by Jaroslaw.Surowiec@Stream.1.JarekSurowiec

	#jira UE-23358 - MDD relies on hard-coded P4 depot paths (fixed source context for streams)

Change 2780962 on 2015/11/25 by Jaroslaw.Surowiec@Stream.1.JarekSurowiec

	Stats - Added FStatsWriteStream for basic saving stat messages into a stream, initial support for reading a regular stats file, minor performance optimization, coding standard fixes

	#jira UECORE-170 - Improve profiler loading performance (wip)

Change 2781887 on 2015/11/26 by Jaroslaw.Surowiec@Stream.1.JarekSurowiec

	Profiler/ProfilerClient - Removed unneeded synchronization points, replaces with task graph SendTo jobs, removed PROFILER_THREADED_LOAD, replaced with new stats loading mechanism,  should be around 2x times faster (4x since the optimization pass)

	#jira UECORE-170 - Improve profiler loading performance (wip)

Change 2781893 on 2015/11/26 by Steve.Robb@Dev-Core

	TCachedOSPageAllocator abstracted from MallocBinned2.
	Misc tidy-ups.

Change 2782198 on 2015/11/27 by Jaroslaw.Surowiec@Stream.1.JarekSurowiec

	Profiler - Better indication of the loading progress, should no longer freeze without a progress bar

	#jira UECORE-170 - Improve profiler loading performance (wip)

Change 2782446 on 2015/11/29 by Steve.Robb@Dev-Core

	Warn when calling delegates' Create* functions when they're not assigned to anything.

	#codereview robert.manuszewski

Change 2782538 on 2015/11/30 by Robert.Manuszewski@Robert.Manuszewski_NCL_Stream1

	#UE4 Removed DiskCachedAssetDataBuffer as it was not strictly necessary and was triggering a crash when loading the cached registry from disk. This data is now stored directly in DiskCachedAssetDataMap. It was already true that this map does not change outside of SerializeCache but now it is critical since NewCachedAssetDataMap keeps pointers directly to its values.

	Asset registry fixes by Bob Tellez. Possible fix for UE-23783.

Change 2782564 on 2015/11/30 by Robert.Manuszewski@Robert.Manuszewski_NCL_Stream1

	FReferenceCollector::AddReferenceObjects performance improvements for TArrays. ARO will no longer call HandleObjectReference multiple times but instead will call HandleObjectReferences just once (currently only implemented for FGCCollector). Reduces the number of virtual function calls while GC'ing.

Change 2782716 on 2015/11/30 by Steve.Robb@Dev-Core

	UObject serial number array initialized for debug visualization.

Change 2782933 on 2015/11/30 by Steve.Robb@Dev-Core

	Critical sections are no longer copyable.

Change 2783061 on 2015/11/30 by Steve.Robb@Dev-Core
2015-12-03 14:21:29 -05:00