Files
UnrealEngineUWP/Engine/Source/ThirdParty/Python/Win64/Lib/threading.py
Ben Marsh 348616bd78 Copying //UE4/Release-Staging-4.19 to //UE4/Dev-Main (Source: //UE4/Release-4.19 @ 3873914)
============================
  MAJOR FEATURES & CHANGES
============================

Change 3873906 by Dan.Oconnor

	Revised fix for preventing delegate functions from appearing in context menu when using the compilation manager - needed to run after CallDelegateHandler changed statement type

	#jira UE-51726

Change 3873614 by Dan.Oconnor

	Prevent delegate functions from appearing in context menu when using the compilation manager and prevent crash if such a function is somehow compiled

	#jira UE-51726

Change 3873428 by Ben.Zeigler

	#jira UE-54753 Fix class pin on SpawnActorFromClass to correctly support browse. It was inherting from the wrong pin widget
	Copy of 3873408

Change 3873083 by Ethan.Geller

	[4.19]#jira UE-54865 fix seconds -> milliseconds conversion issue. #rb aaron.mcleran #lockdown cristina.riveron

Change 3872714 by Dan.Oconnor

	Further revise fix for UE-53840, mistakenly reverted to old behavior when compiling synchronously
	#jira UE-53840

Change 3872648 by Ben.Zeigler

	#jira UE-54845 Fix crash in NextDebugTarget when there are no valid debug targets
	Copy of CL #3872636

Change 3872500 by Arne.Schober

	Back out changelist 3870283
	#jira UE-54838

Change 3872412 by Mark.Satterthwaite

	Remove now unnecessary r.Metal.ManualVertexFetch from the device profiles.

	#jira UE-54853

Change 3872313 by Martin.Wilson

	Add UI to Live Link Client to warn live link users about background performance throttling

	#jira none

Change 3872272 by Martin.Wilson

	Fix crash when clearing a skeletal mesh on a skeletal mesh component with an active Post Process Anim Instance

	#jira none

Change 3872238 by Mark.Satterthwaite

	Duplicate CL #3871025
	Make Manual Vertex Fetch a property of the shader platform for Metal - only the desktop platforms (METAL SM5/SM5_NoTess/MRT) will use manual vertex fetch. The mobile platforms use vertex descriptors. Prevents problems with cooked versions of games not working properly on Metal due to a mismatch between the runtime's Manual-Vertex-Fetch state versus the state used by the cooker when compiling shaders.

	#jira UE-54843

Change 3872087 by Yasiman.Ahsani

	Adding Python, libdisasm, musl, and LSS licenses.

	#JIRA n/a - adding licenses for new TPS

Change 3872037 by Ben.Marsh

	BuildGraph: Add a task for compiling MSBuild projects.

	#jira

Change 3871934 by Lina.Halper

	#jira: UE-54703

Change 3871595 by Michael.Trepka

	Fixed a problem with Mac editor not exiting with error code returned from GuardedMain function

	#jira UE-54830

Change 3870829 by Joe.Barnes

	Fix unintentional change to PhysX libs used in Debug builds.

	#jira ue-54817

Change 3870820 by Nick.Atamas

	Copying //Tasks/UE4/Dev-VR-4.19a@3870772 to Release-4.19 (//UE4/Release-4.19)

	#jira UE-54816

Change 3870755 by Chance.Ivey

	Min Android version set - should fix camera permissions issue found in #JIRA UE-54024 #rb none #fyi nick.atamas

Change 3870547 by Krzysztof.Narkowicz

	Fixed FreezeRendering on non editor builds: ComputeAndMarkRelevanceForViewParallel was calling FrozenMatricesGuard on multiple threads, reading and writing view matrices state in parallel.

	#jira UE-53640

Change 3870546 by Krzysztof.Narkowicz

	Fixed tesselation shader tex/uniform initialization in OpenGL3/4 path

	#jira UE-54471

Change 3870284 by Ben.Zeigler

	#jira UE-54583 Fix issue where loading multiple already loaded assets via Async Load Asset node could return the wrong asset on the loaded pin. The assign variable node was happening at a slightly wrong time
	Copy of CL #3870279

Change 3870283 by Arne.Schober

	SafeRelease SRVs that might be hold by the Vertexfactories (maybe due to indirect use in GlobalResources)
	Note that the VFs are not owners of the data, e.g the underlying Buffers might be released before this and this reference counting should be uneccessary

	#jira none

Change 3870098 by Ben.Marsh

	Fix OptimizeCode = CodeOptimization.Always causing compile errors in Linux debug builds. bUseInlining was not set correctly on the global compile environment used to build shared PCHs. Also fixed other settings not being propogated down from the target correctly.

	#jira UE-53855

Change 3870013 by Ben.Marsh

	UBT: Add an error if a user attempts to clean a target through hot-reload, rather than just failing to delete DLLs because they are locked.

	#jira UE-54179

Change 3870010 by Ben.Marsh

	UBT: Add an option to format output messages in a form that can be parsed by MSBuild. Prevents errors showing as "EXEC: Error:", and displays them correctly in the error list window.

	#jira

Change 3869814 by Ben.Marsh

	UBT: Unify command line arguments to use -Name=Value syntax. -Module <Name>, -ModuleWithSuffix <Name> <Suffix>, -Plugin <Path> and -Receipt <Path> are no longer supported.

	Also remove the RemoteRoot option, which was not used anywhere.

	#jira

Change 3869786 by Martin.Wilson

	Fix Live Link Remove Source button not working

	#Jira UE-54652

Change 3869660 by Martin.Wilson

	Fix missing message bus sources in the live link client (not repolling for new sources)

	#Jira UE-54712

Change 3869659 by Guillaume.Abadie

	Fixes SimpleComposure's BP_AdvancedCompositing keep allocating memory every frame.

	Credits for fixing the issue to Ron Radeztsk.

	#jira UE-54780

Change 3869401 by Lauren.Ridge

	Adding to the conditional in FindAllAncestorNamedSlotHostWidgetsForContent

	#jira UE-51470

Change 3869384 by Brandon.Schaefer

	Open the project in the explorer when NullSourceCodeAccess is done creating the project

	#jira UE-54630

Change 3869308 by Ben.Marsh

	PR #4452: Fixed FindFilesRecursively in IPlatformFilePak (Contributed by user37337)


	#jira UE-54568

Change 3869265 by Martin.Wilson

	Fix crash and subsequently found issues with unloading/reloading Maya Live Link plugin
	-Crash on reload due to core code getting reinitialize, made this a one time only thing (as Maya never gets rid of the module from memory)
	-Added manual ticking of FTicker, allows message bus objects to be cleaned up properly (cannot do this as part of normal flow as it is in engine code which we dont have)
	-rebuild binaries for Engine/Extras

	#Jira UE-54643

Change 3869206 by Benn.Gallagher

	Fixed crash updating clothing paint mode after tab spawners have been destroyed by the hosting application.

	#jira UE-54116

Change 3869064 by Benn.Gallagher

	Resolved skeletal mesh data changes and clothing section disable changes after collision. Re-added ability to disable sections at the mesh level and removed the ability to strip editor sections as these are required for a number of other features to correctly function.

	#jira UE-52557

Change 3869062 by Guillaume.Abadie

	Fixes "dynamic resolution is not supported on this platform" warning message being always visible.

	#jira UE-54655

Change 3868202 by Lauren.Ridge

	Fix for assert on expanding vector param in layered material
	#jira UE-54737

Change 3868161 by zak.parrish

	Replacing FaceARDebugUI with a blank UI until a new one can be created, due to a crash bug. #rb none #JIRA UE-54639

Change 3867750 by Ethan.Geller

	[4.19] #jira UE-54725 Fix for Envelope attack and release values not being properly set on Synth Components #rb aaron.mcleran #lockdown cristina.riveron

Change 3867657 by Lauren.Ridge

	Adding if with editor wrapper to new function
	#jira cis fix

Change 3867646 by Aaron.McLeran

	#jira UE-53867 Access violation on Switch when playing sound with specified time out of range

Change 3867340 by Lauren.Ridge

	Fixes to Material Layers from 4.19 preview feedback
	-Need to pass through base attributes better/have better default nodes in layer (optional Example Layers and Blends checkbox now enables this in Experimental Settings)
	-Parent in function should be editable
	-Enforce only two layers in a blend
	-Mat layer should warn if it has incorrect output in the layer itself
	-Enforce not being able to delete outputs
	-Warn about creating a MAL node inside a function

	#jira UETOOL-1312

Change 3867317 by Aaron.McLeran

	#jira UE-53867 Access violation on Switch when playing sound with specified time out of range

Change 3867000 by Lauren.Ridge

	Fix for folder favorites possibly becoming very large
	#jira UE-54704

Change 3866892 by Martin.Wilson

	Fix crash if clicking ok on message bus add source without having a source selected

	#jira UE-54572

Change 3866391 by Matt.Kuhlenschmidt

	Fix static analysis

	#jira UE-53379

Change 3866241 by Ryan.Vance

	#jira UE-54681
	Fixed missing Vulkan texture GetNativeResource implementation that was lost in a bad merge.

Change 3866071 by Nick.Shin

	UDN 412414

	update HTMl5 readme file

	#jira none

Change 3866005 by Max.Preussner

	Messaging: Preventing dangling references when removing message subscribers

	#jira UE-54680

Change 3865988 by Simon.Tourangeau

	Fix static analysis warnings

	#jira none

Change 3865895 by zachary.wilson

	Renaming QA-PhysicalLightUnits to TM-PhysicalLightUnits in QAGame.

	#JIRA UE-29618

Change 3865469 by Simon.Tourangeau

	Support for DX11 quad buffer stereo rendering

	#jira UEENT-704

Change 3865461 by Chris.Babcock

	Add a wait for audio thread to pause audio on going to background
	#jira UE-54301
	#ue4
	#android

Change 3865350 by Matt.Kuhlenschmidt

	Fix issue where cascade emitter UI would disapper

	#jira UE-53379

Change 3865336 by Arne.Schober

	REL - Fix UE-52356 Bone Weight

	#jira UE-52356

Change 3865257 by Ben.Marsh

	Fix editor failing to load content-only projects when compiled in DebugGame.

	#jira UE-54661

Change 3865238 by Simon.Tovey

	Pulling Olaf's fix from 3832595 over to release

	#tests EngineTests boots in vulkan
	#JIRA UE-54394

Change 3865191 by Sorin.Gradinaru

	UE-54317 DXT apps fail on first launch when 'For Distribution' enabled, Unsupported Texture Format

	#jira UE-54317
	#Android
	#4.19

	From //Dev-Mobile/3863322

Change 3865190 by Sorin.Gradinaru

	UE-54175 Selecting For Distribution no longer sets configuration to Shipping

	#UE4
	#4.19
	#jira UE-54175

	From //Dev-Mobile/3863371
	"For Distribution" enable-> reset "Build COnfiguration" to Shipping and force serialization to DefaultGame.ini (see UE-52845)

Change 3865056 by Jamie.Dale

	Fixed culture being incorrect when added via the Localization Dashboard

	#jira none

Change 3864826 by Max.Preussner

	WmfMedia: Added missing scope lock

	#jira UE-54365

Change 3864055 by Aaron.McLeran

	#jira UE-54213

	Crash fix for GC'd plugin settings objects. Adding to AddReferencedObjects.

Change 3863775 by Andrew.Porter

	MediaFrameworkTest: Removing Platform Media Source TVOS test content

	#jira UE-29618

Change 3863714 by Dan.Oconnor

	Make array, set, and map nodes no longer switch object type when a pin is disconnected. Also, you can no longer attach unrelated map types to the MakeMap node if only the value pin has been inferred

	#jira UE-54634

Change 3863696 by Max.Chen

	Sequencer: Fix crash open a new sequence while another is still active.

	#jira UE-54620
	#jira UE-54624

Change 3863638 by Dan.Oconnor

	Make sure all blueprints that are skeleton compiled get a BroadcastChanged notification, also run BroadcastChanged after reinstancing, matching pre compilation manager behavior
	#jira UE-54193

Change 3863494 by Jamie.Dale

	Ensure Py_SetPythonHome is set correctly before the embedded Python interpreter is initialized

	#jira UE-54345

Change 3863433 by Max.Preussner

	MediaPlayerEditor: Removed tvOS from list of available platforms in Platform Media Source assets

	tvOS currently reports itself as iOS, so it cannot have its own setting.

	#jira UE-54559

Change 3863406 by Lauren.Ridge

	Fix for a crash when filtering a dropdown with no set value
	#jira UE-54631

Change 3863238 by Michael.Kirzinger

	#jira UE-52730: Fix mac voip crash

Change 3862586 by Marc.Audy

	Clean up rendering thread heartbeat checks that got mangled in various integrations

	#jira

Change 3862247 by Guillaume.Abadie

	Fixes editor lines that were using a simple element blending mode that was generating an incorrect alpha channel.

	#jira UE-53830

Change 3862120 by Guillaume.Abadie

	Fixes USceneCaptureComponent::HiddenActors always staying gray out in world editor.

	#jira UE-51458

Change 3861363 by Jamie.Dale

	Static analysis fixes

	#jira none

Change 3861150 by Matt.Kuhlenschmidt

	Fix static mesh editor displaying 0 for all stats on LODs > 0

	#jira UE-53776

Change 3860990 by Dan.Oconnor

	Clear subobjects not recreated on load, e.g. because we're running with -game and the subobject was created using CreateEditorOnlySubobject
	#jira UE-54551

Change 3860972 by Nick.Shin

	HTML5 - detect "SyntaxError: " and do a forced reload

	- an actual syntax error would be caught during compile time
	- this is usually error condition is usually seen when browser is running "old/partial" cached data and it's fairly safe to just reload the page

	#jira UE-54017  QAGame fails to launch properly on HTML5 Firefox 64 bit

Change 3860951 by Arne.Schober

	Fix not releaseing SRV on render thread for FPositionVertexBuffer, FStaticMeshVertexBuffer, FColorVertexBuffer, FStaticMeshInstanceBuffer.

	#jira UE-54587

Change 3860950 by tim.gautier

	QAGame: Updating ML_Stone to include Texture Coordinate and Panner functions
	#jira UE-29618

Change 3860833 by Michael.Dupuis

	#jira UE-54181: Repopulate the foliage list when existing simulate mode
	Let the GC know that internal struct hold UObject ptr

Change 3860762 by Jamie.Dale

	Ensure we invoke the correct version of Python from UBT

	#jira UE-54345

Change 3860676 by Simon.Tourangeau

	Remove DirectoryWatcher warning in output log

	#jira UEENT-846

Change 3860598 by Lauren.Ridge

	Fix for crash on opening new material instance
	#jira UE-54589

Change 3860338 by Michael.Lentine

	Integrate changes for fixing MorphTargets.

	#jira 54398

Change 3860215 by Ben.Marsh

	UAT: Fix exception consturcting target rules assembly inside UAT, now that there's an abstract TargetRules class.

	#jira UE-54578

Change 3860186 by Matt.Kuhlenschmidt

	Fix crash top 10 with the font editor shutting down and then a dpi event occuring

	#jira UE-54543

Change 3859854 by Graeme.Thornton

	PR #4124: VSCode: Changed VisualStudioCodeSourceCodeAccessor to work with non-default VSCode install paths on Linux (Contributed by christopherreed)


	#jira UE-51289

Change 3859848 by Graeme.Thornton

	Fix crypto.cs reading the wrong ini setting names for uasset encryption settings

	#jira UE-54566

Change 3859684 by Ben.Marsh

	PR #4436: Fix compile error when building BlankProgram because incorrect directory path (Contributed by windkey)


	#jira UE-54392

Change 3859657 by Ben.Marsh

	Fix FTransform being passed by value, causing alignment error when compiling for Win32.

	#jira

Change 3859312 by nick.bullard

	Updating AEOverviewMan to remove reference to sub-level AEOverviewSWP which was deleted in 3859278
	Still need to update menu to remove selection

	#jira UE-50784

Change 3859278 by Nick.Bullard

	Deleteing AEOverviewSWP.umap per request of Developer. Also updated TM-AnimPhys which refereneced this map as well.

	"This test map is a custom C++ implementation of a "procedural sound wave". This code is super janky and not worth fixing up. I'm sure there's a legit thread safety problem in there but the code is in QAGame only.

	We've since implemented "synth components" which do what this test does in a much safer/better way and maintaining/testing this old thing is *not* worth the time."

	#jira UE-50784

Change 3859124 by Dan.Oconnor

	Fix long standing crash when duplicating a blueprint that is missing its parent class. Entries in CrashReport go back to 4.16

	#jira UE-54468

Change 3859086 by Ryan.Vance

	#jira UE-54470

	We need to set the viewport in both cases.

Change 3859006 by Dan.Oconnor

	Revise fix for archetype lookup when reinstancing. During compilation we do not want to force use of the up to date class

	#jira UE-54541

Change 3858990 by mason.seay

	Cleaned up blueprints to remove compile errors

	#jira UE-29618

Change 3858945 by Aaron.McLeran

	#jira UE-54265

	PR #4428: Fixing Envelope Bug in AudioMixer (Contributed by Chrispykins)


Change 3858719 by Aaron.McLeran

	#jira UE-54552 Fix for sample buffer reader

Change 3858647 by Ethan.Geller

	PR #4439: Removes ambiX -> FuMa conversion (Contributed by mgorzel)

	#jira UE-54407

Change 3858364 by Michael.Dupuis

	#jira UE-52049: There was a case where adding and removing multiple time would lead to reordering the instances and this would cause the regeneration of the random stream for all the reorded instances.

Change 3858268 by Dan.Oconnor

	Prevent postload logic running on REINST and SKEL classes

	#jira UE-54531

Change 3858205 by Mitchell.Wilson

	Removed particle looping from some emitters to resolve anim notify warnings.
	#jira UE-53823

Change 3858148 by Lauren.Ridge

	Changes based on Material Layer Feedback from previews
	- (Temporary) Disabled being able to create a layer or blend in the asset dropdown
	- Sections of the stack that have been disabled now inactivate that part of the UI
	- Create Function Instance now indicates if you are making a layer or a blend
	- Parent dropdown has been removed from layers and blends. Where relevant, a filter button has been added instead.

	#jira UETOOL-1328

Change 3857933 by Michael.Dupuis

	#jira UE-45854: Properly unregister callback when replacing foliage type with another one

Change 3857898 by Michael.Dupuis

	#jira UE-54396: Remove the Ensure as it could be possible that the Landscape Info is invalid during an undo operation

Change 3857878 by Max.Chen

	Sequencer: Assign the sequence id after the template is compiled.

	Copy from Dev-Sequencer

	#jira UE-54462

Change 3857808 by Michael.Dupuis

	#jira UE-54421: Prevent edition during Simulate when clicking on actor

Change 3857786 by Rolando.Caloca

	UE4.19 - Fix recompute tangents and skin cache for OpenGL

	#jira UE-42108
	PR #3271

Change 3857549 by Lina.Halper

	another jittering issue due to revision number change
	clear the motion vector after compile

	#jira: UE-53930

Change 3857439 by Lina.Halper

	Clear motion vector when end of sequencer when in editor world

	#jira:UE-54057

Change 3857384 by Graeme.Thornton

	Restore fix for visual studio source code accessor not correctly determining that a content project has no solution and opening a fresh instance of visual studio

	#jira UE-50020

Change 3856596 by Chris.Babcock

	Fix ResonanceAudioApi Android library architecture filtering
	#jira UE-54478
	#ue4
	#android

Change 3856449 by Michael.Dupuis

	#jira UE-35097: Various cosmetic changes that were done in phase 2 that help improve clarity of the design.

Change 3856415 by Dan.Oconnor

	Fix regression when loading instances that have had their class deleted. Issue detected by static analysis

	#jira UE-54467

Change 3856332 by Ben.Marsh

	Resaving TP_HandheldARBP assets to fix version again.

	#jira

Change 3856319 by Ben.Marsh

	Back out changelist 3855588

	Causes build failure in UFE because it introduces a dependency on the Engine module. UFE compiles with WITH_ENGINE = 0.

	#jira UE-54472

Change 3856292 by Ben.Marsh

	Fix app-local dependencies not being included in binary builds, and only working for code projects in source builds.

	#jira UE-54448

Change 3856190 by Martin.Wilson

	Fix bone access mismatch between raw mesh bones and final bones (that include virtual bones too)

	#jira UE-54266

Change 3856169 by Ben.Marsh

	Tag XGEControlWorker.exe as a build product, so it's included in the binary distro.

	#jira UE-54283

Change 3856123 by Chris.Babcock

	Fix missing ARCore file
	#jira UE-54453
	#ue4
	#android

Change 3856005 by Richard.Wallis

	Clone of Dev-Rendering CL 3855993 - turn off FShaderCache.

	#jira UE-52928

Change 3855961 by Jian.Ru

	Copy 3855047 - fix DFAO Nan problem
	#jira UE-54403

Change 3855811 by Martin.Wilson

	Add build process for Maya Live Link plugin (standalone, manually triggered) + add built binaries to Engine/Extras (Maya 2016, Maya 2017, Maya 2018)

	#jira none

Change 3855758 by Cosmin.Sulea

	UE-53569 - tvOS does not package or launch-on

	#jira UE-53569

Change 3855727 by Ben.Marsh

	Resaving assets with a versioned build in the editor, to fix warnings building DDC.

	#jira

Change 3855626 by Adrian.Siminciuc

	https://jira.it.epicgames.net/browse/UE-50979 (MP4 doesn't resume playback when iOS 11 device wakes from sleep)

	#jira UE-50979

Change 3855588 by Graeme.Thornton

	Fix visual studio solution path being incorrect for content projects

	#jira UE-50020

Change 3855283 by Ben.Marsh

	Fix race condition where stdout/stderr write handles could be inherited by multiple processes, resulting in them not being closed (and the process exit being detected) until all processes that inherited them had been closed. Improves performance of ParallelExecutor.

	#jira

Change 3855009 by Chance.Ivey

	Resaving with version number. #JIRA-54330 #rb none

Change 3854943 by Dan.Oconnor

	Fix archetype lookup when searching hierarchy that has been partially reinstanced

	#jira UE-53840

Change 3854882 by Ryan.Vance

	#jira UE-54438

	Removing vr related references to screen percentage.
	Removing previously removed gvr screen percentage code that came back in an integration from google.

Change 3854806 by Mike.Beach

	Mirroring part of CL 3802176 to fix a crash that can occur when users try to use the default 'DisplayModel' on MotionController components. Also provided users a better error message to explain why a model might not be showing up.

	#jira UE-54214

Change 3854680 by Chance.Ivey

	Saving assets with version number#JIRA UE-54330 #rb none

Change 3854652 by Uriel.Doyon

	Added a tooltip to the EV100 slider in the exposure menu.
	Using game settings now disables the slider.
	#jira UE-53945

Change 3854605 by Dan.Oconnor

	Make sure we don't create objects outered to a placeholder object, also make sure that archetypes that are reinstanced on load are relinked in to the linker table so that they are postload'd (and the old instance isn't)

	#jira UE-53954

Change 3854274 by Brandon.Schaefer

	Changes in CL 3842286 changed the function glslang::GlslangToSpv and requires a rebuild on Linux

	#jira UE-54302

	#codeview Arciel.Rekman

Change 3854255 by Phillip.Kavan

	Fix a scoping issue for local instanced subobject references in nativized Blueprint C++ code. Also, don't emit redundant assignment statements for instanced subobject reference properties.

	- Mirrored from //UE4/Dev-Framework (3853349).

	#jira UE-53960

Change 3854177 by Ethan.Geller

	#jira UE-54415 set EnabledByDefault to false for Resonance Audio

Change 3854123 by Ethan.Geller

	#jira UE-54410 set AudioComponentID

Change 3853775 by Lauren.Ridge

	Minor cleanup
	#jira UE-54054

Change 3853772 by Lauren.Ridge

	Don't create widgets when just testing if the selected widget is a replacement candidate
	#jira UE-54054

Change 3853715 by Rolando.Caloca

	UE4.19 - Fix for OpenGL overwriting texture units

	#jira UE-54401

Change 3853655 by Ben.Marsh

	Add a retry loop on creating the first directory before copying files. Attempt to work around problems copying to shared folders.

	#jira

Change 3853535 by Ben.Marsh

	Expose the engine compatible changelist to Perforce. If EnginePatchVersion > 0, this will be the changelist of the original .0 release.

	#jira

Change 3852583 by Nick.Atamas

	Resaved assets so they don't produce DDC warnings.
	#jira none

Change 3852552 by Uriel.Doyon

	Fixed Pre-Exposure shader compilation and Temporal AA issue.

	#jira UE-54276

Change 3852354 by Nick.Atamas

	Hopefully fixes the static analysis warning from jira issue.
	#jira UE-54332

Change 3852281 by Nick.Atamas

	Merging CL 3851690 from //Tasks/UE4/Dev-VR-4.19a/... to //UE4/Telease-4.19/...
	#jira UE-54331

Change 3852274 by Simon.Tourangeau

	Back out changelist 3851041 until Win7 issue is resolved.

	#jira UE-54354

Change 3852208 by Jamie.Dale

	Merging CL# 3821754 from //UE4/Dev-Enterprise

	Class property conversion now goes through NativizeClass/PythonizeClass

	This allows it to coerce from Python wrapped object types

	#jira none

Change 3852202 by Jamie.Dale

	More explicit handling of EngineDir for Python SDK

	#jira UE-54345

Change 3851982 by Brandon.Schaefer

	Workaround using a hardcoded path

	#jira UE-54136

Change 3851748 by Michael.Dupuis

	#jira UE-53904: Put the code back to what it was before, as we really only want to perform this code if bIsLayerThumbnail is true, it was changed to fix another issue complaining about a missing shader, that end up being added to the existing functions

Change 3851545 by Marc.Audy

	Remove debugging code that slipped in

	#jira none

Change 3851461 by Ben.Marsh

	Fix #includes with backslashes from crashing UBT.

	#jira UE-53996

Change 3851391 by Jamie.Dale

	Updated Python to prefer our TPS SDK

	#jira UE-54345

Change 3851372 by Jamie.Dale

	Added bat file to copy the Python SDK into TPS

	#jira UE-54345

Change 3851218 by Ben.Marsh

	Add missing template to installed engine build.

	#jira UE-54339

Change 3851117 by andrew.porter

	QAGame: Removing duplicate map

	#jira UE-29618

Change 3851041 by Simon.Tourangeau

	Support for DX11 quad buffer stereo rendering

	#jira UEENT-704

Change 3850548 by Ben.Marsh

	Add TP_HandheldARBP to installed engine build.

	#jira

Change 3850424 by Ben.Zeigler

	Fix reported memory for asset registry to be correct, extracted from a larger change

	#jira none

Change 3850324 by Ryan.Vance

	#jira UEVR-1050

	Hook up the Rift dynamic res to the new IDynamicResolutionState framework

Change 3849819 by JeanLuc.Corenthin

	Unhide Datasmith plugins in "regular" projects to allow easy project conversion

	#jira UEENT-795

Change 3849302 by Martin.Wilson

	Live Link is no longer experimental in 4.19 (moved from experimental folder to Animation)

	#jira none

Change 3849238 by Max.Chen

	Sequencer: Remove bKeepStaleTracks so that stale tracks are now always purged.

	#jira UE-54248

Change 3849211 by Michael.Dupuis

	#jira UE-54181: Prevent foliage edition during PIE or simulate mode

	#coderevire jack.porter

Change 3849123 by Benn.Gallagher

	Fixed crash in clothing actor creation when the clothing simulation mesh has no simulated vertices

	#jira UE-53741

Change 3849120 by Benn.Gallagher

	Fixed crash adding empty materials to destructible meshes in the destructible mesh editor

	#jira UE-53938

Change 3849047 by Jurre.deBaare

	Move material baking out of experimental
	#fix follow-up also remove the entry from experimental settings
	#jira UE-52685

Change 3848808 by Michael.Dupuis

	#jira UE-35097: Remove IncludeTessellationInShadowLOD & RestrictTessellationToShadowCascade from 4.19 as there is currently a bug regarding this in the dynamic rendering code path and would be too risky to fix for now.
	Fixed dynamic shadow code path issue vs static code path

Change 3848659 by Lina.Halper

	Fix issue with animation resetting in the sequencer

	#jira: UE-54047

Change 3848635 by Rolando.Caloca

	UE4.19 - Fix static analysis

	#jira UE-50449

Change 3848515 by Sorin.Gradinaru

	Unshelved from pending changelist '3843541':

	WebBrowser Android crash on 4.4.3

	#jira UE-53247
	#Android
	#4.19

	Crash on Android 4.x.x caused by a call to a Api Level 21 method

Change 3848514 by Jurre.deBaare

	Moving over:
	CL 3832173
	"Failed to import Alembic files
	#jira UE-53941
	#fix Change Alembic thirdparty library setup
		- Removed old unused library files
		- Updated batch files for building HDF5 ILMBase and Alembic libraries to use new AlembicDeploy path
		- HDF5 is now build as a Dynamic library (which also enableds multi-threading)
		- Added DLLs for HDF5 to build.cs file

	CL 3838053
	"Adding missing hdf5 dynamic libraries

Change 3848245 by Ben.Marsh

	Downgrade warning about not using XGE shader compilation to a log message. Build machines do not have XGE.

	#jira UE-54237

Change 3847300 by Phillip.Kavan

	Fix shadowed local variable.

	#jira UE-54141

Change 3846922 by Max.Preussner

	ImgMedia: Fixed image media player never finished initialization if loading failed

	Copied from Dev-Sequencer CL# 3846902

	#jira UE-54247

Change 3846831 by Arciel.Rekman

	Linux: only use lld for x86_64 (UE-54144).

	- lld support for other architectures seems to not ready for prime time.

	#jira UE-54144

Change 3846771 by Lauren.Ridge

	Material window now uses the background color set in preview scene.
	#jira UE-52215

Change 3846705 by Ben.Marsh

	Fix batch file paths not being quoted correctly when run through XGE.

	#jira

Change 3846550 by Lauren.Ridge

	Toggling Show Background now updates the background as well
	#jira UE-52250

Change 3846417 by Matt.Kuhlenschmidt

	Fix crash resizing shootergame window

	#jira UE-53137

Change 3846295 by Rolando.Caloca

	UE4.19 - New Vulkan descriptor pooling mechanism (enabled on non-android)

	#jira UE-50449

Change 3846273 by tim.gautier

	QAGame: Updating Material Layer test assets to include more Params
	- Added temp assets, quicker repros for bugs
	#jira UE-54176, UE-54165

Change 3846255 by Lauren.Ridge

	Parameter tab is the primary tab for material instances

	#jira UE-54092

Change 3846086 by Chris.Babcock

	Add missing SecureHash.h include
	#jira UE-54026
	#PR #4417
	#ue4
	#android

Change 3846049 by Martin.Wilson

	Fix Set Root Motion Enabled Anim Data Modifier node (previously didn't set the enabled flag)

	#jira UE-54220

Change 3846033 by Martin.Wilson

	Fix root motion being repeatedly applied ( Clear() call only clears bHasRootMotion flag, not the transform itself )

	#Jira UE-54219

Change 3845991 by andrew.porter

	QAGame: Updating bindings on activechannels sequencer test content

	#jira UE-29618

Change 3845933 by Lauren.Ridge

	Check for original material being valid

	#jira UE-54166

Change 3845920 by Martin.Wilson

	Optimized redundant key removal

	#jira UE-51303

Change 3845812 by Matt.Kuhlenschmidt

	Fix not being able to change BSP brush shape

	#jira UE-53738

Change 3845790 by Martin.Wilson

	Fix for assert failure when accumulating root motion in debug.

	#jira UE-53955

Change 3845730 by JeanLuc.Corenthin

	Fix build breakage
	warning: resaved disc.uasset with 4.19.0 preview #1
	failure: set the correct default mesh for AreaLightStruct.uasset

	#jira none

Change 3845693 by Lina.Halper

	Fix issue with previewing pose asset with curve data

	#jira: UE-53967

Change 3845533 by Andrew.Rodham

	Sequencer: Fixed sub sequences potentially being loaded during AddReferencedObjects

	#jira UE-54173

Change 3845472 by Thomas.Sarkanen

	Prevented debug object selection dropdown from displaying objects with pending kill outers

	#jira UE-54045 - Animation Blueprint Editor Crashes on Compile if the Debug Instance Selection is Other Than No Object or Preview Instance

Change 3845401 by Yannick.Lange

	Reverting thumbnail capture from viewport.
	#jira UE-53775
	#jira UE-53701

Change 3844693 by JeanMichel.Dignard

	Changed IES texture brightness to be the max candela value and set the texture multiplier to be 1. This fixes the IES lights intensity being too high.

	#jira UEENT-632

Change 3844689 by JeanLuc.Corenthin

	Update Datasmith content assets to latest

	Copying fix from Dev-Enterprise by Jean-Luc Corenthin CL 3809803
	Updated assets with correct release version
	Cleanup some paths on static meshes and texture

	#jira UEENT-759
	#jira UEENT-657

Change 3844571 by Martin.Wilson

	Fix motion controller motion source pin still showing when pin is connected to something

	#Jira UE-53236

Change 3844564 by Martin.Wilson

	Due to previously fixed bug some anim blueprint nodes could have duplicated guids, this fixes them

	#Jira UE-54174

Change 3844545 by Jamie.Dale

	No longer attempt to parse group separators for numeric inputs

	This avoids some ambiguity when parsing numbers for languages such as German

	#jira UE-54170

Change 3844221 by Nick.Shin

	HTML5 - filter out "windows/super" keys - these are not used in UE4

	- but, keycode are not the expected "91 or 92" values, SDL keys are "227 & 231" instead...

	#jira UE-54056  HTML5 crashes inside browser upon pressing windows key

Change 3843937 by JeanMichel.Dignard

	Fixed a crash when right clicking on a static mesh for which its AssetImportData class is currently unavailable (ie: in an unloaded plugin).

	#jira UEENT-764

Change 3843929 by Peter.Sauerbrei

	pull over fix for bad directory when copying launch images
	#jira UE-53177

Change 3843658 by Thomas.Sarkanen

	Text is red again in anim viewports

	#jira UE-53224 - Colouring removed from "Animation is being edited" warning messages

Change 3843657 by Thomas.Sarkanen

	Enable picking via Enter for details panel asset pickers

	The previous (4.18) behavior was to only allow selection of the previous/next item in the list with arrow keys. A fix (CL 3783114) for pickers with many assets broke this. This change now allows for selection of any item by navigating with arrow keys and pressing Enter, as suggested in the Jira.

	#jira UE-53440 - Unable to select assets within modal Asset Selection dropdowns after navigating with keyboard entry

Change 3843120 by Dan.Oconnor

	Avoid fixing component template games outside the editor, this logic does not work for games that have been nativized
	#jira UE-54009

Change 3842841 by Ben.Zeigler

	#jira UE-50020
	Switch visual studio module back to using absolute paths so go to definition works, broken in CL #3796157

Change 3842582 by Lauren.Ridge

	Guards against the widget passed to scrollwidgetintoview being null
	#jira UE-54037

Change 3842575 by Max.Chen

	Sequence Recorder: Stop recording if the preview window is destroyed.

	#jira UE-49778

Change 3842551 by Michael.Dupuis

	#jira UE-35097: Minor bug fix, documentation, etc. to the landscape optim that was done in the phase 2 that have 0 risk.

Change 3842371 by Max.Preussner

	Media: Merged 4.19 fixes from Dev-Sequencer

	CL 3807293 WmfMedia: Fixed YUY2 video format strides
	CL 3827988 MediaPlayerAssets: Fixed Media Texture is not linked to Media Player when created together
	CL 3805414 MediaAssets: Setting valid GUID when initializing media texture resource
	CL 3804183 MediaAssets: Added missing lock in media sound component
	CL 3831580 MfMedia: Media open events generated in same order as on other platforms
	CL 3807193 WmfMedia: Fixed incorrect buffer stride for RGB32 video tracks

	#jira UE-53532
	#jira UE-53328

Change 3842356 by Max.Preussner

	ImgMedia: Fixed ensure when cooking project that uses ImgMediaSource

	#jira UE-51631

Change 3842335 by Aaron.McLeran

	#jira UE-54087

	PR #4419: Fixes a crash due to nullptr dereference (Contributed by mgorzel)


Change 3842286 by Rolando.Caloca

	UE4.19 - Fix for static analysis
	- Glslang 1.0.65.1

	#jira UE-54128

Change 3842222 by andrew.porter

	QAGame: Updating framerate of EXR_Sequence

	#jira UE-29618

Change 3842211 by Ben.Marsh

	Fix determination of Windows version string. The manifest for UE4 applications declares compatibility with Windows 10 nowadays, so we get accurate version numbers returned from GetOSVersionInfo().

	#jira UE-54035

Change 3842163 by Cosmin.Sulea

	UE-53303 - We do not check for remote connection before attempting remote shader compile, causing crashes when misconfigured

	#jira UE-53303

Change 3841770 by Max.Chen

	Sequencer: Fix to allow keying of an arbitrarily deep property path.

	#jira UE-54095

Change 3841758 by Max.Chen

	Sequencer: Fix unbound possessable components when pasting spawnables.

	#jira UE-54104

Change 3841415 by Lauren.Ridge

	Renaming a material layer or material layer blend will no longer cause the asset to appear removed from the stack
	#jira UE-53942

Change 3841327 by Arciel.Rekman

	Linux: fix Debug build (UE-53855)

	- A workaround. UBT should be using proper PCH files instead.

	#jira UE-53855

Change 3840975 by Rolando.Caloca

	UE4.19 - Updated VulkanRHI
	- Fixes for GPU frame time
	- Fixes for CPU performance

	#jira UE-50449

Change 3840838 by Michael.Dupuis

	#jira UE-53944: Make sure the LOD generated is in the valid range to prevent the crash

Change 3840693 by Ben.Zeigler

	#jira UE-53923 Fix regression in 4.19 where PrintScriptCallstack is not always safe to call from the immediate window. I narrowed down the issue to some confusing optimized code so I turned off optimization
	Copy of CL #3840692

Change 3840680 by Aaron.McLeran

	Bringing fixes from Dev-AnimPhys to 4.19.

	#jira UE-53903 crash on load with oculus audio and old audio engine
	#jira UE-52786, UE-53910 Fix for broken spatialization on xaudio2, old audio engine.

Change 3840663 by Rolando.Caloca

	UE4.19 - Fix for layout ensure on HMD projects on Vulkan

	#jira UE-50265

Change 3840577 by Rolando.Caloca

	UE4.19 - Fix for CPUs with more than 16 cores

	#jira UE-53434

Change 3840551 by andrew.porter

	QAGame: Setting Allow Bindings from Asset to false

	#jira UE-29618

Change 3840491 by Ben.Zeigler

	#jira UE-31662 Fix regression with renaming parent inherited function. It was not correctly searching the parent's skeleton class during the child's recompile so it was erroneously detecting the parent function as missing
	Copy of CL #3840489

Change 3840297 by Max.Chen

	Sequencer: Fix copy/paste crash for lights

	#jira UE-54084

Change 3840284 by Michael.Dupuis

	#jira UE-53053: Was having a mismatch between the remove reorder and the actual remove

Change 3840215 by Sorin.Gradinaru

	Unshelved from pending changelist '3812852':

	UE-53550 Level doesn't render on Lenovo 939
	UE-53592 Assertion right after rendering scene on Lenovo S939

	#jira UE-53550
	#jira UE-53592
	#4.19
	#Android

	UE-53550
	Removed force disabling texture2DLodEXT and textureCubeLodEXT on Mali-400 devices
	The problem was that the shader compiler complains about code lines before the #extension directives.
	Placeholder // end extensions in the original shader code - to be replaced with round() functions

	UE-53592
	Always use a new task for devices that have GIsThreadedRendering=false, even when the call is from the rendering thread

Change 3840048 by Michael.Dupuis

	#jira UE-52975: Was always performing the equivalent of an Add, so now we use the Transform during the duplicate

Change 3840005 by Richard.Wallis

	Clone of CL 3835252

	Fix for shader library not working with iterative cooking, also fixes native Metal library not working with iterative cook.  This works by saving a "backup" of the shader library file - this is reloaded only during iterative cook and adds back in shaders that are missing from the current cook.

	Fix for extracting/searching .metal files in different directories so debug tgz archiving wasn't working correctly and also support iterative cook.

	Includes first pass code review suggestions by Mark Sat and Dmitriy Dyomin.

	#jira UE-53815

Change 3839968 by Michael.Dupuis

	#jira UE-52289: When OnRegister is called on the component make sure our PerInstanceRenderData is up to date
	Prevent a possible crash if ClearInstanceSelection was called on a component with no PerInstanceRenderData existing

	##codereview jack.porter

Change 3839924 by Richard.Wallis

	Clone of CL 3838093

	Fix for rewind / seek bugs in AvfMediaPlayer.

	- Don't initialise and send audio buffers that have a duration of Zero from the audio tap.  This chokes the audio sink and adds overhead we don't need.
	- Don't faff around with current play rates during loop operation - normal seek doesn't do this so loop seek shouldn't either.
	- SetRate() should not required to be passed to media tracks - should be enough to do this on audio track select only.

	#jira UE-54019, UE-53027

Change 3839321 by andrew.porter

	QAGame: Adding missing bookmark to QA-Sequencer_Blending

	#jira UE-29618

Change 3839286 by Marcus.Wassmer

	Duplicate CL: 3823296
	#jira UE-52784

Change 3839229 by Brandon.Schaefer

	Fix audio clean up crash when exiting PIE

	#jira UE-54050

	#review-3839109 @Arciel.Rekman, @Aaron.McLeran

Change 3839223 by andrew.porter

	QAGame: Rebinding pointlight actor to sequences

	#jira UE-29618

Change 3839098 by andrew.porter

	QAGame: Fixing missing sequencer blending test content

	#jira UE-29618

Change 3838919 by Mike.Erwin

	glTF: binary format's BIN chunk is not necessarily right after the JSON chunk.

	Discussed this with glTF spec authors. See https://github.com/KhronosGroup/glTF/issues/1177

	Rearranged how we read "chunks" from the file since they are not fixed in number or order, besides JSON always being first.

	#jira UE-50695

Change 3838909 by Mike.Erwin

	glTF: base64 decoding of data buffers and images

	Data can be encoded inside the glTF JSON as a Base64 data URI. This CL addresses a known limitation of initial commit CL 3793018. I added this after because base64 is expected to be uncommon in the wild. Authoring software will typically use external BIN files (.gltf) or a BIN chunk (.glb) to store data.

	#jira UE-50695

Change 3838812 by Uriel.Doyon

	Integrated CL 3838576, 3838581 and 3831760 from DevRendering, fixing issues with texture streaming.

	#jira none

Change 3838773 by Lauren.Ridge

	Fixing material layer filters

	#jira UE-54064

Change 3838748 by Michael.Trepka

	Fixed EngineTest runtime warning caused by CL 3838626

	#jira UE-53893

Change 3838730 by Max.Chen

	Sequencer: Add notification when the blend type is changed.

	#jira UE-54046

Change 3838626 by Michael.Trepka

	Changed FMacMenu to store MultiBox and MenuEntryBox as weak pointers instead of shared pointers. This way we avoid a situation where FMacMenu would try to release them on the main Cocoa thread or where we'd try to execute a menu action for menu items that Slate considers released.

	#jira UE-53893

Change 3838392 by Arciel.Rekman

	Fix assert on a policy removal (UE-54042).

	- Applying Gil's safe fix which just sweeps the problem under the rug.

	#jira UE-54042

Change 3838162 by Arciel.Rekman

	Linux: fix crash due to lambda lifetime issues (UE-54040).

	- The lambda goes out of scope in FBufferVisualizationMenuCommands::CreateVisualizationCommands, crashing the editor if compiled with a recent clang (5.0+).

	#jira UE-54040

	(Edigrating 3819174 to Release-4.19)

Change 3838156 by Rolando.Caloca

	UE4.19 - Support for Vulkan devices that have no cached memory type

	#jira UE-54039

Change 3838096 by Brandon.Schaefer

	Set sound to unfocused volume multiplier if not focused

	#jira UE-51327

	#review-3835736 arciel.rekman

Change 3838087 by Brandon.Schaefer

	Fix arm server building. A fix from CL 3617084 remove inheriting from IHeadMountedDisplayVulkanExtensions. Remove overrides

	#jira UE-53901

	#review-3838088 arciel.rekman

Change 3837072 by Phillip.Kavan

	Fix for a build failure that could occur with Blueprint nativization enabled and EDL disabled. This was a regression introduced in 4.18.

	- Mirrored from //UE4/Dev-Framework (3836768).

	#jira UE-53908

Change 3837071 by Phillip.Kavan

	Emit proper syntax for set/map fields containing converted assets to generated C++ Blueprint class headers when Blueprint nativization is enabled.

	- Mirrored from //UE4/Dev-Framework (3835944, 3835965).

	#jira UE-42614

Change 3837070 by Phillip.Kavan

	#4202 -  Blueprint nativization bug fixes (PR).

	- Mirrored from //UE4/Dev-Framework (3830562, 3832292).

	#jira UE-52188

Change 3836507 by Ryan.Vance

	#jira UE-53992

	Due to hijacking the depth target directly from the scene context, we can't support depth compositing if it's being scaled by screen percentage since it wont match our color render target dimensions.

Change 3836390 by Dan.Oconnor

	Fix failure to resolve archetype when using the compilation manager

	#jira UE-53840

Change 3836251 by Ryan.Vance

	#jira UE-53992

Change 3835852 by Mark.Satterthwaite

	Fix tessellation shaders in Metal with Manual Vertex Fetch enabled:
	- The control points idnex buffer shouldn't collide with anything else.
	- We can't use the optimisation of loading texture width & height from the buffer meta-table in tessellation shaders as the combined stages don't guarantee not to clobber unused buffer slots and screw it up when we use linear textures.

	#jira UE-53851

Change 3835802 by JeanMichel.Dignard

	UBT changes for Enterprise deployment
	- Allow building enterprise against an installed engine version
	- Added enterprise to the cleanup process if its not installed

	#jira UEENT-748

Change 3835625 by Bogdan.Vasilache

	UE-50257 --> (Skeletal meshes silently fail to render if they have more than 75 bones) --> changed logged error with a warning
	#jira UE-50257

Change 3833649 by Mike.Beach

	Enabling debug layer when multiview is enabled.

	#jira UE-49954

Change 3833525 by Ethan.Geller

	Fix copyright in ActiveSound.cpp #jira none #rb none

Change 3642649 by Stewart.Lynch

	Renamed loctext key to stop it clashing with an existing entry

	#jira UE-49432

Change 3644762 by Stewart.Lynch

	LLM update: Added Total, Untracked and FMalloc Unused to LLMFULL stat page. Fixed occasionally missing allocs/frees. Removed platform csv.

	* removed CheckSize arg from OnLowLevelFree
	* show a warning in LLM Map when replacing existing value. This means that there has been an alloc/free mismatch.
	* minor optimization in LLMMap::GetMaxIndex - cache Mask value
	* added Total and Untracked stats to LLMFULL. LLMFULL now tracks almost all of the memory that LLMPlatform does, so there is no real need to use LLMPlatform. Removed the LLMPlatform csv.
	* added FMalloc Unused stat to LLMFULL to account for memory Binned2 has allocated internally. This can be used to track Binned2 fragmentation over time.
	* renamed Binned stats to FMemory to make it more general
	* added Default tracking to CustomVirtualAlloc and disable it where necessary. This catches the few VirtualAlloc calls that were missed.
	* added AllocType arg to all allocation tracking. This was needed in order to track the FMalloc total, and also to fix the pausing
	* fixed a bug in pausing where alloc/frees were being missed. Now only pauses a specific allocation type.
	* Trackers now maintain totals for each enum tag
	* tracking of Texture and mesh allocation on Windows D3D11 & D3D12

Change 3651334 by Joe.Barnes

	Fix misspelled function name.

	#jira 39441
	#3016

Change 3653857 by Ben.Woodhouse

	Integrate from //UE4/Main/...@3653675 to //UE4/Dev-Console/...

Change 3656553 by Joe.Barnes

	Add path for SCS_DeviceDepth.

Change 3662703 by Ben.Woodhouse

	Merging CL 3659069 from //Fortnite/Main/... to to //ue4/dev-console/...:

	[FORTNITE] [CONSOLE] [+] Improved frame syncing mechanism
	 - Improves input latency by allowing the game thread to sync to the swap chain flip of the previous frame.
	 - Added "r.GTSyncType" CVar to control how the game thread syncs with the rest of the pipe.
	 - r.GTSyncType 2 will sync the game thread with the flip of the swap chain, preventing the pipe from getting too long and causing excess input latency.

	Platforms are required to implement RHIWaitForFlip and RHISignalFlipEvent, and call RHIInitializeFlipTracking on RHI startup.
	A separate thread monitors the progress of frame flips and signals task graph events as they pass their corresponding frame index.
	In r.GTSyncType 2 mode, the game thread is signaled by this flip tracking thread.

	[~] Unified platform specific sync interval CVars into one: rhi.SyncInterval
	 - 1 == 60Hz
	 - 2 == 30Hz
	 - 3 == 20Hz

Change 3675239 by Keith.Judge

	Fixed thread priorities for Windows so that BelowNormal and SlightlyBelowNormal are the same. Bumped Lowest down a notch so they all fit.

	#jira UE-50626

Change 3676709 by Ben.Woodhouse

	Integrate from //UE4/Main/...@3675008 to //UE4/Dev-Console/...

Change 3689712 by Ben.Woodhouse

	Integrate from //UE4/Main/...@3687781 to //UE4/Dev-Console/...

Change 3701778 by Ben.Woodhouse

	Integrate from //UE4/Main/...@3699491 to //UE4/Dev-Console/...

Change 3677043 by Ben.Woodhouse

	From StewartL:
	I also found that on Windows Fortnite is overflowing int32 values due to the number of allocations. I changed LLMArray and LLMMap to use uint32 and it seems to be Ok now. I didn't want to check this in at the last minute, so here's the shelf: 3645336

Change 3726532 by Luke.Thatcher

	[CONSOLE] [~] Enable XGE shader compilation by default.
	 - Also set XGE mode to "force interception". With XGE on by default, we don't want people with an old Incredibuild version using the old system inadvertently.

Change 3726554 by Luke.Thatcher

	[CONSOLE] [+] Improved Scoped Named Events
	 - Added SCOPED_NAMED_EVENT(_FSTRING/_TEXT/_F) macros to allow use of string literals, FString and printf in scoped named event strings.
	 - Replaced explicit use of FPlatformMisc::Begin/EndNamedEvent throughout the engine with macros, so the events can be compiled out.
	 - Fixed performance issues on various platforms with named events. SCOPED_NAMED_EVENT macros were not considering whether the platform uses wide or ansi char strings.

Change 3751378 by Ben.Woodhouse

	Integrate from //UE4/Main/...@3748735 to //UE4/Dev-Console/...

Change 3751812 by Ben.Woodhouse

	Integrate from //UE4/Main/...@3750870 to //UE4/Dev-Console/...

Change 3728571 by Luke.Thatcher

	[CONSOLE] [!] Drop another XGE controller warning to log.

Change 3747150 by Joe.Barnes

	Add AuthoringToolHelper.bat file to installed build copy list.

Change 3768585 by Ben.Woodhouse

	Integrate from //UE4/Main/...@3767531 to //UE4/Dev-Console/...

Change 3772333 by Ben.Woodhouse

	Integrate from //UE4/Main/...@3771573 to //UE4/Dev-Console/...

Change 3786872 by Ben.Woodhouse

	Integrate from //UE4/Main/...@3786785 to //UE4/Dev-Console/...

Change 3787279 by Luke.Thatcher

	[CONSOLE] [~] Unified present threshold CVars.
	 - CVars are now in RHIUtilities.cpp, and are renamed to rhi.PresentThreshold.Top and rhi.PresentThreshold.Bottom.
	 - Platform implementations will be checked in shortly.

Change 3787445 by Luke.Thatcher

	[CONSOLE] [^] Merging (as edit) improvements to low-latency frame syncing from //Fortnite/Main to //UE4/Dev-Console

	Original CLs
	 - 3708949 - Added rhi.SyncSlackMS cvar to allow an offset of the game thread sync time by a number of milliseconds.
	 - 3712693 - Fix for crash on startup in new frame syncing.
	 - 3735765 - Fix r.GTSyncType logic when vsync is disabled (falls back to old behaviour when vsync is off).

Change 3788417 by Ben.Woodhouse

	Duplicate from FN CL 3712515
	CSV profiler GPU and pre-declared stat support
	- refactor the GPU profiler so it's no longer dependent on the stats system and can work in Test builds
	- add support for pre-declared CSV stats, using FNames (these are required for GPU stats)
	- add DECLARE_GPU_STAT macro which handles STATS and CsvProfiler declarations
	Note: still a few issues to resolve with GPU stats: these randomly go to 0 at times during a replay on XB1, the GPU total is lower than the stat unit number, and the unaccounted stat is too large due to missing stats

Change 3807818 by Ben.Woodhouse

	Integrate from //UE4/Main/...@3803271 to //UE4/Dev-Console/...

Change 3818577 by Ben.Woodhouse

	Integrate from //UE4/Main/...@3812936 to //UE4/Dev-Console/...

Change 3821198 by Ben.Woodhouse

	Integrate from //UE4/Release-4.19/...@3820127 to //UE4/Dev-Console/...

Change 3821519 by Ben.Woodhouse

	Integrate from //UE4/Release-4.19/...@3820753 to //UE4/Dev-Console/...
	#robomerge none

Change 3813513 by Ben.Woodhouse

	CSV profiler refactor+ bug fix
	- Move the CSV profiler to core, so we can use it in modules other than engine
	- Profiler no longer enqueues RT commands - this is handled by a new core delegate type
	- Make begin/end requests more robust, enqueued via a command queue and processed in order
	- Defer end capture requests by a frame. This ensures stats on threads other than the gamethread (e.g. renderthread etc) are complete, so the last frame is not truncated
	- Fix long-standing bug with multiple captures where old/stale frames would appear in the first <128 frames of captures
	- Move unit stats and dynamic resolution out of the profiler itself and into the engine. Only frametime is recorded in the profiler now
	- Fix longstanding bug in first frame renderthread time in the engine

Change 3814039 by Ben.Woodhouse

	More Csvprofiler improvements
	- Event support (via CSV_EVENT macros). These appear in the "EVENTS" column of the CSV and can be used to add context to the stat data
	- Reduced memory overhead for timestamps through bit-packing (now 16 bytes per marker instead of 24), and splitting stats into different types

Change 3814041 by Ben.Woodhouse

	Integrate as edit CL 3796390 from Fortnite/Main:

	Fix CsvProfiler not compiling in shipping for now

Change 3814229 by Ben.Woodhouse

	Integrate + refactor of CL 3792591 to reduce complexity and fix bugs

	Original changelist description:
	CSV profiler improvements:
	- The CSV profiler is now always compiled in on the server
	- The CSV profiler can now handle both int32 and float stats
	- In BeginCapture, the function can take additional arguments for some customization of filenames

	Fixes to the above:
	 - Remove FCustomValue class
		- Fixes bug where all timestamp values from CSV_SCOPED_STATs were garbage (due to issues resulting from FCustomValue type ambiguity when adding values together)
		- FCsvCustomStat now just uses simple union + bitfield flag to reduce size and complexity (4 bytes instead of 8 per value)
		- FCsvColumn class modified to use doubles, which can represent both ints and floats without loss of precision - this class is not memory or time critical
	 - Replace multiple overrides used by the server for filenameprefix/folder/postfix etc with a single FilenameOverride

Change 3814242 by Ben.Woodhouse

	Disable CSV unit stats on the dedicated server

Change 3817339 by Ben.Woodhouse

	Duplicate from 3816641: CSV profiler improvements
	- Added a low-pri processing thread to compress raw timing data into a much more efficient format
	   - Reduces memory usage for 15 minute 30k frame capture with GPU stats enabled from 110MB to 3.4MB
	   - Processing time : 0.1ms per frame
	- Improved name handling for char literal stats. We now use the string as the ID rather than the pointer so multiple stat uses of the same name string apply only to  a single stat
	- Reworked thread data access to eliminate locking for anything time-critical. Frame boundaries now handled via a lock-free helper class
	-  Fixed bug with queue implementation where 1 in 128 reads would duplicate the last block of stat data
	-  Reduced #include dependencies for CSVProfiler.h
	-  Removed AccumulateMax because it doesn't work, and implementing properly would add a lot of complexity
	-  Added a simple test harness

Change 3817582 by Ben.Woodhouse

	Fix android compile warning

Change 3823242 by Ben.Woodhouse

	Integrate as edit from Fortnite/Main 3820067:

	Fix threading issue with D3D12 pipeline state caches for windows. This was caused by the usage of FRwScopeLock::RaiseLockToWrite. A pointer read before RaiseLockToWrite was called was invalidated because RaiseLockToWrite has to release the read lock before acquiring the write lock.

	Rename FRwScopeLock::RaiseLockToWrite to ReleaseReadOnlyLockAndAcquireWriteLock_USE_WITH_CAUTION to make it more explicit what's happening. As the comment says:

		// This function should be used with caution.
		// It releases the read lock _before_ acquiring a new write lock. This is not an atomic operation and the caller should
		// not treat it as such.
		// E.g. Pointers read from protected data structures prior to this call may be invalid after the function is called.

Change 3823840 by Ben.Woodhouse

	Edigrate from 3823816
	Fix an issue where the csvprofile console commands would sometimes result in an empty CSV. The root cause was GFrameNumber incrementing between the console command being read and FCSVProfiler::EndFrame(). We now use our own frame counter which is updated in EndFrame, so this can't happen

	Also fix an issue where calling csvprofile stop twice would cause all further commands to be ignored.

Change 3827787 by Ben.Woodhouse

	Integrate-as-edit CL 3820678 from Fortnite/Main
	Allow the CSV Profiler to be compiled in to shipping dedicated server builds

Change 3827842 by Ben.Woodhouse

	Integrate-as-edit CL 3827079 from Fortnite/Main
	CSV profiler category support

Change 3827918 by Luke.Thatcher

	[CONSOLE] [!] Fix compile error in CSV custom stats.

Change 3827964 by Luke.Thatcher

	[CONSOLE] [!] Fix inverted logic and spelling of boolean.
	 - Functionally, the boolean did the correct thing, but the logic was backwards inside the build tool.

Change 3831661 by Ben.Woodhouse

	Integrate-as-edit CL 3830630 from Fortnite/Main
	Fix CSVProfiler assert in dev builds on XB1

Change 3860300 by Joe.Barnes

	Use same method for src and dest rect calculation as other post processing passes so rects match between passes. Prevents read of unprocessed pixels.

Change 3860347 by Joe.Barnes

	Delete existing SourceConfigFile before allocating a new one to prevent them leaking.

Change 3860348 by Joe.Barnes

	Completely encapsulate GetLLMAlloc() in #ifdef.

Change 3861772 by Ben.Woodhouse

	Integrate-as-edit CL 3861688 from Fortnite/Main: Forward lighting GPU crash fixes

Change 3861774 by Ben.Woodhouse

	Integrate as edit CL 3833918 from dev-rendering (courtesy of DanielW):
	D3D12 RHI: only refcount uniform buffers if GRHINeedsExtraDeletionLatency is false, which is no longer the case for PC. The refcounting was heavy on performance as reported by a licensee because FRHIResource uses atomics for refcounting, which is only necessary when GRHINeedsExtraDeletionLatency is disabled.

Change 3862214 by Ben.Woodhouse

	Integrate-as-edit CL 3859637 from Fortnite/Main
			Dynamic resolution console tweaks
			- Dynamic resolution high level switch driven by a cvar instead of code
			- Disable user settings dynamic resolution handling on non-desktop platforms. DynamicRes as a user setting does not make sense on consoles/mobile - we'll drive it from device profiles/scalability
			- Modify naming of GPUHeadRoom to make it clear that it's a percentage, not millseconds

Change 3863919 by Ben.Woodhouse

	[Copy] Integrate console dynamic resolution interface changes from CL 3863354, 3862754, 3862639

Change 3864347 by Ben.Woodhouse

	Fix the editor build. This will need a proper fix before 4.19 ships.

#lockdown Nick.Penwarden
#rb none

[CL 3913399 by Ben Marsh in Main branch]
2018-02-27 17:30:35 -05:00

1323 lines
46 KiB
Python

"""Thread module emulating a subset of Java's threading model."""
import sys as _sys
try:
import thread
except ImportError:
del _sys.modules[__name__]
raise
import warnings
from collections import deque as _deque
from itertools import count as _count
from time import time as _time, sleep as _sleep
from traceback import format_exc as _format_exc
# Note regarding PEP 8 compliant aliases
# This threading model was originally inspired by Java, and inherited
# the convention of camelCase function and method names from that
# language. While those names are not in any imminent danger of being
# deprecated, starting with Python 2.6, the module now provides a
# PEP 8 compliant alias for any such method name.
# Using the new PEP 8 compliant names also facilitates substitution
# with the multiprocessing module, which doesn't provide the old
# Java inspired names.
# Rename some stuff so "from threading import *" is safe
__all__ = ['activeCount', 'active_count', 'Condition', 'currentThread',
'current_thread', 'enumerate', 'Event',
'Lock', 'RLock', 'Semaphore', 'BoundedSemaphore', 'Thread',
'Timer', 'setprofile', 'settrace', 'local', 'stack_size']
_start_new_thread = thread.start_new_thread
_allocate_lock = thread.allocate_lock
_get_ident = thread.get_ident
ThreadError = thread.error
del thread
# sys.exc_clear is used to work around the fact that except blocks
# don't fully clear the exception until 3.0.
warnings.filterwarnings('ignore', category=DeprecationWarning,
module='threading', message='sys.exc_clear')
# Debug support (adapted from ihooks.py).
# All the major classes here derive from _Verbose. We force that to
# be a new-style class so that all the major classes here are new-style.
# This helps debugging (type(instance) is more revealing for instances
# of new-style classes).
_VERBOSE = False
if __debug__:
class _Verbose(object):
def __init__(self, verbose=None):
if verbose is None:
verbose = _VERBOSE
self.__verbose = verbose
def _note(self, format, *args):
if self.__verbose:
format = format % args
# Issue #4188: calling current_thread() can incur an infinite
# recursion if it has to create a DummyThread on the fly.
ident = _get_ident()
try:
name = _active[ident].name
except KeyError:
name = "<OS thread %d>" % ident
format = "%s: %s\n" % (name, format)
_sys.stderr.write(format)
else:
# Disable this when using "python -O"
class _Verbose(object):
def __init__(self, verbose=None):
pass
def _note(self, *args):
pass
# Support for profile and trace hooks
_profile_hook = None
_trace_hook = None
def setprofile(func):
"""Set a profile function for all threads started from the threading module.
The func will be passed to sys.setprofile() for each thread, before its
run() method is called.
"""
global _profile_hook
_profile_hook = func
def settrace(func):
"""Set a trace function for all threads started from the threading module.
The func will be passed to sys.settrace() for each thread, before its run()
method is called.
"""
global _trace_hook
_trace_hook = func
# Synchronization classes
Lock = _allocate_lock
def RLock(*args, **kwargs):
"""Factory function that returns a new reentrant lock.
A reentrant lock must be released by the thread that acquired it. Once a
thread has acquired a reentrant lock, the same thread may acquire it again
without blocking; the thread must release it once for each time it has
acquired it.
"""
return _RLock(*args, **kwargs)
class _RLock(_Verbose):
"""A reentrant lock must be released by the thread that acquired it. Once a
thread has acquired a reentrant lock, the same thread may acquire it
again without blocking; the thread must release it once for each time it
has acquired it.
"""
def __init__(self, verbose=None):
_Verbose.__init__(self, verbose)
self.__block = _allocate_lock()
self.__owner = None
self.__count = 0
def __repr__(self):
owner = self.__owner
try:
owner = _active[owner].name
except KeyError:
pass
return "<%s owner=%r count=%d>" % (
self.__class__.__name__, owner, self.__count)
def acquire(self, blocking=1):
"""Acquire a lock, blocking or non-blocking.
When invoked without arguments: if this thread already owns the lock,
increment the recursion level by one, and return immediately. Otherwise,
if another thread owns the lock, block until the lock is unlocked. Once
the lock is unlocked (not owned by any thread), then grab ownership, set
the recursion level to one, and return. If more than one thread is
blocked waiting until the lock is unlocked, only one at a time will be
able to grab ownership of the lock. There is no return value in this
case.
When invoked with the blocking argument set to true, do the same thing
as when called without arguments, and return true.
When invoked with the blocking argument set to false, do not block. If a
call without an argument would block, return false immediately;
otherwise, do the same thing as when called without arguments, and
return true.
"""
me = _get_ident()
if self.__owner == me:
self.__count = self.__count + 1
if __debug__:
self._note("%s.acquire(%s): recursive success", self, blocking)
return 1
rc = self.__block.acquire(blocking)
if rc:
self.__owner = me
self.__count = 1
if __debug__:
self._note("%s.acquire(%s): initial success", self, blocking)
else:
if __debug__:
self._note("%s.acquire(%s): failure", self, blocking)
return rc
__enter__ = acquire
def release(self):
"""Release a lock, decrementing the recursion level.
If after the decrement it is zero, reset the lock to unlocked (not owned
by any thread), and if any other threads are blocked waiting for the
lock to become unlocked, allow exactly one of them to proceed. If after
the decrement the recursion level is still nonzero, the lock remains
locked and owned by the calling thread.
Only call this method when the calling thread owns the lock. A
RuntimeError is raised if this method is called when the lock is
unlocked.
There is no return value.
"""
if self.__owner != _get_ident():
raise RuntimeError("cannot release un-acquired lock")
self.__count = count = self.__count - 1
if not count:
self.__owner = None
self.__block.release()
if __debug__:
self._note("%s.release(): final release", self)
else:
if __debug__:
self._note("%s.release(): non-final release", self)
def __exit__(self, t, v, tb):
self.release()
# Internal methods used by condition variables
def _acquire_restore(self, count_owner):
count, owner = count_owner
self.__block.acquire()
self.__count = count
self.__owner = owner
if __debug__:
self._note("%s._acquire_restore()", self)
def _release_save(self):
if __debug__:
self._note("%s._release_save()", self)
count = self.__count
self.__count = 0
owner = self.__owner
self.__owner = None
self.__block.release()
return (count, owner)
def _is_owned(self):
return self.__owner == _get_ident()
def Condition(*args, **kwargs):
"""Factory function that returns a new condition variable object.
A condition variable allows one or more threads to wait until they are
notified by another thread.
If the lock argument is given and not None, it must be a Lock or RLock
object, and it is used as the underlying lock. Otherwise, a new RLock object
is created and used as the underlying lock.
"""
return _Condition(*args, **kwargs)
class _Condition(_Verbose):
"""Condition variables allow one or more threads to wait until they are
notified by another thread.
"""
def __init__(self, lock=None, verbose=None):
_Verbose.__init__(self, verbose)
if lock is None:
lock = RLock()
self.__lock = lock
# Export the lock's acquire() and release() methods
self.acquire = lock.acquire
self.release = lock.release
# If the lock defines _release_save() and/or _acquire_restore(),
# these override the default implementations (which just call
# release() and acquire() on the lock). Ditto for _is_owned().
try:
self._release_save = lock._release_save
except AttributeError:
pass
try:
self._acquire_restore = lock._acquire_restore
except AttributeError:
pass
try:
self._is_owned = lock._is_owned
except AttributeError:
pass
self.__waiters = []
def __enter__(self):
return self.__lock.__enter__()
def __exit__(self, *args):
return self.__lock.__exit__(*args)
def __repr__(self):
return "<Condition(%s, %d)>" % (self.__lock, len(self.__waiters))
def _release_save(self):
self.__lock.release() # No state to save
def _acquire_restore(self, x):
self.__lock.acquire() # Ignore saved state
def _is_owned(self):
# Return True if lock is owned by current_thread.
# This method is called only if __lock doesn't have _is_owned().
if self.__lock.acquire(0):
self.__lock.release()
return False
else:
return True
def wait(self, timeout=None):
"""Wait until notified or until a timeout occurs.
If the calling thread has not acquired the lock when this method is
called, a RuntimeError is raised.
This method releases the underlying lock, and then blocks until it is
awakened by a notify() or notifyAll() call for the same condition
variable in another thread, or until the optional timeout occurs. Once
awakened or timed out, it re-acquires the lock and returns.
When the timeout argument is present and not None, it should be a
floating point number specifying a timeout for the operation in seconds
(or fractions thereof).
When the underlying lock is an RLock, it is not released using its
release() method, since this may not actually unlock the lock when it
was acquired multiple times recursively. Instead, an internal interface
of the RLock class is used, which really unlocks it even when it has
been recursively acquired several times. Another internal interface is
then used to restore the recursion level when the lock is reacquired.
"""
if not self._is_owned():
raise RuntimeError("cannot wait on un-acquired lock")
waiter = _allocate_lock()
waiter.acquire()
self.__waiters.append(waiter)
saved_state = self._release_save()
try: # restore state no matter what (e.g., KeyboardInterrupt)
if timeout is None:
waiter.acquire()
if __debug__:
self._note("%s.wait(): got it", self)
else:
# Balancing act: We can't afford a pure busy loop, so we
# have to sleep; but if we sleep the whole timeout time,
# we'll be unresponsive. The scheme here sleeps very
# little at first, longer as time goes on, but never longer
# than 20 times per second (or the timeout time remaining).
endtime = _time() + timeout
delay = 0.0005 # 500 us -> initial delay of 1 ms
while True:
gotit = waiter.acquire(0)
if gotit:
break
remaining = endtime - _time()
if remaining <= 0:
break
delay = min(delay * 2, remaining, .05)
_sleep(delay)
if not gotit:
if __debug__:
self._note("%s.wait(%s): timed out", self, timeout)
try:
self.__waiters.remove(waiter)
except ValueError:
pass
else:
if __debug__:
self._note("%s.wait(%s): got it", self, timeout)
finally:
self._acquire_restore(saved_state)
def notify(self, n=1):
"""Wake up one or more threads waiting on this condition, if any.
If the calling thread has not acquired the lock when this method is
called, a RuntimeError is raised.
This method wakes up at most n of the threads waiting for the condition
variable; it is a no-op if no threads are waiting.
"""
if not self._is_owned():
raise RuntimeError("cannot notify on un-acquired lock")
__waiters = self.__waiters
waiters = __waiters[:n]
if not waiters:
if __debug__:
self._note("%s.notify(): no waiters", self)
return
self._note("%s.notify(): notifying %d waiter%s", self, n,
n!=1 and "s" or "")
for waiter in waiters:
waiter.release()
try:
__waiters.remove(waiter)
except ValueError:
pass
def notifyAll(self):
"""Wake up all threads waiting on this condition.
If the calling thread has not acquired the lock when this method
is called, a RuntimeError is raised.
"""
self.notify(len(self.__waiters))
notify_all = notifyAll
def Semaphore(*args, **kwargs):
"""A factory function that returns a new semaphore.
Semaphores manage a counter representing the number of release() calls minus
the number of acquire() calls, plus an initial value. The acquire() method
blocks if necessary until it can return without making the counter
negative. If not given, value defaults to 1.
"""
return _Semaphore(*args, **kwargs)
class _Semaphore(_Verbose):
"""Semaphores manage a counter representing the number of release() calls
minus the number of acquire() calls, plus an initial value. The acquire()
method blocks if necessary until it can return without making the counter
negative. If not given, value defaults to 1.
"""
# After Tim Peters' semaphore class, but not quite the same (no maximum)
def __init__(self, value=1, verbose=None):
if value < 0:
raise ValueError("semaphore initial value must be >= 0")
_Verbose.__init__(self, verbose)
self.__cond = Condition(Lock())
self.__value = value
def acquire(self, blocking=1):
"""Acquire a semaphore, decrementing the internal counter by one.
When invoked without arguments: if the internal counter is larger than
zero on entry, decrement it by one and return immediately. If it is zero
on entry, block, waiting until some other thread has called release() to
make it larger than zero. This is done with proper interlocking so that
if multiple acquire() calls are blocked, release() will wake exactly one
of them up. The implementation may pick one at random, so the order in
which blocked threads are awakened should not be relied on. There is no
return value in this case.
When invoked with blocking set to true, do the same thing as when called
without arguments, and return true.
When invoked with blocking set to false, do not block. If a call without
an argument would block, return false immediately; otherwise, do the
same thing as when called without arguments, and return true.
"""
rc = False
with self.__cond:
while self.__value == 0:
if not blocking:
break
if __debug__:
self._note("%s.acquire(%s): blocked waiting, value=%s",
self, blocking, self.__value)
self.__cond.wait()
else:
self.__value = self.__value - 1
if __debug__:
self._note("%s.acquire: success, value=%s",
self, self.__value)
rc = True
return rc
__enter__ = acquire
def release(self):
"""Release a semaphore, incrementing the internal counter by one.
When the counter is zero on entry and another thread is waiting for it
to become larger than zero again, wake up that thread.
"""
with self.__cond:
self.__value = self.__value + 1
if __debug__:
self._note("%s.release: success, value=%s",
self, self.__value)
self.__cond.notify()
def __exit__(self, t, v, tb):
self.release()
def BoundedSemaphore(*args, **kwargs):
"""A factory function that returns a new bounded semaphore.
A bounded semaphore checks to make sure its current value doesn't exceed its
initial value. If it does, ValueError is raised. In most situations
semaphores are used to guard resources with limited capacity.
If the semaphore is released too many times it's a sign of a bug. If not
given, value defaults to 1.
Like regular semaphores, bounded semaphores manage a counter representing
the number of release() calls minus the number of acquire() calls, plus an
initial value. The acquire() method blocks if necessary until it can return
without making the counter negative. If not given, value defaults to 1.
"""
return _BoundedSemaphore(*args, **kwargs)
class _BoundedSemaphore(_Semaphore):
"""A bounded semaphore checks to make sure its current value doesn't exceed
its initial value. If it does, ValueError is raised. In most situations
semaphores are used to guard resources with limited capacity.
"""
def __init__(self, value=1, verbose=None):
_Semaphore.__init__(self, value, verbose)
self._initial_value = value
def release(self):
"""Release a semaphore, incrementing the internal counter by one.
When the counter is zero on entry and another thread is waiting for it
to become larger than zero again, wake up that thread.
If the number of releases exceeds the number of acquires,
raise a ValueError.
"""
with self._Semaphore__cond:
if self._Semaphore__value >= self._initial_value:
raise ValueError("Semaphore released too many times")
self._Semaphore__value += 1
self._Semaphore__cond.notify()
def Event(*args, **kwargs):
"""A factory function that returns a new event.
Events manage a flag that can be set to true with the set() method and reset
to false with the clear() method. The wait() method blocks until the flag is
true.
"""
return _Event(*args, **kwargs)
class _Event(_Verbose):
"""A factory function that returns a new event object. An event manages a
flag that can be set to true with the set() method and reset to false
with the clear() method. The wait() method blocks until the flag is true.
"""
# After Tim Peters' event class (without is_posted())
def __init__(self, verbose=None):
_Verbose.__init__(self, verbose)
self.__cond = Condition(Lock())
self.__flag = False
def _reset_internal_locks(self):
# private! called by Thread._reset_internal_locks by _after_fork()
self.__cond.__init__(Lock())
def isSet(self):
'Return true if and only if the internal flag is true.'
return self.__flag
is_set = isSet
def set(self):
"""Set the internal flag to true.
All threads waiting for the flag to become true are awakened. Threads
that call wait() once the flag is true will not block at all.
"""
with self.__cond:
self.__flag = True
self.__cond.notify_all()
def clear(self):
"""Reset the internal flag to false.
Subsequently, threads calling wait() will block until set() is called to
set the internal flag to true again.
"""
with self.__cond:
self.__flag = False
def wait(self, timeout=None):
"""Block until the internal flag is true.
If the internal flag is true on entry, return immediately. Otherwise,
block until another thread calls set() to set the flag to true, or until
the optional timeout occurs.
When the timeout argument is present and not None, it should be a
floating point number specifying a timeout for the operation in seconds
(or fractions thereof).
This method returns the internal flag on exit, so it will always return
True except if a timeout is given and the operation times out.
"""
with self.__cond:
if not self.__flag:
self.__cond.wait(timeout)
return self.__flag
# Helper to generate new thread names
_counter = _count().next
_counter() # Consume 0 so first non-main thread has id 1.
def _newname(template="Thread-%d"):
return template % _counter()
# Active thread administration
_active_limbo_lock = _allocate_lock()
_active = {} # maps thread id to Thread object
_limbo = {}
# Main class for threads
class Thread(_Verbose):
"""A class that represents a thread of control.
This class can be safely subclassed in a limited fashion.
"""
__initialized = False
# Need to store a reference to sys.exc_info for printing
# out exceptions when a thread tries to use a global var. during interp.
# shutdown and thus raises an exception about trying to perform some
# operation on/with a NoneType
__exc_info = _sys.exc_info
# Keep sys.exc_clear too to clear the exception just before
# allowing .join() to return.
__exc_clear = _sys.exc_clear
def __init__(self, group=None, target=None, name=None,
args=(), kwargs=None, verbose=None):
"""This constructor should always be called with keyword arguments. Arguments are:
*group* should be None; reserved for future extension when a ThreadGroup
class is implemented.
*target* is the callable object to be invoked by the run()
method. Defaults to None, meaning nothing is called.
*name* is the thread name. By default, a unique name is constructed of
the form "Thread-N" where N is a small decimal number.
*args* is the argument tuple for the target invocation. Defaults to ().
*kwargs* is a dictionary of keyword arguments for the target
invocation. Defaults to {}.
If a subclass overrides the constructor, it must make sure to invoke
the base class constructor (Thread.__init__()) before doing anything
else to the thread.
"""
assert group is None, "group argument must be None for now"
_Verbose.__init__(self, verbose)
if kwargs is None:
kwargs = {}
self.__target = target
self.__name = str(name or _newname())
self.__args = args
self.__kwargs = kwargs
self.__daemonic = self._set_daemon()
self.__ident = None
self.__started = Event()
self.__stopped = False
self.__block = Condition(Lock())
self.__initialized = True
# sys.stderr is not stored in the class like
# sys.exc_info since it can be changed between instances
self.__stderr = _sys.stderr
def _reset_internal_locks(self):
# private! Called by _after_fork() to reset our internal locks as
# they may be in an invalid state leading to a deadlock or crash.
if hasattr(self, '_Thread__block'): # DummyThread deletes self.__block
self.__block.__init__()
self.__started._reset_internal_locks()
@property
def _block(self):
# used by a unittest
return self.__block
def _set_daemon(self):
# Overridden in _MainThread and _DummyThread
return current_thread().daemon
def __repr__(self):
assert self.__initialized, "Thread.__init__() was not called"
status = "initial"
if self.__started.is_set():
status = "started"
if self.__stopped:
status = "stopped"
if self.__daemonic:
status += " daemon"
if self.__ident is not None:
status += " %s" % self.__ident
return "<%s(%s, %s)>" % (self.__class__.__name__, self.__name, status)
def start(self):
"""Start the thread's activity.
It must be called at most once per thread object. It arranges for the
object's run() method to be invoked in a separate thread of control.
This method will raise a RuntimeError if called more than once on the
same thread object.
"""
if not self.__initialized:
raise RuntimeError("thread.__init__() not called")
if self.__started.is_set():
raise RuntimeError("threads can only be started once")
if __debug__:
self._note("%s.start(): starting thread", self)
with _active_limbo_lock:
_limbo[self] = self
try:
_start_new_thread(self.__bootstrap, ())
except Exception:
with _active_limbo_lock:
del _limbo[self]
raise
self.__started.wait()
def run(self):
"""Method representing the thread's activity.
You may override this method in a subclass. The standard run() method
invokes the callable object passed to the object's constructor as the
target argument, if any, with sequential and keyword arguments taken
from the args and kwargs arguments, respectively.
"""
try:
if self.__target:
self.__target(*self.__args, **self.__kwargs)
finally:
# Avoid a refcycle if the thread is running a function with
# an argument that has a member that points to the thread.
del self.__target, self.__args, self.__kwargs
def __bootstrap(self):
# Wrapper around the real bootstrap code that ignores
# exceptions during interpreter cleanup. Those typically
# happen when a daemon thread wakes up at an unfortunate
# moment, finds the world around it destroyed, and raises some
# random exception *** while trying to report the exception in
# __bootstrap_inner() below ***. Those random exceptions
# don't help anybody, and they confuse users, so we suppress
# them. We suppress them only when it appears that the world
# indeed has already been destroyed, so that exceptions in
# __bootstrap_inner() during normal business hours are properly
# reported. Also, we only suppress them for daemonic threads;
# if a non-daemonic encounters this, something else is wrong.
try:
self.__bootstrap_inner()
except:
if self.__daemonic and _sys is None:
return
raise
def _set_ident(self):
self.__ident = _get_ident()
def __bootstrap_inner(self):
try:
self._set_ident()
self.__started.set()
with _active_limbo_lock:
_active[self.__ident] = self
del _limbo[self]
if __debug__:
self._note("%s.__bootstrap(): thread started", self)
if _trace_hook:
self._note("%s.__bootstrap(): registering trace hook", self)
_sys.settrace(_trace_hook)
if _profile_hook:
self._note("%s.__bootstrap(): registering profile hook", self)
_sys.setprofile(_profile_hook)
try:
self.run()
except SystemExit:
if __debug__:
self._note("%s.__bootstrap(): raised SystemExit", self)
except:
if __debug__:
self._note("%s.__bootstrap(): unhandled exception", self)
# If sys.stderr is no more (most likely from interpreter
# shutdown) use self.__stderr. Otherwise still use sys (as in
# _sys) in case sys.stderr was redefined since the creation of
# self.
if _sys and _sys.stderr is not None:
print>>_sys.stderr, ("Exception in thread %s:\n%s" %
(self.name, _format_exc()))
elif self.__stderr is not None:
# Do the best job possible w/o a huge amt. of code to
# approximate a traceback (code ideas from
# Lib/traceback.py)
exc_type, exc_value, exc_tb = self.__exc_info()
try:
print>>self.__stderr, (
"Exception in thread " + self.name +
" (most likely raised during interpreter shutdown):")
print>>self.__stderr, (
"Traceback (most recent call last):")
while exc_tb:
print>>self.__stderr, (
' File "%s", line %s, in %s' %
(exc_tb.tb_frame.f_code.co_filename,
exc_tb.tb_lineno,
exc_tb.tb_frame.f_code.co_name))
exc_tb = exc_tb.tb_next
print>>self.__stderr, ("%s: %s" % (exc_type, exc_value))
# Make sure that exc_tb gets deleted since it is a memory
# hog; deleting everything else is just for thoroughness
finally:
del exc_type, exc_value, exc_tb
else:
if __debug__:
self._note("%s.__bootstrap(): normal return", self)
finally:
# Prevent a race in
# test_threading.test_no_refcycle_through_target when
# the exception keeps the target alive past when we
# assert that it's dead.
self.__exc_clear()
finally:
with _active_limbo_lock:
self.__stop()
try:
# We don't call self.__delete() because it also
# grabs _active_limbo_lock.
del _active[_get_ident()]
except:
pass
def __stop(self):
# DummyThreads delete self.__block, but they have no waiters to
# notify anyway (join() is forbidden on them).
if not hasattr(self, '_Thread__block'):
return
self.__block.acquire()
self.__stopped = True
self.__block.notify_all()
self.__block.release()
def __delete(self):
"Remove current thread from the dict of currently running threads."
# Notes about running with dummy_thread:
#
# Must take care to not raise an exception if dummy_thread is being
# used (and thus this module is being used as an instance of
# dummy_threading). dummy_thread.get_ident() always returns -1 since
# there is only one thread if dummy_thread is being used. Thus
# len(_active) is always <= 1 here, and any Thread instance created
# overwrites the (if any) thread currently registered in _active.
#
# An instance of _MainThread is always created by 'threading'. This
# gets overwritten the instant an instance of Thread is created; both
# threads return -1 from dummy_thread.get_ident() and thus have the
# same key in the dict. So when the _MainThread instance created by
# 'threading' tries to clean itself up when atexit calls this method
# it gets a KeyError if another Thread instance was created.
#
# This all means that KeyError from trying to delete something from
# _active if dummy_threading is being used is a red herring. But
# since it isn't if dummy_threading is *not* being used then don't
# hide the exception.
try:
with _active_limbo_lock:
del _active[_get_ident()]
# There must not be any python code between the previous line
# and after the lock is released. Otherwise a tracing function
# could try to acquire the lock again in the same thread, (in
# current_thread()), and would block.
except KeyError:
if 'dummy_threading' not in _sys.modules:
raise
def join(self, timeout=None):
"""Wait until the thread terminates.
This blocks the calling thread until the thread whose join() method is
called terminates -- either normally or through an unhandled exception
or until the optional timeout occurs.
When the timeout argument is present and not None, it should be a
floating point number specifying a timeout for the operation in seconds
(or fractions thereof). As join() always returns None, you must call
isAlive() after join() to decide whether a timeout happened -- if the
thread is still alive, the join() call timed out.
When the timeout argument is not present or None, the operation will
block until the thread terminates.
A thread can be join()ed many times.
join() raises a RuntimeError if an attempt is made to join the current
thread as that would cause a deadlock. It is also an error to join() a
thread before it has been started and attempts to do so raises the same
exception.
"""
if not self.__initialized:
raise RuntimeError("Thread.__init__() not called")
if not self.__started.is_set():
raise RuntimeError("cannot join thread before it is started")
if self is current_thread():
raise RuntimeError("cannot join current thread")
if __debug__:
if not self.__stopped:
self._note("%s.join(): waiting until thread stops", self)
self.__block.acquire()
try:
if timeout is None:
while not self.__stopped:
self.__block.wait()
if __debug__:
self._note("%s.join(): thread stopped", self)
else:
deadline = _time() + timeout
while not self.__stopped:
delay = deadline - _time()
if delay <= 0:
if __debug__:
self._note("%s.join(): timed out", self)
break
self.__block.wait(delay)
else:
if __debug__:
self._note("%s.join(): thread stopped", self)
finally:
self.__block.release()
@property
def name(self):
"""A string used for identification purposes only.
It has no semantics. Multiple threads may be given the same name. The
initial name is set by the constructor.
"""
assert self.__initialized, "Thread.__init__() not called"
return self.__name
@name.setter
def name(self, name):
assert self.__initialized, "Thread.__init__() not called"
self.__name = str(name)
@property
def ident(self):
"""Thread identifier of this thread or None if it has not been started.
This is a nonzero integer. See the thread.get_ident() function. Thread
identifiers may be recycled when a thread exits and another thread is
created. The identifier is available even after the thread has exited.
"""
assert self.__initialized, "Thread.__init__() not called"
return self.__ident
def isAlive(self):
"""Return whether the thread is alive.
This method returns True just before the run() method starts until just
after the run() method terminates. The module function enumerate()
returns a list of all alive threads.
"""
assert self.__initialized, "Thread.__init__() not called"
return self.__started.is_set() and not self.__stopped
is_alive = isAlive
@property
def daemon(self):
"""A boolean value indicating whether this thread is a daemon thread (True) or not (False).
This must be set before start() is called, otherwise RuntimeError is
raised. Its initial value is inherited from the creating thread; the
main thread is not a daemon thread and therefore all threads created in
the main thread default to daemon = False.
The entire Python program exits when no alive non-daemon threads are
left.
"""
assert self.__initialized, "Thread.__init__() not called"
return self.__daemonic
@daemon.setter
def daemon(self, daemonic):
if not self.__initialized:
raise RuntimeError("Thread.__init__() not called")
if self.__started.is_set():
raise RuntimeError("cannot set daemon status of active thread");
self.__daemonic = daemonic
def isDaemon(self):
return self.daemon
def setDaemon(self, daemonic):
self.daemon = daemonic
def getName(self):
return self.name
def setName(self, name):
self.name = name
# The timer class was contributed by Itamar Shtull-Trauring
def Timer(*args, **kwargs):
"""Factory function to create a Timer object.
Timers call a function after a specified number of seconds:
t = Timer(30.0, f, args=[], kwargs={})
t.start()
t.cancel() # stop the timer's action if it's still waiting
"""
return _Timer(*args, **kwargs)
class _Timer(Thread):
"""Call a function after a specified number of seconds:
t = Timer(30.0, f, args=[], kwargs={})
t.start()
t.cancel() # stop the timer's action if it's still waiting
"""
def __init__(self, interval, function, args=[], kwargs={}):
Thread.__init__(self)
self.interval = interval
self.function = function
self.args = args
self.kwargs = kwargs
self.finished = Event()
def cancel(self):
"""Stop the timer if it hasn't finished yet"""
self.finished.set()
def run(self):
self.finished.wait(self.interval)
if not self.finished.is_set():
self.function(*self.args, **self.kwargs)
self.finished.set()
# Special thread class to represent the main thread
# This is garbage collected through an exit handler
class _MainThread(Thread):
def __init__(self):
Thread.__init__(self, name="MainThread")
self._Thread__started.set()
self._set_ident()
with _active_limbo_lock:
_active[_get_ident()] = self
def _set_daemon(self):
return False
def _exitfunc(self):
self._Thread__stop()
t = _pickSomeNonDaemonThread()
if t:
if __debug__:
self._note("%s: waiting for other threads", self)
while t:
t.join()
t = _pickSomeNonDaemonThread()
if __debug__:
self._note("%s: exiting", self)
self._Thread__delete()
def _pickSomeNonDaemonThread():
for t in enumerate():
if not t.daemon and t.is_alive():
return t
return None
# Dummy thread class to represent threads not started here.
# These aren't garbage collected when they die, nor can they be waited for.
# If they invoke anything in threading.py that calls current_thread(), they
# leave an entry in the _active dict forever after.
# Their purpose is to return *something* from current_thread().
# They are marked as daemon threads so we won't wait for them
# when we exit (conform previous semantics).
class _DummyThread(Thread):
def __init__(self):
Thread.__init__(self, name=_newname("Dummy-%d"))
# Thread.__block consumes an OS-level locking primitive, which
# can never be used by a _DummyThread. Since a _DummyThread
# instance is immortal, that's bad, so release this resource.
del self._Thread__block
self._Thread__started.set()
self._set_ident()
with _active_limbo_lock:
_active[_get_ident()] = self
def _set_daemon(self):
return True
def join(self, timeout=None):
assert False, "cannot join a dummy thread"
# Global API functions
def currentThread():
"""Return the current Thread object, corresponding to the caller's thread of control.
If the caller's thread of control was not created through the threading
module, a dummy thread object with limited functionality is returned.
"""
try:
return _active[_get_ident()]
except KeyError:
##print "current_thread(): no current thread for", _get_ident()
return _DummyThread()
current_thread = currentThread
def activeCount():
"""Return the number of Thread objects currently alive.
The returned count is equal to the length of the list returned by
enumerate().
"""
with _active_limbo_lock:
return len(_active) + len(_limbo)
active_count = activeCount
def _enumerate():
# Same as enumerate(), but without the lock. Internal use only.
return _active.values() + _limbo.values()
def enumerate():
"""Return a list of all Thread objects currently alive.
The list includes daemonic threads, dummy thread objects created by
current_thread(), and the main thread. It excludes terminated threads and
threads that have not yet been started.
"""
with _active_limbo_lock:
return _active.values() + _limbo.values()
from thread import stack_size
# Create the main thread object,
# and make it available for the interpreter
# (Py_Main) as threading._shutdown.
_shutdown = _MainThread()._exitfunc
# get thread-local implementation, either from the thread
# module, or from the python fallback
try:
from thread import _local as local
except ImportError:
from _threading_local import local
def _after_fork():
# This function is called by Python/ceval.c:PyEval_ReInitThreads which
# is called from PyOS_AfterFork. Here we cleanup threading module state
# that should not exist after a fork.
# Reset _active_limbo_lock, in case we forked while the lock was held
# by another (non-forked) thread. http://bugs.python.org/issue874900
global _active_limbo_lock
_active_limbo_lock = _allocate_lock()
# fork() only copied the current thread; clear references to others.
new_active = {}
current = current_thread()
with _active_limbo_lock:
for thread in _enumerate():
# Any lock/condition variable may be currently locked or in an
# invalid state, so we reinitialize them.
if hasattr(thread, '_reset_internal_locks'):
thread._reset_internal_locks()
if thread is current:
# There is only one active thread. We reset the ident to
# its new value since it can have changed.
ident = _get_ident()
thread._Thread__ident = ident
new_active[ident] = thread
else:
# All the others are already stopped.
thread._Thread__stop()
_limbo.clear()
_active.clear()
_active.update(new_active)
assert len(_active) == 1
# Self-test code
def _test():
class BoundedQueue(_Verbose):
def __init__(self, limit):
_Verbose.__init__(self)
self.mon = RLock()
self.rc = Condition(self.mon)
self.wc = Condition(self.mon)
self.limit = limit
self.queue = _deque()
def put(self, item):
self.mon.acquire()
while len(self.queue) >= self.limit:
self._note("put(%s): queue full", item)
self.wc.wait()
self.queue.append(item)
self._note("put(%s): appended, length now %d",
item, len(self.queue))
self.rc.notify()
self.mon.release()
def get(self):
self.mon.acquire()
while not self.queue:
self._note("get(): queue empty")
self.rc.wait()
item = self.queue.popleft()
self._note("get(): got %s, %d left", item, len(self.queue))
self.wc.notify()
self.mon.release()
return item
class ProducerThread(Thread):
def __init__(self, queue, quota):
Thread.__init__(self, name="Producer")
self.queue = queue
self.quota = quota
def run(self):
from random import random
counter = 0
while counter < self.quota:
counter = counter + 1
self.queue.put("%s.%d" % (self.name, counter))
_sleep(random() * 0.00001)
class ConsumerThread(Thread):
def __init__(self, queue, count):
Thread.__init__(self, name="Consumer")
self.queue = queue
self.count = count
def run(self):
while self.count > 0:
item = self.queue.get()
print item
self.count = self.count - 1
NP = 3
QL = 4
NI = 5
Q = BoundedQueue(QL)
P = []
for i in range(NP):
t = ProducerThread(Q, NI)
t.name = ("Producer-%d" % (i+1))
P.append(t)
C = ConsumerThread(Q, NI*NP)
for t in P:
t.start()
_sleep(0.000001)
C.start()
for t in P:
t.join()
C.join()
if __name__ == '__main__':
_test()