Commit Graph

29 Commits

Author SHA1 Message Date
Joe Kirchoff
18fe2bc53f ManagedProcess: Remove unnecessary locks from merged stream copy when creating background threads to merge output streams
#preflight 614225ba9bba9a0001980a37
#rb Ben.Marsh
#rnx

[CL 17525066 by Joe Kirchoff in ue5-main branch]
2021-09-15 15:22:22 -04:00
jonathan adamczewski
c2e6a3ce78 EpicGames.Core ManagedProcess:
On systems with multiple process groups, distribute launched processes explicitly between groups to improve overall utiliization.

#jira none

[CL 17392872 by jonathan adamczewski in ue5-main branch]
2021-09-01 17:21:43 -04:00
Ben Marsh
1cba718cc9 Partially revert CL 17374535. Closing stdout/stderr before joining with background threads means that we lose whatever is still in the pipe. Causing exceptions on GitSync service due to corrupt output.
#fyi Joe.Kirchoff

[CL 17385481 by Ben Marsh in ue5-main branch]
2021-09-01 08:53:05 -04:00
Joe Kirchoff
af35f4f4a1 ManagedProcess: Flush stdout & err on process exit when merging output streams (Mac & Linux)
#rb none
#rnx
#preflight 612e6d3479d62b00019882e5

[CL 17374535 by Joe Kirchoff in ue5-main branch]
2021-08-31 14:51:48 -04:00
Ben Marsh
8a1de16645 Horde: Fix leak of stream objects from ManagedProcess.
[CL 16960514 by Ben Marsh in ue5-main branch]
2021-07-26 16:03:44 -04:00
joe kirchoff
cb67a63224 UnrealBuildTool: Use process directly to get wall clock execution time
Process.StartTime raises an exception on non-windows platforms if the process has already exited, so cache it immediately after starting the FrameworkProcess
Process.TotalProcessorTime raises an exception on non-windows if the process has also exited, so don't do that

#rb none
#rnx

[CL 16863633 by joe kirchoff in ue5-main branch]
2021-07-15 12:25:31 -04:00
joe kirchoff
c096a7ee9e [Backout] - CL16863209
#fyi joe.kirchoff
Original CL Desc
-----------------------------------------------------------------
UnrealBuildTool: Use process directly to get wall clock execution time

Process.StartTime raises an exception on non-windows platforms if the process has already exited, so cache it immediately after starting the FrameworkProcess

#rb none
#rnx

[CL 16863370 by joe kirchoff in ue5-main branch]
2021-07-15 12:06:07 -04:00
joe kirchoff
546d9539ee UnrealBuildTool: Use process directly to get wall clock execution time
Process.StartTime raises an exception on non-windows platforms if the process has already exited, so cache it immediately after starting the FrameworkProcess

#rb none
#rnx

[CL 16863209 by joe kirchoff in ue5-main branch]
2021-07-15 11:53:11 -04:00
joe kirchoff
a602b76893 [Backout] - CL16856917
#fyi Joe.Kirchoff
Original CL Desc
-----------------------------------------------------------------
UnrealBuildTool: Use process directly to get wall clock execution time

#rb none
#rnx

[CL 16857196 by joe kirchoff in ue5-main branch]
2021-07-14 19:22:25 -04:00
Joe Kirchoff
90a0d3f9ce UnrealBuildTool: Use process directly to get wall clock execution time
#rb none
#rnx

[CL 16856917 by Joe Kirchoff in ue5-main branch]
2021-07-14 18:56:32 -04:00
danny couture
6f8451b821 Fix crash when jobs are not available (i.e. MacOS)
Only show processor time when available

#rnx
#rb Will.Damon

[CL 16836540 by danny couture in ue5-main branch]
2021-07-13 09:24:53 -04:00
danny couture
8a2853f1da Add cpu time spent per compilation unit in parallel and task executor
Add top 20 report when compilation ends for parallel and task executor
Add total cpu usage across all cores when compilation finishes for parallel and task executor
Use real processor time reported by OS when possible to account for internal usage of multiple threads (i.e. cl.exe codegen uses 8 threads)
Enabled by default for Epic developers to increase awareness on most problematic compilation times

#rnx
#rb Ben.Marsh, Joe.Kirchoff
#preflight 60ec463819a0060001dc5eb2

[CL 16835803 by danny couture in ue5-main branch]
2021-07-13 06:51:27 -04:00
Joe Kirchoff
b4b06d3119 EpicGames.Core: Ensure ManagedProcess threads start before Process.Exited callback runs, to handle cases where the spawned process completes extremely quickly
#rb trivial

[CL 16682085 by Joe Kirchoff in ue5-main branch]
2021-06-15 19:31:11 -04:00
Ben Marsh
0428c9e85b Join stdout/stderr threads before closing the buffering thread.
[CL 16669144 by Ben Marsh in ue5-main branch]
2021-06-14 23:23:34 -04:00
Ben Marsh
ed9295fb88 Swallow exceptions when copying output from stdout if a process is killed.
[CL 16668977 by Ben Marsh in ue5-main branch]
2021-06-14 22:58:32 -04:00
Ben Marsh
62a32a650b Manually copy stdout/stderr from managed framework processes asynchronously, in order to prevent garbled output when converting to/from strings.
[CL 16662357 by Ben Marsh in ue5-main branch]
2021-06-14 15:06:17 -04:00
joe kirchoff
b1dbb8de60 UnrealBuildTool: Remove LocalExecutor and use ParallelExecutor for all host platforms
Update ManagedProcess to capture stderr if using C# Process objects

#jira UE-117670
#rb Ben.Marsh

[CL 16647451 by joe kirchoff in ue5-main branch]
2021-06-11 17:19:03 -04:00
joe kirchoff
adb0960f09 [Backout] - CL16642253
#fyi Joe.Kirchoff
Original CL Desc
-----------------------------------------------------------------
UnrealBuildTool: Remove LocalExecutor and use ParallelExecutor for all host platforms
Update ManagedProcess to capture stderr if using C# Process objects

#jira UE-117670
#rb Ben.Marsh
#preflight 60c2a1d3730f8a000185ae66
#preflight 60c2a5bab1a4a300012ea45d

[CL 16642682 by joe kirchoff in ue5-main branch]
2021-06-11 12:13:57 -04:00
Joe Kirchoff
96db72f088 UnrealBuildTool: Remove LocalExecutor and use ParallelExecutor for all host platforms
Update ManagedProcess to capture stderr if using C# Process objects

#jira UE-117670
#rb Ben.Marsh
#preflight 60c2a1d3730f8a000185ae66
#preflight 60c2a5bab1a4a300012ea45d

[CL 16642253 by Joe Kirchoff in ue5-main branch]
2021-06-11 11:40:36 -04:00
jonathan adamczewski
0af1ea99af Add a common C# method for detecting the current host platform to EpicGames.Core, and move many a test to use that code.
#jira none
#rb joe.kirchoff

[CL 16593582 by jonathan adamczewski in ue5-main branch]
2021-06-08 17:01:23 -04:00
Ben Marsh
d2f7a4d9e6 Horde: Automatically spawn a bundled copy of MongoDB if available. Data is stored in C:\ProgramData\Horde\Mongo by default.
[CL 16552409 by Ben Marsh in ue5-main branch]
2021-06-03 16:16:09 -04:00
Ben Marsh
f5b02c809e Fix ManagedProcess not correctly saving stdout/stderr handles on non-Windows platforms.
[CL 16247626 by Ben Marsh in ue5-main branch]
2021-05-10 09:09:21 -04:00
jonathan adamczewski
1f82bedc58 Removal some references to Mono
#trivial
#jira none

[CL 16171278 by jonathan adamczewski in ue5-main branch]
2021-04-30 14:34:17 -04:00
Ben Marsh
ab6a1064c1 Add new CopyToAsync() overloads.
[CL 16152979 by Ben Marsh in ue5-main branch]
2021-04-28 23:07:00 -04:00
Ben Marsh
3e5c8c98a5 Close stdin after writing input data to child processes. Perforce waits for the handle to be closed before processing the data.
#rb none
#rnx

[CL 15325165 by Ben Marsh in ue5-main branch]
2021-02-04 21:44:06 -04:00