Commit Graph

116 Commits

Author SHA1 Message Date
Ben Marsh
6b013e9a18 Horde: Fix superfluous length field being written out with object data.
[CL 16888502 by Ben Marsh in ue5-main branch]
2021-07-19 16:32:24 -04:00
Ben Marsh
bd841f5f2e Horde: Additional logging when stats file disappears.
[CL 16885085 by Ben Marsh in ue5-main branch]
2021-07-19 12:55:40 -04:00
Ben Marsh
802f7dbd51 Missing copyright notice.
[CL 16883799 by Ben Marsh in ue5-main branch]
2021-07-19 11:13:00 -04:00
Ben Marsh
43761d0be1 Horde: Separate implementation of stream snapshots from tree objects.
[CL 16883430 by Ben Marsh in ue5-main branch]
2021-07-19 10:34:55 -04:00
Ben Marsh
07cd00e3e8 Add Slice() extension method for List<T>, allowing use of ranges.
[CL 16883390 by Ben Marsh in ue5-main branch]
2021-07-19 10:30:48 -04:00
Ben Marsh
3e931e256f Horde: Reduce peak memory usage when building workspace snapshots. Was previously keeping references to Memory<byte> ranges within a much larger buffer, preventing them from being GC'd.
[CL 16867148 by Ben Marsh in ue5-main branch]
2021-07-15 16:56:05 -04:00
joe kirchoff
cb67a63224 UnrealBuildTool: Use process directly to get wall clock execution time
Process.StartTime raises an exception on non-windows platforms if the process has already exited, so cache it immediately after starting the FrameworkProcess
Process.TotalProcessorTime raises an exception on non-windows if the process has also exited, so don't do that

#rb none
#rnx

[CL 16863633 by joe kirchoff in ue5-main branch]
2021-07-15 12:25:31 -04:00
joe kirchoff
c096a7ee9e [Backout] - CL16863209
#fyi joe.kirchoff
Original CL Desc
-----------------------------------------------------------------
UnrealBuildTool: Use process directly to get wall clock execution time

Process.StartTime raises an exception on non-windows platforms if the process has already exited, so cache it immediately after starting the FrameworkProcess

#rb none
#rnx

[CL 16863370 by joe kirchoff in ue5-main branch]
2021-07-15 12:06:07 -04:00
Ben Marsh
c0b9519921 Horde: Output additional debug info when StatsV2 file is not found.
[CL 16863233 by Ben Marsh in ue5-main branch]
2021-07-15 11:55:01 -04:00
joe kirchoff
546d9539ee UnrealBuildTool: Use process directly to get wall clock execution time
Process.StartTime raises an exception on non-windows platforms if the process has already exited, so cache it immediately after starting the FrameworkProcess

#rb none
#rnx

[CL 16863209 by joe kirchoff in ue5-main branch]
2021-07-15 11:53:11 -04:00
joe kirchoff
a602b76893 [Backout] - CL16856917
#fyi Joe.Kirchoff
Original CL Desc
-----------------------------------------------------------------
UnrealBuildTool: Use process directly to get wall clock execution time

#rb none
#rnx

[CL 16857196 by joe kirchoff in ue5-main branch]
2021-07-14 19:22:25 -04:00
Joe Kirchoff
90a0d3f9ce UnrealBuildTool: Use process directly to get wall clock execution time
#rb none
#rnx

[CL 16856917 by Joe Kirchoff in ue5-main branch]
2021-07-14 18:56:32 -04:00
Ben Marsh
9f10c7c335 Fix AsBinary() returning the incorrect payload buffer for binary fields. Was causing Horde agents to deserialize incorrect hash values.
[CL 16852536 by Ben Marsh in ue5-main branch]
2021-07-14 13:55:32 -04:00
aurel cordonnier
ab8e255930 P4 Blame tool for P4V (requires input of the line number)
#rb marc.audy, ben.marsh
#jira none

[CL 16852242 by aurel cordonnier in ue5-main branch]
2021-07-14 13:23:58 -04:00
danny couture
6f8451b821 Fix crash when jobs are not available (i.e. MacOS)
Only show processor time when available

#rnx
#rb Will.Damon

[CL 16836540 by danny couture in ue5-main branch]
2021-07-13 09:24:53 -04:00
danny couture
8a2853f1da Add cpu time spent per compilation unit in parallel and task executor
Add top 20 report when compilation ends for parallel and task executor
Add total cpu usage across all cores when compilation finishes for parallel and task executor
Use real processor time reported by OS when possible to account for internal usage of multiple threads (i.e. cl.exe codegen uses 8 threads)
Enabled by default for Epic developers to increase awareness on most problematic compilation times

#rnx
#rb Ben.Marsh, Joe.Kirchoff
#preflight 60ec463819a0060001dc5eb2

[CL 16835803 by danny couture in ue5-main branch]
2021-07-13 06:51:27 -04:00
Ben Marsh
e64a5aff50 Missing copyright notice.
[CL 16817994 by Ben Marsh in ue5-main branch]
2021-07-09 09:50:24 -04:00
Ben Marsh
1e124c6a43 Missing file.
[CL 16817970 by Ben Marsh in ue5-main branch]
2021-07-09 09:28:59 -04:00
Ben Marsh
66c692643d Add missing Md5Hash file.
[CL 16817966 by Ben Marsh in ue5-main branch]
2021-07-09 09:26:25 -04:00
Ben Marsh
b3fb0f0503 Horde: Write managed workspace snapshots using Compact Binary.
[CL 16811525 by Ben Marsh in ue5-main branch]
2021-07-06 11:00:26 -04:00
Ben Marsh
a855dca4ea Rename ReadOnlyUtf8String to Utf8String. Strings are always immutable in C#.
[CL 16807051 by Ben Marsh in ue5-main branch]
2021-07-02 12:16:55 -04:00
Ben Marsh
abb6871d5e Fix buffer overrun in compact binary writer.
[CL 16807025 by Ben Marsh in ue5-main branch]
2021-07-02 12:08:03 -04:00
Ben Marsh
7e3b00497a Horde: Various compact binary library improvements.
CbObject/CbBinary/CbArray:
* Added proxy classes for viewing objects in the debugger. CbArray now displays as an array, CbObject displays as an array of fields, etc...
* Removed overloaded "View" methods, which are not required in C# due to GC.

CbWriter:
* Significantly fewer memory allocations/copying. Output buffer is now allocated in large chunks, with length fields only inserted when copying to the final output buffer.
* Methods for writing named/unnamed fields are now more idomatic with other C# serialization classes (ie. WriteXXX(Name, Value) and WriteXXXValue(Value)).

Other:
* Added serialization tests to Horde projects and fixed NuGet dependencies.

[CL 16804192 by Ben Marsh in ue5-main branch]
2021-07-01 09:12:10 -04:00
Ben Marsh
f48053c6f1 Missing file for Blake3 hashes.
[CL 16781033 by Ben Marsh in ue5-main branch]
2021-06-24 22:00:56 -04:00
Ben Marsh
c02be00304 Horde: Basic implementation of storage API.
Current limitations:

* Only supports a filesystem backend
* No garbage collection
* Refs are stored in the database, and do not automatically expire.
* Validation of refs is slow and does not cache any data.

[CL 16778495 by Ben Marsh in ue5-main branch]
2021-06-24 16:57:21 -04:00