Commit Graph

173 Commits

Author SHA1 Message Date
jonathan adamczewski
c3dea22f71 UnrealBuildTool, AutomationTool:
Print path to log file, print a timestamp at the top of each log file, don't print (wrong) callstack location.

#jira none
#rb ben.marsh

[CL 17351052 by jonathan adamczewski in ue5-main branch]
2021-08-30 08:44:12 -04:00
Ben Marsh
9b91e88f28 Ignore missing EpicGames.Perforce.Native.pdb if it does not exist. Fix CIS in Dev-EngineMerge.
[CL 17348211 by Ben Marsh in ue5-main branch]
2021-08-29 20:31:53 -04:00
Ben Marsh
390c78654c Add a native implementation of IPerforceConnection, which uses the Perforce C++ API (via a DLL called EpicGames.Perforce.Native) to output records to a buffer shared with managed code (which is double-buffered with the processing thread in managed code). Interaction with the API is lightweight and done on a background thread, with an async API overlaid on top. Prebuilt binaries for the native component are currently tested on Windows and Linux.
Performance has been tested against the P4 .NET API and the command line client by performing a fstat of non-deleted files in //UE5/Main.

EpicGames.Perforce (this library): 47s
Command line client piping to file: 59s
P4 .NET API: ~40 minutes

[CL 17347502 by Ben Marsh in ue5-main branch]
2021-08-29 16:35:57 -04:00
Ben Marsh
d0a856c8b7 P4: Add an interface for Perforce responses, and move the record deserialization logic into extension methods on it.
[CL 17335637 by Ben Marsh in ue5-main branch]
2021-08-27 12:19:41 -04:00
Ben Marsh
a05d88f335 Quality of life improvements for EpicGames.Perforce.
* Introduce a new FileSpecList class which supports implicit conversion from a single string and arrays/lists of strings.
* Make the cancellation token argument to all commands optional.

[CL 17335316 by Ben Marsh in ue5-main branch]
2021-08-27 11:43:35 -04:00
Ben Marsh
d98420677a P4: Specify command arguments as a list rather than a concatenated string, for future compatibility with C++ library.
[CL 17335163 by Ben Marsh in ue5-main branch]
2021-08-27 11:21:50 -04:00
Ben Marsh
24b7152d41 Introduce an interface for Perforce connections in C#.
[CL 17334845 by Ben Marsh in ue5-main branch]
2021-08-27 10:23:34 -04:00
Ben Marsh
24076d2685 Fix xmldoc markup for EpicGames.Perforce.
[CL 17334551 by Ben Marsh in ue5-main branch]
2021-08-27 09:21:43 -04:00
Ben Marsh
db9e56c5ca Move functionality on PerforceConnection to extension methods, so we can have different implementations of the core connection handler.
[CL 17334540 by Ben Marsh in ue5-main branch]
2021-08-27 09:19:56 -04:00
Ben Marsh
cd1c736721 Horde: Fix arguments when logging p4 protects commands.
[CL 17283029 by Ben Marsh in ue5-main branch]
2021-08-24 05:44:01 -04:00
Ben Marsh
611f190f96 Horde: Remove size from compute directory objects.
[CL 17250950 by Ben Marsh in ue5-main branch]
2021-08-20 14:41:57 -04:00
Ben Marsh
564ecec2bb Horde: Add support for serializing requests and responses as compact binary objects.
[CL 17250722 by Ben Marsh in ue5-main branch]
2021-08-20 14:28:49 -04:00
Ben Marsh
ede1eb74e8 Additional utility methods for Utf8String.
[CL 17236037 by Ben Marsh in ue5-main branch]
2021-08-19 12:55:24 -04:00
Ben Marsh
ef5988a226 Horde: Include file and directory sizes in DirectoryTree objects, as well as file attributes.
#fyi Joe.Kirchoff

[CL 17235171 by Ben Marsh in ue5-main branch]
2021-08-19 12:03:00 -04:00
Ben Marsh
706cfc2624 Horde: Add REST API for compute service.
* New tasks can be queued with a POST to /compute/{channelid}
* Status updates can be dequeued with a POST to /compute/{channelid}/updates
* ChannelId can be any series of lowercase letters, numbers, and the characters _ and -.
* Request/response objects are declared in Api.cs.

#fyi Joe.Kirchoff

[CL 17226912 by Ben Marsh in ue5-main branch]
2021-08-18 21:40:02 -04:00
Josh Engebretson
038626592b Horde: Fix missing copyright
#jira none
#rnx
#fyi ben.marsh

[CL 17197748 by Josh Engebretson in ue5-main branch]
2021-08-17 12:06:01 -04:00
Ben Marsh
8a2122b51c Horde: New API for remote execution.
Now allows for a stateless API and supports work-stealing between pods (via a shared task list stored in Redis). Clients now send requests for work to be executed with messages returned through a particular channel, which can be polled for outputs.

ExecuteV2 command in HordeAgent allows exercising the new code via the command line. Now mirrors the entire input directory by default, and only takes a JSON file to describe the command to perform.

[CL 17171093 by Ben Marsh in ue5-main branch]
2021-08-14 17:55:46 -04:00
Ben Marsh
75a96a6ac6 Allow serializing structs to compact binary objects using reflection, and automatically instantiate converter types with the same generic arguments as the tagged class if necessary.
[CL 17171079 by Ben Marsh in ue5-main branch]
2021-08-14 17:47:38 -04:00
Ben Marsh
c34957e4ef Allow implicit conversion from HashSet<T> to ReadOnlyHashSet<T>.
[CL 17171063 by Ben Marsh in ue5-main branch]
2021-08-14 17:45:59 -04:00
Ben Marsh
7146bb7f90 Horde: Add additional functionality to Redis library, including support for hashes.
[CL 17171062 by Ben Marsh in ue5-main branch]
2021-08-14 17:45:23 -04:00
Ben Marsh
48b62d47f6 Horde: Add a typed abstraction for Redis pub/sub channels.
[CL 17162181 by Ben Marsh in ue5-main branch]
2021-08-12 21:02:12 -04:00
Ben Marsh
36646e683f Add support for serializing int, DateTime, Nullable, and enum types to compact binary through reflection.
Also fix deserialized DateTime values being treated as local times rather than UTC.

[CL 17161724 by Ben Marsh in ue5-main branch]
2021-08-12 20:15:32 -04:00
Ben Marsh
16badae74a Fix ListSegment excluding the first item in the collection.
[CL 17161703 by Ben Marsh in ue5-main branch]
2021-08-12 20:12:32 -04:00
Ben Marsh
7d78e8e295 Horde: Add a library for accessing typed values in Redis.
[CL 17161674 by Ben Marsh in ue5-main branch]
2021-08-12 20:07:50 -04:00
jonathan adamczewski
4680079d8b update diagnostic text
#jira none
#rb none
#trivial

[CL 17159330 by jonathan adamczewski in ue5-main branch]
2021-08-12 17:33:54 -04:00