241 Commits

Author SHA1 Message Date
robert millar
094ab3eddf Optimize FPerforceGetPendingChangelistsWorker when getting files for all changelists by executing a single 'opened' command and parsing the results.
Speeds up query from 20+s to around 1.5s in an example workspace with 54 changelists.

#rb brooke.hubert

[CL 26294240 by robert millar in 5.3 branch]
2023-06-28 14:52:20 -04:00
paul chipchase
d6387b344c Don't save empty perforce setting keys to the config file, remove them instead to make the file more readable.
#rb trivial
#rnx

[CL 25973570 by paul chipchase in ue5-main branch]
2023-06-14 06:09:28 -04:00
paul chipchase
d8503fff99 Running FGetFile while the perforce source control provider is active will no longer incorrectly mark the connection as dropped, preventing future commands from being run.
#rb Wouter.Burgers
#rnx

### Bug Fix
- In CL 24941987 FPerforceGetFileWorker::Execute was modified to remove the perforce connection being created as that connection was not used and FPerforceSourceControlRevision::Get would create a second connection to be used.
- Removing this connection has an unintended knock on effect in that the command passed to the worker was never marked as having a successful server connection. So next tick the source control provider would check the now completed command, see that the connection was not a success (even though the command returned a success) and mark the connection to the server as broken, preventing future commands from working.
- Over the last year or so we have tried to move the code away from these sort of non-obvious side effects but there is a lot of work to do still.
- For now it is safest to add the connection back to FPerforceGetFileWorker::Execute but also add an overload to FPerforceSourceControlRevision::Get that allows us to pass in the connection to be used, to avoid the original problem of creating two connections as connection creation is super slow and can be costly on the server.

### Additional Changes
- There was no reason to allocate the FPerforceSourceControlRevision in FPerforceGetFileWorker::Execute on the heap as it is used immediately then deleted. We might as well just create it on the stack.
- Changed some code logic to early out rather than having long conditional scopes.
- The input parameters for FGetFile are still a bit odd (no revision == #0, revision -1 == #head etc) but leaving that alone for now until we merge FGetFile and FDownloadFiles to eliminate the duplicate code functionality.

[CL 25867524 by paul chipchase in ue5-main branch]
2023-06-08 06:56:20 -04:00
logan buchy
97474aeb9c Algorithmic optimization in PerforceRevertWorker
* Especially slow when large numbers of files are checked out (in the 25k+ range)
* Existing algorithm scales cubically
* Optimization: Changed to take advantage of the fact that filenames are already in a map, so a cheaper lookup to check if file in changelist is part of the to-be-deleted set
* Benchmark for this change is a map with ~32k empty actors in it.  Source control enabled and using world partition (and thus One File Per Actor).  It is important to check out all __ExternalActors__ files into default changelist and have the "View changes" window open.
- Delete N actors
- Save the level and measure time

Before:
100 actors : ~20s
200 actors : ~1m 20s
300 actors : ~2m 40s
500 actors : ~7m 20s

After:
100 actors : ~6s
200 actors : ~14s
500 actors : ~25s

There is plenty of room for more optimization here.

#jira UE-187093
#rb Brooke.Hubert
#preflight none

[CL 25652227 by logan buchy in ue5-main branch]
2023-05-26 16:28:22 -04:00
kirill zorin
b6ee3a6c64 Fix UE_LOG callsites that have format string-related UB
#rb trivial
#preflight 647107780a6634dbb236a0a6

[CL 25651638 by kirill zorin in ue5-main branch]
2023-05-26 15:52:39 -04:00
steve robb
ba1fe48aca Preparation for deprecating the hashing arrays and strings by pointer - now you have to be explicit and call the right function for hashing a pointer, hashing an array contents or hashing a TCHAR* pointer as a null-terminated string.
Added GetArrayHash() for hashing arrays by their contents.
Added a new pair of keyfuncs for TSets and TMaps which allow TCHAR* keys (or TCHAR* values passed to *ByHash functions) to be hashed by FCrc::Strihash_DEPRECATED().

#rb devin.doucette
#preflight 64708e7c296b2b37c6fd894a

[CL 25647216 by steve robb in ue5-main branch]
2023-05-26 12:51:39 -04:00
logan buchy
f633a49fdb Precautionary undo changelist 25604555
#jira UE-186978
#rb none

[CL 25644501 by logan buchy in ue5-main branch]
2023-05-26 12:06:42 -04:00
logan buchy
c8a4da1b00 Algorithmic optimization in PerforceRevertWorker
* Especially slow when large numbers of files are checked out (in the 25k+ range)
* Existing algorithm scales cubically
* Optimization #1: Changed to take advantage of the fact that filenames are already in a map, so a cheaper lookup to check if file in changelist is part of the to-be-deleted set
* Optimization #2: Another optimization takes advantage of the fact that it is typical the changelist to update is often the one in the to be deleted set.  This is not going to be true for all workflows but helpful anyways.
* Benchmark shows deleting 100 actors from a level with 25k one file per actors checked out in default changelist takes the time to complete the operation from 12s to ~200us. Without optimization #2, this takes about 2s to complete.

#rb Brooke.Hubert
#preflight none

[CL 25604910 by logan buchy in ue5-main branch]
2023-05-24 11:54:08 -04:00
Steve Robb
be221ad4a0 Fixed mismatched copy constructor/assignment operator for FPerforceSourceControlChangelist.
#rb none
#jira none
#preflight none

[CL 25579963 by Steve Robb in ue5-main branch]
2023-05-23 07:59:09 -04:00
jordan hoffmann
2b08e650c7 [CDPR] Added more information about changelist to source control changelists window context menu
+ Expand source control menu context with selected changelist,
+ Expose information about if changelist is default and its source control specific identifier,

#rb Patrick.Enfedaque, Patrick.Laflamme, zach.rammell
#preflight skip
#p4v-cherrypick 25526575
#preflight 64665919b91ab13b09c2089c

[CL 25528171 by jordan hoffmann in ue5-main branch]
2023-05-18 13:20:35 -04:00
bryan sefcik
da92084a12 Optimized out more private modules includes and dependencies.
#preflight 64627c382965f6ea8ea83bd6

[CL 25479683 by bryan sefcik in ue5-main branch]
2023-05-15 16:26:12 -04:00
jordan hoffmann
1625c5cc66 [BugFix] Deprecated ISourceControlState::GetBaseRevForMerge and replaced it with ISourceControlState::GetResolveInfo so we can properly support merges and integrations between branches and merging changes that aren't the latest
#rb brooke.hubert, mattias.jansson, wouter.burgers
#preflight 64556c1ad77a82d6a78e3388

[CL 25365368 by jordan hoffmann in ue5-main branch]
2023-05-06 02:07:43 -04:00
paul chipchase
a55296a3b3 Improve the error message given via FPerforceDownloadFileWorker if a depot for one of the requested files cannot be found.
#rb none
#jira UE-183529
#rnx
#preflight 643fc10ba35280ed4f6a5590

- If p4 print is run on a path for a depot that either does not exist on the server or cannot be seen due to permissions then the server will return "-must refer to client" along with the name of the current client spec. This can be quite misleading since p4 print does not actually need a client.
-- To improve things for the user we can replace the error message provided by the p4 server with "no such depot"

[CL 25103494 by paul chipchase in ue5-main branch]
2023-04-19 06:43:35 -04:00
paul chipchase
88a5010391 Clean up of FDownloadFile
#rb none
#jira UE-183529
#rnx
#preflight 643e6671b91c130758304d30

- Added code documentation
- Renamed ::AddFileData to ::__Internal_AddFileData and documented that it is for internal use in the source control api. This is a standard problem with our operations being both input and output structures and needs a better solution.
- Moved ::GetFileData to the cpp to reduce the amount of code in the header file.

[CL 25083228 by paul chipchase in ue5-main branch]
2023-04-18 06:32:01 -04:00
jordan hoffmann
cf8eb1d3c9 [Feature] AssetDefinition Merging
#jira UE-181566
#jira UE-181560
#rb ben.hoffman, dave.jones
#preflight 6439c1fec947f6523a79534d

[CL 25057796 by jordan hoffmann in ue5-main branch]
2023-04-14 21:26:21 -04:00
jordan hoffmann
4970c7e9ee [BugFix] Fixed Diff temp file Extensions missing '.' before them which caused .uassets to not be loadable
#rb none
#preflight trivial

[CL 25012259 by jordan hoffmann in ue5-main branch]
2023-04-12 14:58:02 -04:00
jordan hoffmann
fbe917e059 [BugFix] Review tool failing to connect to swarm
- Failed if user had multiple p4 tickets
- Failed if default p4 username didn't match the provided username in UE

#jira UE-181982
#rb paul.chipchase
#preflight 643061621d61ecec3b63c465

[CL 24970980 by jordan hoffmann in ue5-main branch]
2023-04-07 17:50:46 -04:00
jordan hoffmann
69baf07d45 [Attempted BugFix] Editor crashes when loading certain CLs in the Revision Changelist window of the UEQA_SC_P4V project
made PerforceSourceControlRevision.cpp temp file naming multithread friendly
#jira UE-173524
#rb none
#preflight 642f20051d61ecec3b044979

[CL 24954802 by jordan hoffmann in ue5-main branch]
2023-04-06 15:56:32 -04:00
paul chipchase
f5464ba58b Allow additional error info to be stored in FSourceControlResultInfo.
#rb wouter.burgers
#jira UE-181418
#preflight 642ece61df6c5c78fc9b99c9

- Added FSourceControlResultInfo::EAdditionalErrorContext that describes additional info about errors that the operation might have encountered that the caller can poll for rather than trying to parse error strings.
-- NOTE that we cannot enforce this info to be provided by source control provider implementations so this additional info should only be used to improve error messages.
-- Initially we support the connection to the provider failing and the connection to the provider dropping mid operation.
- Moved FSourceControlResultInfo to its own code files.
- I wanted to move the implementation of FSourceControlResultInfo::Append to the cpp, but this causes linker problems even with SOURCECONTROL_API as some modules are not including source control properly. For now I have left it in the header file and will look into this in another code pass.

[CL 24943799 by paul chipchase in ue5-main branch]
2023-04-06 10:04:00 -04:00
paul chipchase
5f0e71408d Remove unused perforce connection from FPerforceGetFileWorker
#rb Wouter.Burgers
#jira UE-173524
#rnx
#preflight 642e806dda7f9583702d81e4

- FPerforceGetFileWorker::Execute doesn't actually do any perforce work directly, it creates a new FPerforceSourceControlRevision and lets that object do the perforce work. So the connection that FPerforceGetFileWorker was making was never used and FPerforceSourceControlRevision would then create a second connection.

[CL 24941987 by paul chipchase in ue5-main branch]
2023-04-06 04:29:28 -04:00
logan buchy
a67304583f Add CVar to change perforce idle connection timeout
* Aids in testing this feature
* Use CMD:  SourceControl.Perforce.IdleConnectionDisconnectSeconds = <seconds>

#jira UE-173884
#rb johan.duparc
#preflight 642af82e6a37fb35e9bd0e42

[CL 24905944 by logan buchy in ue5-main branch]
2023-04-03 22:30:03 -04:00
marco anastasi
f6978a065c Created new Source Control feature flag UsesSnapshots() to solve inconsistency in use of UsesChangelists() and UsesFileRevisions() causing issues for git Source Control Provider
* Option to check in an asset from the Content Browser Source Control Context Menu is now enabled if the SCC does not use Snapshots instead of if it uses File Revisions
* Option to check in an asset from the Scene Outliner Source Control Context Menu is now enabled if the SCC does not use Snapshots instead of if it uses File Revisions
* Ability to uncheck files in the Submit Files Window is now enabled if the SCC does not use Snapshots instead of if it uses File Revisions
* Option to revert the writable flag on an asset in the Content Browser Source Control Context Menu is now enabled if the SCC uses checkout instead of if it uses File Revisions (like the 'Make Writable' option)

#jira UCS-6630
#rb brooke.hubert, wouter.burgers
#lockdown jeanmichel.dignard
#preflight 640e6b41482188d710ff8316

[CL 24724061 by marco anastasi in ue5-main branch]
2023-03-20 17:20:41 -04:00
logan buchy
1a734acb83 Add an idle timeout on perforce persistent connections
A connection to the perforce server can consume a large amount of memory on the server.  This change is to add a timeout to any persistent connections
in order to conserve this memory.
To do so, a connection tracks the platform time of the last successful communication.  The provider checks the persistent connection it has each frame and if
the last successful connect exceeded the timeout, then the connection is closed.  A new persistent connection will be established next time one is required.

The timeout is set as a hardcoded value for now.

#jira UE-173884
#rb Ronald.Koppers
#preflight 64122ee5af3fc352929f8e3b

[CL 24665411 by logan buchy in ue5-main branch]
2023-03-15 19:45:07 -04:00
Johan Duparc
3c59cba430 PerforceSourceControl: Use standard logging instead of an ensure to report unhandled file state
#jira UE-176058
#rb Brooke.Hubert
#preflight 641068d470639dfc94536c02

[CL 24633634 by Johan Duparc in ue5-main branch]
2023-03-14 08:36:48 -04:00
pj kack
4023ee1dde UnrealPak: Optimizations for small workloads.
Add -fastexit parameter to terminate process instead of shutting down engine.
Turn off build target features to build less code (total dll size reduced from 56 MB to 46 MB).
Disable external profiler supports (skips loading of some dlls).
Don't spawn any reserve workers (by adding a DefaultEngine.ini for UnrealPak).
Make UnrealPAk compile in Test config (by fixing compile error in PerforceConnection.cpp).

#rb carlmagnus.nordin
#rnx
#preflight 63e5eaf66473b534f14e377b

[CL 24118943 by pj kack in ue5-main branch]
2023-02-10 10:13:31 -05:00