960c130da0
#lockdown Nick.Penwarden
#rb none
============================
MAJOR FEATURES & CHANGES
============================
Change 3683865 by josh.markiewicz
#UE4 - fixed bad parsing for asset hotfix (curve/table data)
Change 3683724 by eric.newman
Fix to hotfixing of DedicatedServer*.ini
* Previously hotfixes to server were not being merged into the correct hierarchy, eg. Game.ini or Engine.ini, resulting in the hotfix failing to apply
Change 3683212 by josh.markiewicz
#UE4 - added IsUsableRegion function to QOS manager
Change 3652476 by josh.markiewicz
#UE4 - fixed potential crash with references to array members during array reallocation
- added some MoveTemps to speed up copies
Change 3651942 by ian.fox
Fix ensure in beacon client. We explicitly ask to stop ticking immediateyl when we ask to be destroyed
Change 3645565 by paul.moore
- Add ability to manually configure beacon teams.
Change 3643056 by josh.markiewicz
#UE4 - added function to retrieve number of active beacon connections on a given registered beacon type
Change 3639066 by josh.markiewicz
#UE4 - beacon clients tick to monitor socket closure from UChannel::Cleanup
- beacons were previously waiting "timeout" amount of time in a socket closed state
Change 3623393 by josh.markiewicz
#ue4 prevent calling "join server" multiple times when the call has already been made once
Change 3622342 by josh.markiewicz
#UE4 - playerid netconnection variable setup properly on clients and servers for both beacons and game net drivers
- ipconnection prints uniqueid with lowleveldescribe
Change 3616840 by mike.fricker
Added "-SkipHotfixCheck" to avoid hotfixing in non-shipping builds
- Copied from CL 3325969
- Use this command-line locally when doing testing on a client or server that is different than the CL that currently-staged hotfixes are designed for. Hotfixable assets may not be compatible between CLs.
Change 3610005 by mike.fricker
Initial support for hotfixing live assets from .ini files (part 2)
- Fixed an issue where spaces in Json data could crash while parsing configs during hotfixes
- Removed non-functional code to find objects in memory before loading
Change 3608588 by rob.cannaday
Add IsSessionInfoValid to FOnlineSessionSearchResult to allow for checking session validity regardless of OwningUserId validity
Change 3606221 by ryan.gerleve
Fortnite now uses new encryption key hooks to set the key on a client before sending the initial hello message for the world net driver and beacons. Fixes getting stuck on the loading screen if an encryption handshake message was dropped.
Change 3604788 by mike.fricker
Initial support for hotfixing live assets from .ini files
- This allows clients and server to patch certain assets in memory whenever .ini file hotfixes are downloaded
- Only CurveTables and DataTables are supported for now
- The new asset content must be in Json format, the same format the editor uses for importing
- Assets that are hotfixed will be synchronously loaded if they're not already in memory. They'll be retained in memory afterwards.
- IMPORTANT: Json data must be supplied on a single line, and all double quotes must be escaped!
- The changes must go in the Game.ini file and use the following syntax:
[AssetHotfix]
+CurveTable=("/Game/Folder/MyCurveTable","[{\"Name\":\"Default\"}]")
+DataTable=("/Game/Folder2/MyDataTable","[{\"Name\":\"Foo\"}]")
Change 3602034 by rob.cannaday
Fix crash in printf statement where 4 strings are expected but only 3 are provided
[CL 3696389 by Josh Markiewicz in Main branch]
INCLUDE TOOL
The IncludeTool utility can be used to convert existing C++ projects into an include-what-you-use style. It does so by attempting to form self-contained translation units out of each header file.
While it can save a lot of time, it is based on a heuristic, and is highly unlikely to produce perfect output - rather, it may produce a starting point from which you can make manual edits to complete the transition. Even then, it operates by brute force (so is agnonizingly slow; in the order of taking several days to complete for large projects), is painful to iterate on, may require project-specific modifications, and can be obtuse to debug.
It has been used to convert a number of internal Epic projects and the engine itself, and while we provide it as a courtesy, the level of support we can provide for it will be limited.
OPERATION
IncludeTool operates in several phases:
-
UnrealBuildTool is invoked with a target to generate a list of build steps. The project's editor target is recommended, since it typically includes more code paths than any other target type. We also recommend compiling for Linux using the cross-compile toolchain from windows, so as to use the Clang toolchain. Using Clang is important, because Visual C++ doesn't do two phase name lookup. That fails to find dependencies from template classes on dependent classes until those templates are instantiated, which can make a real mess of the output.
-
The source files are partially preprocessed using an internal preprocessor. This does not transform the tokenized source file itself, but determines which blocks of code are active and inactive for the current target, and splits each file into a series of fragments. Each fragment defines a range of lines in the source file between #include directives, meaning that a single-file version of any source file can be assembled at any point by recursively following #include directives and concatenating the list of fragments encountered.
This step is important, because it is the list of fragments which IncludeTool is going to optimize. There are several restrictions that IncludeTool validates during this phase and issues warnings about - some appear pedantic and stylistic, but are required to ensure that the output is valid. Particularly notable are:
- There should be no circular includes between header files
- Included source file must be preprocessed in the same way by every translation unit (ie. any macro must be defined the same way in every context that it is included).
-
Each fragment is written to the working folder. To preserve line numbers, each source file is output in its original layout, but with lines belonging to other fragments commented out.
-
Source files are tokenized and searched for patterns that look like symbols that can be forward declared. (eg. "class Foo { ...")
-
Each source file is brute-force compiled to determine which fragments it depends on to compile successfully. This is the most labor intensive part of the transformation, but the results from this analysis are stored in the working directory, and should be reusable if source files do not change. The tool also supports a "sharded" mode for using multiple PCs to compute the dependency data (see below).
The search is structured as follows:
- An input translation unit is expanded to be represented by a sequence of fragments (1...n).
- A set of the required fragments (r) is initialized to only those fragments that were in the input file (ie. rather than being included by the input file).
- A binary search is performed to find the longest sequence of fragments required for the source file to still successfully compile (1...m, with r).
- The last fragment in the sequence (m), is added to the required set (r). If 'm' has already been optimized, its dependencies are also added to r.
- The binary search is repeated for the sequence of fragments (1...m - 1).
-
Each output file is written with a minimal set of includes for it to compile in isolation. The heuristic used attempts to directly include the header for any dependency containing a symbol that is explicitly referenced by name, and only include other dependencies if they are not included recursively.
EXAMPLE USAGE
To just do the fragment analysis and output diagnostic information:
-Mode=Scan -Target=FooEditor -Platform=Linux -Configuration=Development -WorkingDir=D:\Working
To optimize a codebase:
-Mode=Optimize -Target=FooEditor -Platform=Linux -Configuration=Development -WorkingDir=D:\Working -OutputDir=D:\Output -SourceFiles=-/Engine/... -OptimizeFiles=-/Engine/... -OutputFiles=-/Engine/...
Several filter arguments may be passed to limit source files iteration (-SourceFiles, -OptimizeFiles, -OutputFiles). Each can take a semicolon list of P4-style wildcards to include (/Engine/...) or exclude (-/Engine/Foo.cpp) files, or may specify a response file with rules, one per line (@D:\Filter.txt). In general, you probably don't want to re-optimize engine code, so it makes sense to exclude all of those from analysis.
Since the program can take so long to run, it does have a facility for running in "sharded" mode. If you have several PCs all synced up to the same source tree, you can use the -Shard=N and -NumShards=M to configure a machine to only consider a portion of the input set. The working directories should be in the same location on each machine, and the resulting working directories can be copied together and used for a final run on a single machine to generate output files.
See comments in Program.cs for other modes (the ToolMode enum) and command-line options (the CommandLineOptions class).