Files
UnrealEngineUWP/Engine/Source/Programs/UnrealBuildTool/Platform/Windows/VCSpecificFileAction.cs

137 lines
4.7 KiB
C#
Raw Normal View History

[UBT] Changed how single files are built. We don't want to invalidate makefile everytime we build single files since it destroys turnaround times. With this change a single file compile can take 2s (+ the actual compile time). The new behavior injects a special action per module when creating the makefile. These actions can be used to on-the-fly create a proper compile action that follows the rules of the module that the specific file belongs to.. In a normal build these actions are ignored since the logic deciding which actions to build is backtraced from which binaries we want to create. When a specific file compile is triggered, the logic deciding which files to build search up all these special actions and create a lookup based on which folders the special actions handle. It then try to find the special action that handles the specific file. The matching special action then creates a compile action that can handle that specific file and then queue up the action for execution. If no special action is found it falls back to try to use actions that have this specific file as input (ispc files for example) Details: * Removed lots of custom code for "specific files" handling * Changed so pch (both private and shared) always use definition file. Added #pragma once and change so pch wrapper file include definition file. This made the adaptive path and specific file easy to implement (just disable pch in compile environment and it will just work) * Added SingleFileAction for both VCToolChain and ClangToolChain. It now works to compile specific headers and cpp files. (It creates wrapper files on the fly to be able to compile all header files (compiling headers directly blow up if there are circular includes) * Fixed so GenerateClangDatabase mode works with new changes * Moved the logic that makes sure all (directly) depending cpp files are recompiled when .h are included in the singlefile option #preflight 63dcc46f78716a01e8069649 #rb joe.kirchoff [CL 24094027 by henrik karlsson in ue5-main branch]
2023-02-09 04:20:43 -05:00
// Copyright Epic Games, Inc. All Rights Reserved.
using System.Collections.Generic;
using System.IO;
[UBT] Changed how single files are built. We don't want to invalidate makefile everytime we build single files since it destroys turnaround times. With this change a single file compile can take 2s (+ the actual compile time). The new behavior injects a special action per module when creating the makefile. These actions can be used to on-the-fly create a proper compile action that follows the rules of the module that the specific file belongs to.. In a normal build these actions are ignored since the logic deciding which actions to build is backtraced from which binaries we want to create. When a specific file compile is triggered, the logic deciding which files to build search up all these special actions and create a lookup based on which folders the special actions handle. It then try to find the special action that handles the specific file. The matching special action then creates a compile action that can handle that specific file and then queue up the action for execution. If no special action is found it falls back to try to use actions that have this specific file as input (ispc files for example) Details: * Removed lots of custom code for "specific files" handling * Changed so pch (both private and shared) always use definition file. Added #pragma once and change so pch wrapper file include definition file. This made the adaptive path and specific file easy to implement (just disable pch in compile environment and it will just work) * Added SingleFileAction for both VCToolChain and ClangToolChain. It now works to compile specific headers and cpp files. (It creates wrapper files on the fly to be able to compile all header files (compiling headers directly blow up if there are circular includes) * Fixed so GenerateClangDatabase mode works with new changes * Moved the logic that makes sure all (directly) depending cpp files are recompiled when .h are included in the singlefile option #preflight 63dcc46f78716a01e8069649 #rb joe.kirchoff [CL 24094027 by henrik karlsson in ue5-main branch]
2023-02-09 04:20:43 -05:00
using EpicGames.Core;
using Microsoft.Extensions.Logging;
using UnrealBuildBase;
namespace UnrealBuildTool
{
internal class VcSpecificFileAction : Action, ISpecificFileAction
{
DirectoryReference SourceDir;
DirectoryReference OutputDir;
VCCompileAction BaseAction;
// TODO: Entire CppCompileEnvironment needs to be saved with BinaryArchiveWriter, some options may still be unused
bool bPreprocessOnly;
bool bWithAssembly;
Dictionary<string, List<FileItem>> SingleFiles = new();
[UBT] Changed how single files are built. We don't want to invalidate makefile everytime we build single files since it destroys turnaround times. With this change a single file compile can take 2s (+ the actual compile time). The new behavior injects a special action per module when creating the makefile. These actions can be used to on-the-fly create a proper compile action that follows the rules of the module that the specific file belongs to.. In a normal build these actions are ignored since the logic deciding which actions to build is backtraced from which binaries we want to create. When a specific file compile is triggered, the logic deciding which files to build search up all these special actions and create a lookup based on which folders the special actions handle. It then try to find the special action that handles the specific file. The matching special action then creates a compile action that can handle that specific file and then queue up the action for execution. If no special action is found it falls back to try to use actions that have this specific file as input (ispc files for example) Details: * Removed lots of custom code for "specific files" handling * Changed so pch (both private and shared) always use definition file. Added #pragma once and change so pch wrapper file include definition file. This made the adaptive path and specific file easy to implement (just disable pch in compile environment and it will just work) * Added SingleFileAction for both VCToolChain and ClangToolChain. It now works to compile specific headers and cpp files. (It creates wrapper files on the fly to be able to compile all header files (compiling headers directly blow up if there are circular includes) * Fixed so GenerateClangDatabase mode works with new changes * Moved the logic that makes sure all (directly) depending cpp files are recompiled when .h are included in the singlefile option #preflight 63dcc46f78716a01e8069649 #rb joe.kirchoff [CL 24094027 by henrik karlsson in ue5-main branch]
2023-02-09 04:20:43 -05:00
internal VcSpecificFileAction(DirectoryReference Source, DirectoryReference Output, VCCompileAction Action, CppCompileEnvironment? CompileEnvironment = null) : base(ActionType.Compile)
[UBT] Changed how single files are built. We don't want to invalidate makefile everytime we build single files since it destroys turnaround times. With this change a single file compile can take 2s (+ the actual compile time). The new behavior injects a special action per module when creating the makefile. These actions can be used to on-the-fly create a proper compile action that follows the rules of the module that the specific file belongs to.. In a normal build these actions are ignored since the logic deciding which actions to build is backtraced from which binaries we want to create. When a specific file compile is triggered, the logic deciding which files to build search up all these special actions and create a lookup based on which folders the special actions handle. It then try to find the special action that handles the specific file. The matching special action then creates a compile action that can handle that specific file and then queue up the action for execution. If no special action is found it falls back to try to use actions that have this specific file as input (ispc files for example) Details: * Removed lots of custom code for "specific files" handling * Changed so pch (both private and shared) always use definition file. Added #pragma once and change so pch wrapper file include definition file. This made the adaptive path and specific file easy to implement (just disable pch in compile environment and it will just work) * Added SingleFileAction for both VCToolChain and ClangToolChain. It now works to compile specific headers and cpp files. (It creates wrapper files on the fly to be able to compile all header files (compiling headers directly blow up if there are circular includes) * Fixed so GenerateClangDatabase mode works with new changes * Moved the logic that makes sure all (directly) depending cpp files are recompiled when .h are included in the singlefile option #preflight 63dcc46f78716a01e8069649 #rb joe.kirchoff [CL 24094027 by henrik karlsson in ue5-main branch]
2023-02-09 04:20:43 -05:00
{
SourceDir = Source;
OutputDir = Output;
BaseAction = Action;
bPreprocessOnly = CompileEnvironment?.bPreprocessOnly ?? false;
bWithAssembly = CompileEnvironment?.bWithAssembly ?? false;
[UBT] Changed how single files are built. We don't want to invalidate makefile everytime we build single files since it destroys turnaround times. With this change a single file compile can take 2s (+ the actual compile time). The new behavior injects a special action per module when creating the makefile. These actions can be used to on-the-fly create a proper compile action that follows the rules of the module that the specific file belongs to.. In a normal build these actions are ignored since the logic deciding which actions to build is backtraced from which binaries we want to create. When a specific file compile is triggered, the logic deciding which files to build search up all these special actions and create a lookup based on which folders the special actions handle. It then try to find the special action that handles the specific file. The matching special action then creates a compile action that can handle that specific file and then queue up the action for execution. If no special action is found it falls back to try to use actions that have this specific file as input (ispc files for example) Details: * Removed lots of custom code for "specific files" handling * Changed so pch (both private and shared) always use definition file. Added #pragma once and change so pch wrapper file include definition file. This made the adaptive path and specific file easy to implement (just disable pch in compile environment and it will just work) * Added SingleFileAction for both VCToolChain and ClangToolChain. It now works to compile specific headers and cpp files. (It creates wrapper files on the fly to be able to compile all header files (compiling headers directly blow up if there are circular includes) * Fixed so GenerateClangDatabase mode works with new changes * Moved the logic that makes sure all (directly) depending cpp files are recompiled when .h are included in the singlefile option #preflight 63dcc46f78716a01e8069649 #rb joe.kirchoff [CL 24094027 by henrik karlsson in ue5-main branch]
2023-02-09 04:20:43 -05:00
}
public VcSpecificFileAction(BinaryArchiveReader Reader) : base(ActionType.Compile)
{
SourceDir = Reader.ReadCompactDirectoryReference();
OutputDir = Reader.ReadCompactDirectoryReference();
bPreprocessOnly = Reader.ReadBool();
bWithAssembly = Reader.ReadBool();
[UBT] Changed how single files are built. We don't want to invalidate makefile everytime we build single files since it destroys turnaround times. With this change a single file compile can take 2s (+ the actual compile time). The new behavior injects a special action per module when creating the makefile. These actions can be used to on-the-fly create a proper compile action that follows the rules of the module that the specific file belongs to.. In a normal build these actions are ignored since the logic deciding which actions to build is backtraced from which binaries we want to create. When a specific file compile is triggered, the logic deciding which files to build search up all these special actions and create a lookup based on which folders the special actions handle. It then try to find the special action that handles the specific file. The matching special action then creates a compile action that can handle that specific file and then queue up the action for execution. If no special action is found it falls back to try to use actions that have this specific file as input (ispc files for example) Details: * Removed lots of custom code for "specific files" handling * Changed so pch (both private and shared) always use definition file. Added #pragma once and change so pch wrapper file include definition file. This made the adaptive path and specific file easy to implement (just disable pch in compile environment and it will just work) * Added SingleFileAction for both VCToolChain and ClangToolChain. It now works to compile specific headers and cpp files. (It creates wrapper files on the fly to be able to compile all header files (compiling headers directly blow up if there are circular includes) * Fixed so GenerateClangDatabase mode works with new changes * Moved the logic that makes sure all (directly) depending cpp files are recompiled when .h are included in the singlefile option #preflight 63dcc46f78716a01e8069649 #rb joe.kirchoff [CL 24094027 by henrik karlsson in ue5-main branch]
2023-02-09 04:20:43 -05:00
BaseAction = new VCCompileAction(Reader);
}
public new void Write(BinaryArchiveWriter Writer)
{
Writer.WriteCompactDirectoryReference(SourceDir);
Writer.WriteCompactDirectoryReference(OutputDir);
Writer.WriteBool(bPreprocessOnly);
Writer.WriteBool(bWithAssembly);
[UBT] Changed how single files are built. We don't want to invalidate makefile everytime we build single files since it destroys turnaround times. With this change a single file compile can take 2s (+ the actual compile time). The new behavior injects a special action per module when creating the makefile. These actions can be used to on-the-fly create a proper compile action that follows the rules of the module that the specific file belongs to.. In a normal build these actions are ignored since the logic deciding which actions to build is backtraced from which binaries we want to create. When a specific file compile is triggered, the logic deciding which files to build search up all these special actions and create a lookup based on which folders the special actions handle. It then try to find the special action that handles the specific file. The matching special action then creates a compile action that can handle that specific file and then queue up the action for execution. If no special action is found it falls back to try to use actions that have this specific file as input (ispc files for example) Details: * Removed lots of custom code for "specific files" handling * Changed so pch (both private and shared) always use definition file. Added #pragma once and change so pch wrapper file include definition file. This made the adaptive path and specific file easy to implement (just disable pch in compile environment and it will just work) * Added SingleFileAction for both VCToolChain and ClangToolChain. It now works to compile specific headers and cpp files. (It creates wrapper files on the fly to be able to compile all header files (compiling headers directly blow up if there are circular includes) * Fixed so GenerateClangDatabase mode works with new changes * Moved the logic that makes sure all (directly) depending cpp files are recompiled when .h are included in the singlefile option #preflight 63dcc46f78716a01e8069649 #rb joe.kirchoff [CL 24094027 by henrik karlsson in ue5-main branch]
2023-02-09 04:20:43 -05:00
BaseAction.Write(Writer);
}
public DirectoryReference RootDirectory => SourceDir;
[UBT] Changed how single files are built. We don't want to invalidate makefile everytime we build single files since it destroys turnaround times. With this change a single file compile can take 2s (+ the actual compile time). The new behavior injects a special action per module when creating the makefile. These actions can be used to on-the-fly create a proper compile action that follows the rules of the module that the specific file belongs to.. In a normal build these actions are ignored since the logic deciding which actions to build is backtraced from which binaries we want to create. When a specific file compile is triggered, the logic deciding which files to build search up all these special actions and create a lookup based on which folders the special actions handle. It then try to find the special action that handles the specific file. The matching special action then creates a compile action that can handle that specific file and then queue up the action for execution. If no special action is found it falls back to try to use actions that have this specific file as input (ispc files for example) Details: * Removed lots of custom code for "specific files" handling * Changed so pch (both private and shared) always use definition file. Added #pragma once and change so pch wrapper file include definition file. This made the adaptive path and specific file easy to implement (just disable pch in compile environment and it will just work) * Added SingleFileAction for both VCToolChain and ClangToolChain. It now works to compile specific headers and cpp files. (It creates wrapper files on the fly to be able to compile all header files (compiling headers directly blow up if there are circular includes) * Fixed so GenerateClangDatabase mode works with new changes * Moved the logic that makes sure all (directly) depending cpp files are recompiled when .h are included in the singlefile option #preflight 63dcc46f78716a01e8069649 #rb joe.kirchoff [CL 24094027 by henrik karlsson in ue5-main branch]
2023-02-09 04:20:43 -05:00
public IExternalAction? CreateAction(FileItem SourceFile, ILogger Logger)
{
DirectoryReference IntermediateDirectory = DirectoryReference.Combine(OutputDir, "SingleFile");
// Keep track of all specific files, so the output file can be renamed if there's a naming conflict
string Filename = SourceFile.Name;
if (!SingleFiles.ContainsKey(SourceFile.Name))
{
SingleFiles[SourceFile.Name] = new();
}
else
{
Filename = $"{Path.GetFileNameWithoutExtension(SourceFile.Name)}{SingleFiles[Filename].Count}{Path.GetExtension(SourceFile.Name)}";
}
SingleFiles[SourceFile.Name].Add(SourceFile);
if (!SourceFile.HasExtension(".cpp") && !SourceFile.HasExtension(".h"))
[UBT] Changed how single files are built. We don't want to invalidate makefile everytime we build single files since it destroys turnaround times. With this change a single file compile can take 2s (+ the actual compile time). The new behavior injects a special action per module when creating the makefile. These actions can be used to on-the-fly create a proper compile action that follows the rules of the module that the specific file belongs to.. In a normal build these actions are ignored since the logic deciding which actions to build is backtraced from which binaries we want to create. When a specific file compile is triggered, the logic deciding which files to build search up all these special actions and create a lookup based on which folders the special actions handle. It then try to find the special action that handles the specific file. The matching special action then creates a compile action that can handle that specific file and then queue up the action for execution. If no special action is found it falls back to try to use actions that have this specific file as input (ispc files for example) Details: * Removed lots of custom code for "specific files" handling * Changed so pch (both private and shared) always use definition file. Added #pragma once and change so pch wrapper file include definition file. This made the adaptive path and specific file easy to implement (just disable pch in compile environment and it will just work) * Added SingleFileAction for both VCToolChain and ClangToolChain. It now works to compile specific headers and cpp files. (It creates wrapper files on the fly to be able to compile all header files (compiling headers directly blow up if there are circular includes) * Fixed so GenerateClangDatabase mode works with new changes * Moved the logic that makes sure all (directly) depending cpp files are recompiled when .h are included in the singlefile option #preflight 63dcc46f78716a01e8069649 #rb joe.kirchoff [CL 24094027 by henrik karlsson in ue5-main branch]
2023-02-09 04:20:43 -05:00
{
return null;
}
DirectoryReference.CreateDirectory(IntermediateDirectory);
[UBT] Changed how single files are built. We don't want to invalidate makefile everytime we build single files since it destroys turnaround times. With this change a single file compile can take 2s (+ the actual compile time). The new behavior injects a special action per module when creating the makefile. These actions can be used to on-the-fly create a proper compile action that follows the rules of the module that the specific file belongs to.. In a normal build these actions are ignored since the logic deciding which actions to build is backtraced from which binaries we want to create. When a specific file compile is triggered, the logic deciding which files to build search up all these special actions and create a lookup based on which folders the special actions handle. It then try to find the special action that handles the specific file. The matching special action then creates a compile action that can handle that specific file and then queue up the action for execution. If no special action is found it falls back to try to use actions that have this specific file as input (ispc files for example) Details: * Removed lots of custom code for "specific files" handling * Changed so pch (both private and shared) always use definition file. Added #pragma once and change so pch wrapper file include definition file. This made the adaptive path and specific file easy to implement (just disable pch in compile environment and it will just work) * Added SingleFileAction for both VCToolChain and ClangToolChain. It now works to compile specific headers and cpp files. (It creates wrapper files on the fly to be able to compile all header files (compiling headers directly blow up if there are circular includes) * Fixed so GenerateClangDatabase mode works with new changes * Moved the logic that makes sure all (directly) depending cpp files are recompiled when .h are included in the singlefile option #preflight 63dcc46f78716a01e8069649 #rb joe.kirchoff [CL 24094027 by henrik karlsson in ue5-main branch]
2023-02-09 04:20:43 -05:00
VCCompileAction Action = new VCCompileAction(BaseAction);
Action.ArtifactMode = ArtifactMode.None;
[UBT] Changed how single files are built. We don't want to invalidate makefile everytime we build single files since it destroys turnaround times. With this change a single file compile can take 2s (+ the actual compile time). The new behavior injects a special action per module when creating the makefile. These actions can be used to on-the-fly create a proper compile action that follows the rules of the module that the specific file belongs to.. In a normal build these actions are ignored since the logic deciding which actions to build is backtraced from which binaries we want to create. When a specific file compile is triggered, the logic deciding which files to build search up all these special actions and create a lookup based on which folders the special actions handle. It then try to find the special action that handles the specific file. The matching special action then creates a compile action that can handle that specific file and then queue up the action for execution. If no special action is found it falls back to try to use actions that have this specific file as input (ispc files for example) Details: * Removed lots of custom code for "specific files" handling * Changed so pch (both private and shared) always use definition file. Added #pragma once and change so pch wrapper file include definition file. This made the adaptive path and specific file easy to implement (just disable pch in compile environment and it will just work) * Added SingleFileAction for both VCToolChain and ClangToolChain. It now works to compile specific headers and cpp files. (It creates wrapper files on the fly to be able to compile all header files (compiling headers directly blow up if there are circular includes) * Fixed so GenerateClangDatabase mode works with new changes * Moved the logic that makes sure all (directly) depending cpp files are recompiled when .h are included in the singlefile option #preflight 63dcc46f78716a01e8069649 #rb joe.kirchoff [CL 24094027 by henrik karlsson in ue5-main branch]
2023-02-09 04:20:43 -05:00
Action.SourceFile = SourceFile;
if (SourceFile.HasExtension(".h"))
{
if (BaseAction.CompilerType.IsClang())
{
ClangWarnings.GetHeaderDisabledWarnings(Action.Arguments);
}
string IncludeFileString = SourceFile.AbsolutePath;
if (SourceFile.Location.IsUnderDirectory(Unreal.RootDirectory))
{
IncludeFileString = SourceFile.Location.MakeRelativeTo(Unreal.EngineSourceDirectory);
}
List<string> GeneratedHeaderCppContents = UEBuildModuleCPP.GenerateHeaderCpp(SourceFile.Name, IncludeFileString);
Action.SourceFile = FileItem.GetItemByFileReference(FileReference.Combine(IntermediateDirectory, $"{SourceFile.Name}.cpp"));
File.WriteAllLines(Action.SourceFile.FullName, GeneratedHeaderCppContents);
}
if (bPreprocessOnly)
{
Action.PreprocessedFile = FileItem.GetItemByFileReference(FileReference.Combine(IntermediateDirectory, $"{Filename}.i"));
}
else
{
Action.ObjectFile = FileItem.GetItemByFileReference(FileReference.Combine(IntermediateDirectory, $"{Filename}.obj"));
if (BaseAction.CompilerType.IsMSVC() && bWithAssembly)
{
Action.AssemblyFile = FileItem.GetItemByFileReference(FileReference.Combine(IntermediateDirectory, $"{Filename}.asm"));
}
}
if (BaseAction.CompilerType.IsMSVC())
{
Action.DependencyListFile = FileItem.GetItemByFileReference(FileReference.Combine(IntermediateDirectory, $"{Filename}.dep.json"));
}
else if (BaseAction.CompilerType.IsClang())
{
Action.DependencyListFile = FileItem.GetItemByFileReference(FileReference.Combine(IntermediateDirectory, $"{Filename}.d"));
}
Action.ResponseFile = FileItem.GetItemByFileReference(FileReference.Combine(IntermediateDirectory, $"{Filename}.rsp"));
File.WriteAllLines(Action.ResponseFile.FullName, Action.GetCompilerArguments(Logger));
Action.Arguments.Clear();
[UBT] Changed how single files are built. We don't want to invalidate makefile everytime we build single files since it destroys turnaround times. With this change a single file compile can take 2s (+ the actual compile time). The new behavior injects a special action per module when creating the makefile. These actions can be used to on-the-fly create a proper compile action that follows the rules of the module that the specific file belongs to.. In a normal build these actions are ignored since the logic deciding which actions to build is backtraced from which binaries we want to create. When a specific file compile is triggered, the logic deciding which files to build search up all these special actions and create a lookup based on which folders the special actions handle. It then try to find the special action that handles the specific file. The matching special action then creates a compile action that can handle that specific file and then queue up the action for execution. If no special action is found it falls back to try to use actions that have this specific file as input (ispc files for example) Details: * Removed lots of custom code for "specific files" handling * Changed so pch (both private and shared) always use definition file. Added #pragma once and change so pch wrapper file include definition file. This made the adaptive path and specific file easy to implement (just disable pch in compile environment and it will just work) * Added SingleFileAction for both VCToolChain and ClangToolChain. It now works to compile specific headers and cpp files. (It creates wrapper files on the fly to be able to compile all header files (compiling headers directly blow up if there are circular includes) * Fixed so GenerateClangDatabase mode works with new changes * Moved the logic that makes sure all (directly) depending cpp files are recompiled when .h are included in the singlefile option #preflight 63dcc46f78716a01e8069649 #rb joe.kirchoff [CL 24094027 by henrik karlsson in ue5-main branch]
2023-02-09 04:20:43 -05:00
return Action;
}
}
class VcSpecificFileActionSerializer : ActionSerializerBase<VcSpecificFileAction>
{
/// <inheritdoc/>
public override VcSpecificFileAction Read(BinaryArchiveReader Reader)
{
return new VcSpecificFileAction(Reader);
}
/// <inheritdoc/>
public override void Write(BinaryArchiveWriter Writer, VcSpecificFileAction Action)
{
Action.Write(Writer);
}
}
}