116 Commits

Author SHA1 Message Date
Michael D Kinney
79fdd2b82a BaseTools: GNUMakefiles must use CMD.EXE in Windows
Use $(OS) in all GNUMakefiles to detect if the GNUMakefile
is being used in a Windows OS. If a Windows OS is detected,
then override SHELL to use cmd.exe. This prevents make
utility from using sh.exe if sh.exe happens to be in PATH.

If sh.exe is used, then backslash (\) characters in file
paths are removed and builds break for files not found.

Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
2025-12-24 02:03:38 +00:00
Pierre Gondois
f79717aac2 BaseTools: AutoGen: Remove unnecessary code
Running the vulture tool on the AutoGen folder gave the following
report. Remove the unnecessary code.

- AutoGen/BuildEngine.py:333:
  unused attribute 'SupportedToolChainFamilyList' (60% confidence)
- AutoGen/BuildEngine.py:346:
  unused attribute '_RuleObjectList' (60% confidence)
- AutoGen/GenMake.py:450:
  unused attribute 'FileBuildTargetList' (60% confidence)
- AutoGen/GenMake.py:452:
  unused attribute 'PendingBuildTargetList' (60% confidence)
- AutoGen/GenMake.py:458:
  unused attribute 'LibraryBuildCommandList' (60% confidence)
- AutoGen/GenMake.py:459:
  unused attribute 'LibraryFileList' (60% confidence)
- AutoGen/GenMake.py:462:
  unused attribute 'SystemLibraryList' (60% confidence)
- AutoGen/GenMake.py:1168:
  unused method 'GetFileDependency' (60% confidence)
- AutoGen/GenMake.py:1466:
  unused attribute 'ModuleBuildCommandList' (60% confidence)
- AutoGen/GenPcdDb.py:442:
  unused class 'DbSkuHeadTableItemList' (60% confidence)
- AutoGen/GenVar.py:31:
  unused attribute 'VpdRegionOffset' (60% confidence)
- AutoGen/GenVar.py:43:
  unused attribute 'VpdRegionOffset' (60% confidence)
- AutoGen/ModuleAutoGen.py:257:
  unused attribute 'FileDependCache' (60% confidence)
- AutoGen/ModuleAutoGen.py:684:
  unused method 'BuildOptionIncPathList' (60% confidence)
- AutoGen/PlatformAutoGen.py:994:
  unused method 'EdkBuildOption' (60% confidence)
- AutoGen/UniClassObject.py:171:
  unused attribute 'StringNameByteList' (60% confidence)
- AutoGen/UniClassObject.py:181:
  unused attribute 'StringNameByteList' (60% confidence)
- AutoGen/UniClassObject.py:366:
  unused method 'GetIncludeFile' (60% confidence)
- AutoGen/UniClassObject.py:606:
  unused method 'FindStringValue' (60% confidence)
- AutoGen/UniClassObject.py:616:
  unused method 'FindByToken' (60% confidence)
- AutoGen/ValidCheckingInfoObject.py:228:
  unused attribute 'ValidData' (60% confidence)
- AutoGen/ValidCheckingInfoObject.py:231:
  unused attribute 'ValidData' (60% confidence)

Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>
2025-07-01 09:51:38 +08:00
Michael D Kinney
62b0698309 BaseTools/AutoGen: GenMake response file quotes strings
If command line options are moved into a response file
of a GCC family build, then the file path separators are
converted from '\' to '/'. However, this can corrupt
command line options that are quoted strings.

Update GenMake to no convert '\' to '/' in quoted strings.

Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
2025-02-02 19:22:42 +00:00
Ken Lautner
50871ee0ec BaseTools: GenMake: FIx missing logs from GenMake.py
EdkLogger logs were not showing up as part of the build log output.
Adding the EdkLogger import to GenMake.py fixes the missing log prints.

Signed-off-by: Kenneth Lautner <kenlautner3@gmail.com>
2024-09-02 06:32:06 +00:00
Joey Vagedes
6c488a2f39 BaseTools: Fix raw strings containing valid escape characters
Fixes raw regex strings that contain valid (and purposeful) escape
characters as they are being treated as individual characters rather
than the single escaped character they represent (i.e. '\t' is being
treated as a '\' and a 't' rather than a single tab character).

Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2024-01-10 13:54:01 +00:00
Joey Vagedes via groups.io
9f0061a03b BaseTools: Resolve regex syntax warnings
Switches regex patterns to raw text to resolve python 3.12 syntax
warnings in regards to invalid escape sequences, as is suggested by the
re (regex) module in python.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-12-21 00:33:31 +00:00
Chen, Christine
176016387f BaseTools: add '-p' for Linux 'cp' command.
Currently BaseTools use 'cp' command for PcdValueInit and GenMake
process, as the command can not keep the time info of the source
file, which will cause incremental build issue in Linux system,
thus the '-p' need be added to keep the source file's attributes
in copy process.

This patch fixes this issue.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2022-07-17 02:20:51 +00:00
Feng, Bob C
5914128871 BaseTools: Fix the GenMake bug for .cpp source file
Build-rules.txt lists .cc and .cpp as supported file extensions.
BaseTools commit 05217d210e introduce a regression issue that
ignore the .cc and .cpp file type.

This patch is to fix this bug.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Yuwei Chen<yuwei.chen@intel.com>
2022-06-28 09:14:55 +00:00
Rebecca Cran
708620d29d BaseTools: Remove RVCT support
RVCT is obsolete and no longer used.
Remove support for it.

Signed-off-by: Rebecca Cran <quic_rcran@quicinc.com>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
2022-05-13 14:58:54 +00:00
Pierre Gondois
c214128a38 BaseTools/GenMake: Use ToolDefinition as fallback option
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3653

Use the value set in tools_def.txt when the makefile type is
not explicitly set via BuildOption. This allows to have a
valid default makefile name instead of an empty string.

Also use GMAKE_FILETYPE instead of hard-coded "gmake".

Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Reviewed-by: Chris Jones <christopher.jones@arm.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2021-09-29 14:31:37 +00:00
Michael D Kinney
8d2012035d BaseTools/GenMake: Sort generated makefile tool definitions
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3353

Sort the tool definition content of generated makefiles to help
verify that makefile contents have not changed after BaseTools
code changes.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
2021-04-27 05:56:39 +00:00
Mingyue Liang
8c610e6075 BaseTools: Limit command line length.
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2528

Currently, CL command contains multiple C files will be compiled,
and that caused command line too long, which may trigger build error.

In order to solve this issue, the following rules is used in this scene:

If the number of C files is greater than one, a txt file will be used
to record these C files, and replaces the corresponding content in
command line with the file name.

Else (only one C file listed in the command line), the length of the
whole CL command line will determine whether use a file to record. If
the length exceeds the limited max length, use the recording file; else
C file name directly listed in the command line

Signed-off-by: Mingyue Liang <mingyuex.liang@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-11-10 05:49:38 +00:00
Mingyue Liang
9641a7f975 BaseTools: Normalize case of pathname when evaluating Macros.
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2880

Currently, When doing the Incremental build, the directory
macros extended to absolute path in output Makefile, which
is inconsistent with the output of Clean build.

When we do macro replacement, we can't replace macro due to
inconsistent path case, which results in inconsistent display
of incremental build and clean build in makefile.Therefore,
the path is converted to achieve the correct macro replacement.

Signed-off-by: Mingyue Liang <mingyuex.liang@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
2020-09-25 02:27:35 +00:00
Pierre Gondois
6511277827 BaseTools: Generate multiple rules when multiple output files
This patch modifies the Makefile generation not to stop
adding Makfile rules when the first final target is found.
E.g.:
If the following rules are described in build_rule.txt:
 -[Rule1]: .X files generate .Y and .Z files;
 -[Rule2]: .Z files generate .Z1 files.
Currently, if a File1.X file was part of the sources of a
module, only [Rule1] would be generated in the Makefile.
Indeed, there are no rules to apply to .Y files: .Y files
are a final target. However, there is still [Rule2] to
apply to .Z files.

This patch also adds a dependency between the first
ouput file of a rule and the other output files.
For instance, with the same example as above, File1.Y
and File1.Z are generated by the following rule:
File1.Y: File1.X
    <Generate File1.Y>
    <Generate File1.Z>

and the new dependency is:
File1.Z: File1.Y

This is necessary to keep a dependency order during the
execution of the Makefile. Indeed, .Y and .Z files are
generated by the execution of a common set of commands,
and without this rule, there is no explicit dependency
relation between them.

Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Suggested-by: Tomas Pilar <Tomas.Pilar@arm.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-07-02 10:09:17 +00:00
Pierre Gondois
c44e0a896c BaseTools: Remove caret in NASM_INC macro
NASM_INC contains the list of directory to include when using
the nasm assembler.

In nmake makefiles, a trailing backslash escapes the newline char
and replaces it with a space ' '. To have a literal trailing
backslash, it must be escaped with a caret '^'. This is not
necessary for GNU makefiles.

On windows platforms, for the NASM_INC macro, a caret escaping
a trailing a backslash was appended to the last included
folder regardless of the makefile type.
For instance, "/Include/" was replaced by "/Include/^\".

This is causing a build failure on windows platforms using
GNU makefiles since the caret '^' doesn't escape any chars in
GNU makefiles and is thus conserved.
"/Include^\" was replaced by "/Include\/" in nmake makefiles,
but remained "/Include/^\" in GNU makefiles.

This patch removes the caret '^' on the build using GNU makefiles.

Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-12 02:34:44 +00:00
Pierre Gondois
818283de3f BaseTools: Rationalise makefile generation
The GenMake.py script tests the platform environment
to determine the type of makefile that needs to be
generated. If a Windows build host is detected, the
makefile generated is of Nmake type. Otherwise a
GNUmake type is generated.

Furthermore, the <TARGET>_<TAGNAME>_<ARCH>_MAKE_PATH
option in tools_def.template defines the make tool
to use.
E.g.: for VS2017 this is configured to use Nmake, cf.
*_VS2017_*_MAKE_PATH = DEF(VS2017_BIN_HOST)\nmake.exe
while for GCC5 it is setup to use GNU make.
*_GCC5_*_MAKE_PATH = DEF(GCC_HOST_PREFIX)make

This prevents using the GCC compiler toolchain on a
Windows build host.

To address this issue this patch introduces 2 factors
to determine the generated makefile output.
  1. Platform -> to determine shell commands used
                 in makefile.
  2. MakeTool -> to determine the type of makefile
                 that needs to be generated.

Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>
Signed-off-by: Sami Mujawar <sami.mujawar@arm.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-12 02:34:44 +00:00
Bob Feng
b34ed98694 BaseTools: Fixed a Incremental build issue
The .map file is not update to FFS_OUTPUT_DIR folder
in the incremental build.

Signed-off-by: Guo Dong <guo.dong@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-02-09 09:29:02 +00:00
Bob Feng
5cd3d4bc43 BaseTools: Fixed a incremental build bug
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2451

If removing a header file from source code and file
system, the incremental build will fail.

This patch is to fix this issue by setting each header file
as a target without any actions in makefile.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>

Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-01-19 07:46:18 +00:00
Bob Feng
0c3e8e9947 BaseTools: Enhance Basetool for incremental build
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2311

Include dependency file in Makefile to enhance
incremental build

Signed-off-by: Bob Feng <bob.c.feng@intel.com>

Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-12-10 01:31:55 +00:00
Lin, Derek (HPS SW)
b1c6e9f55e BaseTools: Fix an incremental build issue caused by macro in #include
When c/h file use macro after #include, for example,
In this case, GenMake is not able to create a healthy dependency for the c
file. GenMake used to add $(FORCE_REBUILD) dependency in the c file, this
guarantee the c file is always compiled in incremental build. But, this
function is broken since 05217d210e which
enable /MP for MSVC compiler, in order to compile multiple c files in one
command multi-processing. The fix here is adding '$(FORCE_REBUILD)' back to
retain the original function.

Line number 1728 and 978 are the code pieces which handle this logic.

Signed-off-by: Derek Lin <derek.lin2@hpe.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-10-18 08:30:38 +08:00
Kubacki, Michael A
61af5f2494 BaseTools: Fix GenMake multi-workspace failure
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2232

Commit 0075ab2cec introduced an issue that causes an exception
when multiple workspace packages paths are specified. For example,
if edk2-platforms is used, the root directory will contain an edk
and edk2-platforms directory representing the respective
repositories.

In GenMake, the path to the package DEC file for a module is
discovered by getting the relative path of the INF to the
workspace root directory. Each directory in the relative path
is incrementally joined to the WORKSPACE directory. The file
list in the joined path is searched for a DEC file.

As an example, if the build command is used on a package outside
the edk2 repository, the INF file path is relative to the
edk2-platforms directory not edk2. This causes directory paths
to be built that do not exist. Commit 0075ab2cec replaced the
os.path.exists() call with a try except block that always fails
when os.listdir() is invoked to enumerate the list of files in
the built directory path on packages outside edk2.

This commit restores the original conditional statement which
avoids calling os.listdir() with an invalid directory path.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Signed-off-by: Michael Kubacki <michael.a.kubacki@intel.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by:  Bob Feng <bob.c.feng@intel.com>
2019-10-04 11:13:37 +08:00
Feng, Bob C
4747b92cab BaseTools: init DependencyHeaderFileSet for each MakeFile class
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2203

This patch is going to fix a regression issue that
build breaks when the custom makefile exist.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-24 10:29:09 +08:00
Liming Gao
fcdedafd97 Revert "BaseTools: Improve GetDependencyList function"
This reverts commit bc9e4194cf.
This change causes the dependent header files are missing in Makefile.
It makes the incremental build not work. So, revert this change.

Cc: Bob Feng<bob.c.feng@Intel.com>
Signed-off-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng<bob.c.feng@Intel.com>
2019-09-21 00:25:48 +08:00
Feng, Bob C
0075ab2cec BaseTools: remove unnecessary calls of os.exist
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2101

This patch is going to remove unnecessary calls
of os.exist()

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-17 10:19:05 +08:00
Feng, Bob C
bc9e4194cf BaseTools: Improve GetDependencyList function
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2102

GetDependencyList get the header file via
re.findall in the whole header file.

This patch is to pre-process the header file and
to feed the shorter string to re.findall.

This patch is to improve GetDependencyList() efficiency

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-17 10:19:05 +08:00