The stuart tools automatically add -D WIN_HOST_BUILD to
edk2 build command line if a Windows build environment
is detected. This behavior is added to build.py so that
builds of the EmulatorPkg using build.py are not required
to add the option -D WIN_HOST_BUILD when building in a
Windows environment. This aligns Linux and Windows builds
of the EmulatorPkg removing the need to specify extra
defines.
In order to build the EmulatorPkg for Windows Mingw
environments, EmulatorPkg DSC/FDF files require a way
to detect if Windows Mingw environment is present.
The Windows Mingw environment can be detected if
CLANG_BIN is set and mingw32-make.exe is detected
in the CLANG_BIN directory.
If a Windows Mingw environment is detected, add
-D WIN_MINGW32_BUILD to the edk2 build command line.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
When the number of build threads multiplied by per-thread file
descriptor usage exceeds the system's open file descriptor limit,
some threads may fail to acquire necessary resources (e.g., pipes
or semaphores), leading to deadlocks or hangs during parallel builds.
To prevent this situation, calculate the safety upper limit of
concurrency by dividing the system's maximum file descriptor limit by
3 (An empirical value derived from balancing performance overhead
against the theoretical number of file descriptors consumed per thread).
The actual thread count is then clamped to this safe value.
Other usages of ThreadNum()—such as during actual compilation or log
queue creation—do not significantly contribute to file descriptor
consumption. Therefore, adjusting ThreadNum() globally would be
unwarranted, as it could unnecessarily restrict parallelism in stages
that are not FD-bound.
This ensures stable parallel builds even under constrained resource
limits.
Signed-off-by: Ayden Meng <mengxiangdong@loongson.cn>
Previously, when file descriptors were exhausted in high-concurrency
builds (e.g., 512 threads with 1024 FD limit), the build would hang or
fail silently without clear indication of the root cause.
This change catches relevant OSError instances and terminates the build,
ensuring failures due to resource limits are explicit.
Signed-off-by: Ayden Meng <mengxiangdong@loongson.cn>
The DXE_SAL_DRIVER module type was introduced to support
Itanium (IPF) platforms. Since support for Itanium processors has been
dropped, the instances of DXE_SAL_DRIVER have been removed.
Ref: [3cb0a311cb]
Cc: Sachin Ganesh <sachinganesh@ami.com>
Signed-off-by: Sathya Ravichandran <sathyar@ami.com>
Set BUILD_TIME_EPOCH to the current UTC timestamp if not already present
in the environment.
Use the resulting value to print the "Build start time:" message.
Signed-off-by: Leif Lindholm <leif.lindholm@oss.qualcomm.com>
The function CheckEnvVariable in fact checks several environment
variables. And the comment at its invocation enumerates a specific set of
variables, which defeats half the point of abstracting it out into a
helper function.
Rename the function to the plural form and turn the comment into a list
of examples.
Signed-off-by: Leif Lindholm <leif.lindholm@oss.qualcomm.com>
edk2 is dropping support for the ARM32 architecture. This
commit removes ARM32 code from BaseTools.
Signed-off-by: Oliver Smith-Denny <osde@microsoft.com>
Build report would not detect a nested FV if the nested
FV was not in a subsection of an FFS statement.
Modify the build report to better handle some of the
variations of nested FVs.
Failing Example:
[Fv.FvName1]
INF <path to some driver>.inf
[Fv.FvName0]
FILE FV_IMAGE = B25ACDEF-39CE-4FA5-B50A-33E24DB1BDDF {
SECTION FV_IMAGE = FvName1
}
Working Example:
[Fv.FvName1]
INF <path to some driver>.inf
[Fv.FvName0]
FILE FV_IMAGE = DA04F6BF-A0FD-47EC-928B-5101A6C95026 {
SECTION GUIDED EE4E5898-3914-4259-9D6E-DC7BD79403CF
PROCESSING_REQUIRED = TRUE {
SECTION FV_IMAGE = FvName1
}
}
Signed-off-by: Aaron Pop <aaronpop@microsoft.com>
Add GUID section for build report.
Also, change the GUID format to string format to be easier to parse
Signed-off-by: Zhiguang Liu <zhiguang.liu@intel.com>
If an FV_SPARE_SPACE_THRESHOLD is enabled and an FV
is 100% full with 0 bytes free, then this is likely
a special FV that may have alignment requirements
for the FFS file for both the start address and the
length and the FFS file consumes all the available
FV space.
Reduce FV_SPARE_SPACE_THRESHOLD from an error to a
warning if this FV 0 bytes free condition is
detected.
PR #10828 introduced the generation of these error
conditions for an FV with large alignment requirements.
The pad region before the aligned FFS file used to
be counted as free space even though it could never
be used due to the alignment requirements. There was
actually no free space available. PR #10828 fixed the
free space calculation to properly show it as 0 bytes
free, and this change then caused build error when
FV_SPARE_SPACE_THRESHOLD feature was enabled. The
reduction to a warning for this condition allows the
build to complete with errors and also provides a
build log warning message for review.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Refer to the docs of python, `os.path.normcase(path)` function:
"Normalize the case of a pathname. On Windows, convert all characters in
the pathname to lowercase, and also convert forward slashes to backward
slashes. On other operating systems, return the path unchanged."
`os.path.normpath(path)` also convert forward slashes to backward slashes.
So call `os.path.normcase` after `os.path.normpath` just convert path to
lowercase on Windows(only).
And Windows is case-insensitive but case-preserving.
So the usage of `os.path.normcase(os.path.normpath(path))` can be
simplified to `os.path.normpath(path)`. Then we can use case-preserving
paths rather than lowercase paths in compile_commands.json file
or build log.
But this patch continue to use `os.path.normcase`
when comparing/searching paths.
Signed-off-by: Yang Gang <yanggang@byosoft.com.cn>
This patch updates the GenC logic to generate a random stack cookie value
for the stack check libraries. These random values improve security
for modules which cannot update the global intrinsics.
If the stack cookie value is randomized in the AutoGen.h file each
build, the build system will determine the module/library must be
rebuilt causing effectively a clean build every time. This also makes
binary reproducibility impossible.
This patch updates the early build scripts to create 32 and 64-bit JSON
files in the build output directory which each contain 100 randomized
stack cookie values for each bitwidth. If the JSON files are already
present, then they are not recreated which allows them to be stored and
moved to other builds for binary reproducibility. Because they are in
the build directory, a clean build will cause the values to be
regenerated.
The logic which creates AutoGen.h will read these JSON files and use a
hash of the module GUID (the hash seed is fixed in Basetools) to index
into the array of stack cookie values for the module bitwidth. This
model is necessary because there isn't thread-consistent data so we
cannot use a locking mechanism to ensure only one thread is writing to
the stack cookie files at a time. With this model, the build threads
only need to read from the files.
Signed-off-by: Oliver Smith-Denny <osde@linux.microsoft.com>
This produces output that matches CodeChecker log command
- Set directory to build output path
- Set build destination to the object created instead of the path
- Add recursive macro support
- Add lookup in module.Macros dictionary
- Add leading include flag to include list
- Add source file to compile commands
Signed-off-by: Jeff Brasen <jbrasen@nvidia.com>
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2850
Add "-Y REPORT_INFO" option to build command to generate compile
information as part of BuildReport.
This option generates files to be used by external tools as IDE's
to enhance functionality.
Files are created inside build folder:
<Build>/<BuildTarget>/<ToolChain>/CompileInfo
Files created:
* compile_commands.json - Compilation Database. To be used by IDE's
to enable advance features
* cscope.files - List of files used in compilation. Used by Cscope to parse
C code and provide browse functionality.
* module_report.json - Module data form buildReport in Json format.
Signed-off-by: Guillermo Antonio Palomino Sosa <guillermo.a.palomino.sosa@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3653
The "target.txt" and "tools_def.txt" filenames are hard-coded
at some places when global definitions are available at:
BaseTools/Source/Python/Common/TargetTxtClassObject.py:
DefaultTargetTxtFile
and
BaseTools/Source/Python/Common/ToolDefClassObject.py:
DefaultToolsDefFile
Use these global definitions instead.
Also remove the unused gBuildConfiguration and gToolsDefinition
variables from build.py
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Reviewed-by: Chris Jones <christopher.jones@arm.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3653
Running the following command:
python3 build/build.py -a AARCH64 -t GCC5
-p ArmPlatformPkg/ArmPlatformPkg.dsc -b DEBUG libraries
triggers the following error:
make: *** Build/ArmPlatform/DEBUG_GCC5/AARCH64/MdePkg/Library/
BasePcdLibNull/BasePcdLibNull: Is a directory. Stop.
Indeed, MakefileName is set to en empty string. Setting MakefileName
resolves the error.
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Reviewed-by: Chris Jones <christopher.jones@arm.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
threading camelCase functions have preferred alternatives since
python2.6. python3.10 has started emitting DeprecationWarnings
for them
Signed-off-by: Cole Robinson <crobinso@redhat.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
