This reverts commit 3fe1d56cc9.
PR https://github.com/tianocore/edk2/pull/11757 introduced a
"Breaking Change" feature for out of tree builds of tools.
This breaking change is blocking testing of edk2-stable202602
due to side effects on building FitGen tool in edk2-platforms.
Revert this feature for the edk2-stable202602 release and
work on this feature after the release.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Main EDK2 build supports out-of-tree builds but BaseTools make process
still creates tools and object files in-tree. In order to make
out-of-tree build support complete move the generated tools and
interim obj files to $WORKSPACE location as well.
This patch also changes the location of BaseTools for in-tree builds
(default behavior when WORKSPACE is not provided before calling
edksetup) to $WORKSPACE/BaseTools/Build/... It may potentially break
external workflows that invoke tools from the default location outside
of the build tool.
Signed-off-by: Oleksandr Tymoshenko <ovt@google.com>
The PCD value defined in module subsections can be added to global PCD
database. Therefore the unsolved expressions, even belongs to the global
scope, can incorrectly refer to the value from module subsection.
This only happens when the referred PCD has no value assignment in the
platform dsc file. Which also should raise an error.
Signed-off-by: Paddy Deng <paddydeng@ami.com>
This commit is the first in a series of multiple commits to fix various
typos in the code, originating mostly from copy&paste over the years.
Most of them only affect documentation and not code.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
The DXE_SAL_DRIVER module type was introduced to support
Itanium (IPF) platforms. Since support for Itanium processors has been
dropped, the instances of DXE_SAL_DRIVER have been removed.
Ref: [3cb0a311cb]
Cc: Sachin Ganesh <sachinganesh@ami.com>
Signed-off-by: Sathya Ravichandran <sathyar@ami.com>
Commit f0a2015373 ("UefiPayloadPkg: Add AARCH64 support") changed
`[Components.X64]` to `[Components.X64, Components.AARCH64]` which
resulted in the following code within that section to not work as
expected (the code wasn't there, just providing a real world example
that uncovered the issue):
[Components.X64, Components.AARCH64]
FmpDevicePkg/FmpDxe/FmpDxe.inf {
...
<PcdsFixedAtBuild>
!include .../...PcdFmpDevicePkcs7CertBufferXdr.inc
...
}
At the same time `[Components.X64]` or even `[Components.AARCH64,
Components.X64]` (notice the swapped order) worked fine for X64 target.
The cause of the issue turned out to be skipping includes inside
`_PostProcess()` method of `DscParser` class. This method processes
list of items stored in a database filled on the first pass through a
DSC file in `Start()` method. One of the fields stored in each row
of a table is link to a parent object (owner). A section like
`[Components.X64, Components.AARCH64]` creates two objects and all of
its subelements are duplicated for both X64 and AARCH64. This was not
happening for !include statement in the example above.
Because `_PostProcess()` contracted a sequence of !include objects
disregarding their owner, it did not create instance for each of the
requested targets. Codewise, `self._ContentIndex` was incremented more
than once, while `__ProcessDirective()` method (invoked indirectly as
`Processer[self._ItemType]()`) queried owner of the current directive
as:
if self._InSubsection:
Owner = self._Content[self._ContentIndex - 1][8]
else: # not taken in this case
This is why order of targets made a difference, only the last was fully
initialized in this case.
An alternative fix is completely dropping merging of !include
directives, but hard to say whether it still has some utility (the code
is complex, hard to follow and barely documented). Safer to keep it, in
the worst case it doesn't do anything now.
Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
Adds support for building the C language BaseTools for Windows using
toolchains based on mingw-w64.
Mingw-w64 is a collection of header files, libraries, and tools that
when combined with a compiler enable development of Windows software.
Mingw-w64 is a fork of the original MinGW (Minimalist GNU for Windows).
Most active development on MinGW has ceased and mingw-w64 is now the
actively maintained successor. Mingw-w64 provides a libc implementation
built on top of Microsoft's UCRT (Universal C Runtime) with all
nessesary compiler bindings needed to support the C++11 feature set.
Modern mingw-w64 development appears to have coalesced around MSYS2,
which produces a distributions of both GCC and LLVM/Clang that use
mingw-w64 to target the Windows OS. This MSYS2 Clang distribution has a
UNIX-like directory layout and includes Windows binaries of GNU Make.
Combined with the open source licensing, MSYS2's Clang distribution is a
highly attractive choice as an alternative Windows SDK for open source
projects such as TianoCore.
If one wishes to use EDK II to build UEFI firmware on the Windows
platform, then the C BaseTools need to be compiled as Windows
applications. This includes the PcdValueInit.exe program, which needs
to be recompiled every time a firmware build is run in order to
regenerate the initial values for structured PCDs. Currently, BaseTools
only supports the Visual C++ toolchain on the Windows platform. The
following new features have been added to enable usage of the toolchains
derived from mingw-w64:
- Fixes to the BaseTools C source code to support the use of a
GCC-style compiler on the Windows OS.
- The GNU Make-style Makefiles for the C BaseTools have been modified
to support Windows. Both GCC + mingw-w64 and Clang + mingw-w64 have
been tested and confirmed to build a working BaseTools.
- BaseTools now supports generating GNU Make-style Makefiles on the
Windows platform for the purpose of building firmware.
- edksetup.bat has been modified to optionally build BaseTools via
mingw-w64. There is no impact to the existing support for Visual C++
and Visual C++ remains the default toolchain.
Usage Instructions:
For the vast majority of users, the only system setup change nessesary
to use a mingw-w64 toolchain is to set the BASETOOLS_MINGW_PATH to the
directory containing the desired mingw-w64 based toolchain.
A new command line argument has been added to edksetup.bat: Mingw-w64
If this command line argument is set, then the script will set the
BASETOOLS_MINGW_BUILD environment variable. The user can also opt to set
this environment variable manually before running edksetup.bat
If BASETOOLS_MINGW_BUILD is defined, then the BASETOOLS_MINGW_PATH
environment variable must point to the directory containing the
mingw-w64 toolchain.
If CLANG_BIN is not defined and %BASETOOLS_MINGW_PATH%\bin\clang.exe
exists, then edksetup.bat will set CLANG_BIN=%BASETOOLS_MINGW_PATH%\bin\
This removes the requirement to configure the CLANG_BIN environment
variable manually in order to run a CLANGPDB or CLANGDWARF build if one
has the MSYS2 Clang distribution installed. If one wishes to use a
different copy of Clang (for example official LLVM binaries) to build
firmware and only use the MSYS2 Clang to build BaseTools, then one can
continue to set the CLANG_BIN environment variable, same as before. I
have tested the MSYS2 Clang distribution against the official LLVM
distribution and can confirm that if the compiler version is the same
the emitted machine code is identical between the two. Interestingly,
the MSYS2 Clang distribution emits the path to the PDB file using "/" as
the path seperator instead of "\". That appears to be the only
difference in output. Therefore, using the MSYS2 Clang distribution to
compile firmware seems a reasonable choice.
If CLANG_HOST_BIN is not defined and BASETOOLS_MINGW_BUILD is defined
and %BASETOOLS_MINGW_PATH%\bin\mingw32-make.exe exists, then
edksetup.bat will add %BASETOOLS_MINGW_PATH%\bin\ to the PATH and set
CLANG_HOST_BIN=mingw32-
This enable usage of the GNU Make included in the mingw-w64 toolchain
to build firmware in addition to BaseTools. if BASETOOLS_MINGW_BUILD is
not defined, edksetup.bat will continue to set CLANG_HOST_BIN=n, which
uses nmake to build firmware. This behavior can be overridden by
manually setting the value of CLANG_HOST_BIN before executing
edksetup.bat if one wishes to use a specific Make utility for the
CLANGPDB/CLANGDWARF toolchains.
References:
- https://www.mingw-w64.org/
- https://www.msys2.org/
Co-authored-by: Sandesh Jain <sandesh.jain@intel.com>
Signed-off-by: Nate DeSimone <nathaniel.l.desimone@intel.com>
BaseTools currently does not expand macros for component architecture when
nested !include directives are used. This breaks use cases like
[Component.$(DXE_ARCH)].
The fix is to add explicit macro expansion when computing the Arch value
for component lines in DscParser.
Signed-off-by: Nate DeSimone <nathaniel.l.desimone@intel.com>
When a module M depends on L1, which depends on L2,
which depends on L3, the build fails when the library instance
of L3 cannot be found according to the library class-instance
mapping configuration specified in the DSC file.
When such failure happens, the build tool only prints that the
instance of L3 required by module M cannot be found. But it
does not tell how L3 is required by M.
The change enhances build tool to print the entire dependency
chain when such failure happens.
With the change, the new error message will be as follows:
<dsc-path>(...): error 4000: Instance of library class [L3] is not
found for module [M], [L3] is:
consumed by <instance of L2>
consumed by <instance of L1>
Signed-off-by: Ray Ni <ray.ni@intel.com>
Update retrieval of private guids, protocols, or ppis from a package's
declaration file to use the original path of the module's INF file
rather than the current path. When building the same module multiple
times in the same INF (by override the define's FILE_GUID), a temporary
instance of the module is generated outside the package, causing the
retrieval of private values to fail as the check to access private
values is done by verifying the module to build, is inside the package.
Signed-off-by: Joey Vagedes <Joey.Vagedes@gmail.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
When collecting the required library instances for modules and
libraries, included libraries will be recursed to ensure the module is
built with all the libraries directly linked to it and indirectly
linked to it via included libraries.
Using the following scenario as an example:
[LibraryClasses.common.DXE_CORE]
NULL|Path/To/Library1.inf // Includes DebugLib
[LibraryClasses.common.DXE_DRIVER]
NULL|Path/To/Library2.inf // Includes DebugLib
[LibraryClasses.common.DXE_CORE, LibraryClasses.common.DXE_DRIVER]
DebugLib|MdePkg/Library/BaseDebugLibSerialPort/BaseDebugLibSerialPort.inf
[Components]
MdeModulePkg/Core/Dxe/DxeMain.inf // Includes DebugLib
The DXE_CORE NULL library will be assigned a fake library class like
NULL1 and the DXE_DRIVER will be assigned NULL2. The recursion logic
will see NULL1 as a directly linked and will add an instance of it to
the list of libraries which need to be included in the module. When
DebugLib is evaluated, the recursion logic will add the libraries
DebugLib depends on to the queue which includes both NULL1 and NULL2.
When NULL2 is unqueued, an instance of it will also be added to the
list of libraries needed to build DxeMain which now means that both
NULL1 and NULL2 have been linked.
NULL includes outside of module overrides are not supported according
to the spec, but we do it anyways so this seems like a case which
should be fixed. This change updates the recursion logic to skip
evaluating NULL libraries unless they are linked directly to the
module/library being evaluated.
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Taylor Beebe <taylor.d.beebe@gmail.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
During the Incremental build GenerateByteArrayValue used to generate the
ByteArrayValue even when there is no change in the PCD/VPDs. which is
time consuming API based on the number of PCD/VPDs and SKU IDs.
The optimization is that GenerateByteArrayValue is used to store the
StructuredPcdsData in a JSON file for each of the arch. and during the
Incremental build this API will check, if there is any change in the
Structured PCD/VPDs then rest of the flow remains the same.
if there is no change then it will return the provious build data.
Flow:
during the 1st build StructuredPcdsData.json is not exists,
StructuredPcdsData will be dumped to json file. and it will copy the
output.txt as well.
Note: as the output.txt are different for different Arch, so it will be
stored in the Arch folder.
During the Incremental build check if there is any change in Structured
PCD/VPD. if there is a change in Structured VPD/PCD then recreate the
StructuredPcdsData.json, and rest of the flow remains same.
if there is no change in VPD/PCD read the output.txt and return the data
Unit Test:
Test1: Modified the Structured Pcds default from DEC file. current flow
is executing.
Test2: Override the default value of the PCD from DEC file. current flow
is executing.
Test3: Modified/Override the PCD from DSC file. current flow executing
Test4: Modified/Override the FDF from DSC file. current flow executing
Test5: update the default value from Command Line.current flow executing
Test6: Build without change in PCD in DSC, FDF, DEC and Command Line the
proposed changes will be executing, and the return data remains the same
with and without the changes.
Test7: Build with and without modified the include headers of Structured
PCDs. if there is any change in those Structured PCD header then
current flow will be executed.
With these changes it's helping to save around ~2.5min to ~3.5min of
Incremental build time in my build environment.
Sample PR: https://github.com/tianocore/edk2-basetools/pull/113
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Amy Chan <amy.chan@intel.com>
Cc: Sai Chaganty <rangasai.v.chaganty@intel.com>
Cc: Digant H Solanki <digant.h.solanki@intel.com>
Signed-off-by: Ashraf Ali S <ashraf.ali.s@intel.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
In https://bugzilla.tianocore.org/show_bug.cgi?id=2842 clang support was
added by having users specify "make CXX=llvm" when building BaseTools.
The Makefile then sees that and sets CC=$(CLANG_BIN)clang and
CXX=$(CLANG_BIN)clang++. That requires that the executables 'clang' and
'clang++' exist and for example aren't named 'clang-17' and
'clang++-17'. Also, it's an unusual way of specifying the compiler,
since many users will expect to be able to override CC and CXX on the
make command line.
Rework the BaseTools Makefiles removing the 'BUILD_' prefix (BUILD_CC
and BUILD_CXX) and using the standard name 'LDFLAGS' instead of
'LFLAGS'. This allows clang to be used by running
'make -C BaseTools CC=clang CXX=clang++'.
Signed-off-by: Rebecca Cran <rebecca@quicinc.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Currently it is not possible to initialize all elements in the
array PCD.
For example, this PCD would result to a build failure:
gTokenSpaceGuid.PcdArray|{0x11, 0x22}|UINT8[2]|0x4C4CB9A3
Correct logical operator in the initialization data size checks to
fix the issue.
Signed-off-by: Konstantin Aladyshev <aladyshev22@gmail.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
This patch is a bug fix about FeatureFlagExpression in INF file:
INF [Source] section now unconditionally use Pcd default value in DEC
when handling FeatureFlagExpression, it is wrong.
If a Pcd value has been set in the DSC file, we should use latest
value in DSC instead of default value.
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: Yi Li <yi1.li@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Currently BaseTools use 'cp' command for PcdValueInit and GenMake
process, as the command can not keep the time info of the source
file, which will cause incremental build issue in Linux system,
thus the '-p' need be added to keep the source file's attributes
in copy process.
This patch fixes this issue.
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
