During the Incremental build GenerateByteArrayValue used to generate the
ByteArrayValue even when there is no change in the PCD/VPDs. which is
time consuming API based on the number of PCD/VPDs and SKU IDs.
The optimization is that GenerateByteArrayValue is used to store the
StructuredPcdsData in a JSON file for each of the arch. and during the
Incremental build this API will check, if there is any change in the
Structured PCD/VPDs then rest of the flow remains the same.
if there is no change then it will return the provious build data.
Flow:
during the 1st build StructuredPcdsData.json is not exists,
StructuredPcdsData will be dumped to json file. and it will copy the
output.txt as well.
Note: as the output.txt are different for different Arch, so it will be
stored in the Arch folder.
During the Incremental build check if there is any change in Structured
PCD/VPD. if there is a change in Structured VPD/PCD then recreate the
StructuredPcdsData.json, and rest of the flow remains same.
if there is no change in VPD/PCD read the output.txt and return the data
Unit Test:
Test1: Modified the Structured Pcds default from DEC file. current flow
is executing.
Test2: Override the default value of the PCD from DEC file. current flow
is executing.
Test3: Modified/Override the PCD from DSC file. current flow executing
Test4: Modified/Override the FDF from DSC file. current flow executing
Test5: update the default value from Command Line.current flow executing
Test6: Build without change in PCD in DSC, FDF, DEC and Command Line the
proposed changes will be executing, and the return data remains the same
with and without the changes.
Test7: Build with and without modified the include headers of Structured
PCDs. if there is any change in those Structured PCD header then
current flow will be executed.
With these changes it's helping to save around ~2.5min to ~3.5min of
Incremental build time in my build environment.
Sample PR: https://github.com/tianocore/edk2-basetools/pull/113
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Amy Chan <amy.chan@intel.com>
Cc: Sai Chaganty <rangasai.v.chaganty@intel.com>
Cc: Digant H Solanki <digant.h.solanki@intel.com>
Signed-off-by: Ashraf Ali S <ashraf.ali.s@intel.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
In https://bugzilla.tianocore.org/show_bug.cgi?id=2842 clang support was
added by having users specify "make CXX=llvm" when building BaseTools.
The Makefile then sees that and sets CC=$(CLANG_BIN)clang and
CXX=$(CLANG_BIN)clang++. That requires that the executables 'clang' and
'clang++' exist and for example aren't named 'clang-17' and
'clang++-17'. Also, it's an unusual way of specifying the compiler,
since many users will expect to be able to override CC and CXX on the
make command line.
Rework the BaseTools Makefiles removing the 'BUILD_' prefix (BUILD_CC
and BUILD_CXX) and using the standard name 'LDFLAGS' instead of
'LFLAGS'. This allows clang to be used by running
'make -C BaseTools CC=clang CXX=clang++'.
Signed-off-by: Rebecca Cran <rebecca@quicinc.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Currently it is not possible to initialize all elements in the
array PCD.
For example, this PCD would result to a build failure:
gTokenSpaceGuid.PcdArray|{0x11, 0x22}|UINT8[2]|0x4C4CB9A3
Correct logical operator in the initialization data size checks to
fix the issue.
Signed-off-by: Konstantin Aladyshev <aladyshev22@gmail.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Currently BaseTools use 'cp' command for PcdValueInit and GenMake
process, as the command can not keep the time info of the source
file, which will cause incremental build issue in Linux system,
thus the '-p' need be added to keep the source file's attributes
in copy process.
This patch fixes this issue.
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
The generated Makefile was missing a dependency. This resulted in a
build-time race condition if the recursive make is multi-threaded and
shares job control.
Signed-off-by: Jake Garver <jake@nvidia.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3828
This is a bugfix of
bf9230a9f3.
1.In the current code, gPlatformFinalPcd will save all PCDs used at
whole compile process, which wastes runtime memory and is unnecessary.
This patch makes gPlatformFinalPcd save only the PCDes which are
assigned in the DSC file, and the PCD that has not been assigned will
use the default value in DEC.
2.During the compilation process, gPlatformFinalPcd may be lost, and
the current code cannot selectively assign PCD in DSC by specifying ARCH.
This patch moves gPlatformFinalPcd into datapipe and modifies the
assignment logicto fix this.
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: yi1 li <yi1.li@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
For the multiple SKU case, basetools generates multiple redundant
__FLEXIBLE_SIZE statement that is from the same line in dsc file.
This behavior causes the generated PcdValueInit.c file to have huge size,
and the compilation time is very long.
This patch is going to fix this issue.
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
Fix a Edk2Logger.warn() message format to match the arguments.
We ran into this after a failure in PcdValueInit. The failure was
masked by a new exception, "TypeError: not all arguments converted
during string formatting".
Signed-off-by: Jake Garver <jake@nvidia.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
In order to support secure boot with authenticated type variable store
and non secure boot with normal type variable store, add one flag to
switch them.
User can append '-D VPD_AUTHENTICATED_VARIABLE_STORE' to build command
to enable authenticated type varaible store.
Also, user can add 'VPD_AUTHENTICATED_VARIABLE_STORE = TRUE/FALSE' to the
defines section of Dsc file to switch authenticated/normal type variable
store.
VPD_AUTHENTICATED_VARIABLE_STORE is a new reserved key word for this function.
Signed-off-by: Chen Lin Z <lin.z.chen@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3653
The "target.txt" and "tools_def.txt" filenames are hard-coded
at some places when global definitions are available at:
BaseTools/Source/Python/Common/TargetTxtClassObject.py:
DefaultTargetTxtFile
and
BaseTools/Source/Python/Common/ToolDefClassObject.py:
DefaultToolsDefFile
Use these global definitions instead.
Also remove the unused gBuildConfiguration and gToolsDefinition
variables from build.py
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Reviewed-by: Chris Jones <christopher.jones@arm.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
If VPD PcdNvStoreDefaultValueBuffer is used, all DynamicHii and
DynamicExHii PCD value will be generated into that VPD.
In order to generate the same VPD binary file in every build,
sort the Pcd set when generating VPD.
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2365
This patch is to fix a build tool regression issue which was introduced
by commit 8ddec24dea.
compiler output message includes localized string.
So build failed when code decode the stdout/stderr byte arrays.
The cause of the build failed is that Commit 8ddec24dea
removed "errors='ignore'".
The build tool does not need to deal with localized string,
so we need to add "errors='ignore'".
this function is only invoked for structure PCDs.
Build failed if structurePcd is used in platform dsc file.
The patch is going to fixed this issue
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
https://bugzilla.tianocore.org/show_bug.cgi?id=2496
Reduce the build and env dependencies for the Structured PCD
application by removing the dependency on Common.lib that
is only built when BaseTools is built which does not
happen if pre-compiled BaseToools are used. Change the
makefile for the Structure PCD application to build all
files from sources which adds PcdValueCommon.c to the
makefile. Also remove PcdValueCommon.c from Common.lib.
With the change to the makefile for the Structured PCD
application, multiple C files are compiled. Only
PcdValueInit.c contains the extra information expected
by the error/warning message parser. Only parse the
DSC line number into an error message if there is an
error/warning in PcdValueInit.c. Errors/warnings in
other files should be passed through. This fixes a build
failure with no useful log information that was observed
when there was a compiler error in PcdValueCommon.c.
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
https://bugzilla.tianocore.org/show_bug.cgi?id=2494
When using structured PCDs, a C application is auto generated
to fill in the structured PCD value. The C application uses
the standard include files <stdio.h>, <stdlib.h>, and <string.h>.
This C application also supports include paths from package DEC
files when a structured PCD declaration provides a <Packages>
list. The complete list of include paths are -I options for
include paths from package DEC files and the compiler's standard
include paths.
-I include paths are higher priority than the standard include
paths. If the -I included paths from package DEC files contain
<stdio.h>, <stdlib.h>, or <string.h> the wrong include files are
used to compile the C application for the structured PCD value.
Update GenerateByteArrayValue() to skip a package DEC include
paths that contain <stdio.h>, <stdlib.h>, or <string.h>.
Build failures were observed when adding a structured PCD to
CryptoPkg. CryptoPkg contains <stdio.h>, <stdlib.h>, and
<string.h> in the path CryptoPkg/Library/Include to support
building Open SSL. The Library/Include path is listed as a
private include path in CryptoPkg.dec. Without this change, the
standard include files designed to support build OpenSLL are
used to build the structured PCD C application, and that build
fails.
Other packages that provide a standard C lib or a gasket for
a subset of the standard C lib will run into this same issue
if they also define and use a Structured PCD. So this issue
is not limited to the CryptoPkg.
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2455
BuildOption is used by TargetTxtClassObj.py
GenFdsOption is used by GenFds.py
When the GenFds tool is used alone (e.g. python3 -m GenFds.GenFds -h)
With the OptionParser function, the first detected function
prints the help message
import TargetTxtClassObj to GenFds,
The BuildOption will be executed and replace GenFdsOption
We removed all objects associated with this problem that
were created directly during the import process
(e.g. BuildOption, BuildTarget = MyOptionParser(),
TargetTxt = TargetTxtDict())
The Patch is going to fix this issue
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2270
Currently a PCD (e.g. FeaturePCD) cannot be used in a conditional
statement in a DSC/FDF file without a module in the build referencing
the PCD package DEC file.
An example implementation that to support this is to allow a [Packages]
section in the DSC file to list additional package dependencies for PCD
references in the package DSC/FDF files.
this patch is going to add the ability to have the [packages] section
defined in the DSC file
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Acked-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
