Commit Graph

436 Commits

Author SHA1 Message Date
Philippe Mathieu-Daude 490a62beb7 BaseTools: Avoid "is" with a literal Python 3.8 warnings
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2304

The following statement produces a SyntaxWarning with Python 3.8:

  if str(FdRegion.RegionType) is 'FILE' and self.Platform.VpdToolGuid in \
    str(FdRegion.RegionDataList):
  BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py:168: SyntaxWarning: \
    "is" with a literal. Did you mean "=="?

Change the 'is' operator by the conventional '==' comparator.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Philippe Mathieu-Daude <philmd@redhat.com>
2019-12-06 03:07:37 +00:00
Fan, ZhijuX bf1ea933ec BaseTools:Add [packages] section in dsc file
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2270

Currently a PCD (e.g. FeaturePCD) cannot be used in a conditional
statement in a DSC/FDF file without a module in the build referencing
the PCD package DEC file.

An example implementation that to support this is to allow a [Packages]
section in the DSC file to list additional package dependencies for PCD
references in the package DSC/FDF files.

this patch is going to  add the ability to have the [packages] section
defined in the DSC file

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Acked-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-11-20 07:46:42 +00:00
Lin, Derek (HPS SW) b1c6e9f55e BaseTools: Fix an incremental build issue caused by macro in #include
When c/h file use macro after #include, for example,
In this case, GenMake is not able to create a healthy dependency for the c
file. GenMake used to add $(FORCE_REBUILD) dependency in the c file, this
guarantee the c file is always compiled in incremental build. But, this
function is broken since 05217d210e which
enable /MP for MSVC compiler, in order to compile multiple c files in one
command multi-processing. The fix here is adding '$(FORCE_REBUILD)' back to
retain the original function.

Line number 1728 and 978 are the code pieces which handle this logic.

Signed-off-by: Derek Lin <derek.lin2@hpe.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-10-18 08:30:38 +08:00
Leif Lindholm c1bcb2258c BaseTools: add missing newlines at end of files
Some scripts in Source/Python were missing newlines at end of files,
so add them.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
2019-10-04 11:18:22 +01:00
Kubacki, Michael A 61af5f2494 BaseTools: Fix GenMake multi-workspace failure
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2232

Commit 0075ab2cec introduced an issue that causes an exception
when multiple workspace packages paths are specified. For example,
if edk2-platforms is used, the root directory will contain an edk
and edk2-platforms directory representing the respective
repositories.

In GenMake, the path to the package DEC file for a module is
discovered by getting the relative path of the INF to the
workspace root directory. Each directory in the relative path
is incrementally joined to the WORKSPACE directory. The file
list in the joined path is searched for a DEC file.

As an example, if the build command is used on a package outside
the edk2 repository, the INF file path is relative to the
edk2-platforms directory not edk2. This causes directory paths
to be built that do not exist. Commit 0075ab2cec replaced the
os.path.exists() call with a try except block that always fails
when os.listdir() is invoked to enumerate the list of files in
the built directory path on packages outside edk2.

This commit restores the original conditional statement which
avoids calling os.listdir() with an invalid directory path.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Signed-off-by: Michael Kubacki <michael.a.kubacki@intel.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by:  Bob Feng <bob.c.feng@intel.com>
2019-10-04 11:13:37 +08:00
Feng, Bob C 1f5e4d9128 BaseTools: Fix the lib order in static_library_files.lst
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2216

This patch is going to fix the lib order in static_library_files.lst.
This issue is introduced by commit 673d09a2dd.
Before 673d09a2dd, build tool apply build rule for the module's library
firstly and then apply build rule for module itself. Now, build tool
apply build rule for module self and then for its library. That behavior
impact the lib order in static_library_files.lst.

This patch is to call module's LibraryAutoGenList function
explicitly, where the applying build rule action for
library is triggered.

Cc: Liming Gao <liming.gao@intel.com>
Cc: dann frazier <dann.frazier@canonical.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Tested-by: dann frazier <dann.frazier@canonical.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-29 11:59:31 +08:00
Feng, Bob C 4747b92cab BaseTools: init DependencyHeaderFileSet for each MakeFile class
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2203

This patch is going to fix a regression issue that
build breaks when the custom makefile exist.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-24 10:29:09 +08:00
Liming Gao fcdedafd97 Revert "BaseTools: Improve GetDependencyList function"
This reverts commit bc9e4194cf.
This change causes the dependent header files are missing in Makefile.
It makes the incremental build not work. So, revert this change.

Cc: Bob Feng<bob.c.feng@Intel.com>
Signed-off-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng<bob.c.feng@Intel.com>
2019-09-21 00:25:48 +08:00
Feng, Bob C 0075ab2cec BaseTools: remove unnecessary calls of os.exist
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2101

This patch is going to remove unnecessary calls
of os.exist()

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-17 10:19:05 +08:00
Feng, Bob C bc9e4194cf BaseTools: Improve GetDependencyList function
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2102

GetDependencyList get the header file via
re.findall in the whole header file.

This patch is to pre-process the header file and
to feed the shorter string to re.findall.

This patch is to improve GetDependencyList() efficiency

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-17 10:19:05 +08:00
Fan, ZhijuX e32f7bc96d BaseTools:change some incorrect parameter defaults
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1858

for Dict={},There are pitfalls in the way this default parameter is set
and Dict is not used in functions, other functions have these two cases,
I will change some incorrect parameter defaults

This patch is going to fix this issue

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-09-17 10:18:51 +08:00
Feng, Bob C 7809492c10 BaseTools: Enable --genfds-multi-thread to default build
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1302

This patch enable --genfds-multi-thread to default build.
This patch keep --genfds-multi-thread build option for
compatibility and also add a new build option to disable
genfds-multi-thread as --no-genfds-multi-thread.

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-11 09:43:59 +08:00
Feng, Bob C e3c8311f8a BaseTools: Fixed the bug of multi-thread genffs for override inf
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1302
If there is a inf override, and multi-thread genffs is enabled, build
will fail.

This patch is going to fix this bug.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-11 09:43:58 +08:00
Feng, Bob C 000ab98574 BaseTools: Sort Pcd settings to make PcdTokenNumber be fixed
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2147

This patch is to sort the Pcd settings so that PcdTokenNumber
will not change if the platform's Pcd settings are the same.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-10 08:39:19 +08:00
Feng, Bob C 373298ca0d BaseTools: Fixed issue for IgnoreAutoGen
https://bugzilla.tianocore.org/show_bug.cgi?id=2080

This patch is to improve build -u option to re-use
GlobalVar_<platformguid>_<arch>.bin file which is
introduced by multiple-process-autogen feature.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Acked-by: Liming Gao <liming.gao@intel.com>
2019-09-10 08:39:18 +08:00
Shi, Steven 40db176d9d BaseTools: Support more file types in build cache
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1958

Current build cache does not store and restore all
types file of Hii/vfr, version and dpx. This patch
adds more file types to support them in build cache.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-02 12:17:56 +08:00
Feng, Bob C df43ea6c08 BaseTools: fix an incremental build issue.
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2137

This patch is going to fix a regression issue of incremental
build which was introduced by the commit 94459080c.
The changing on INF file will not trigger module rebuild.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Tested-by: Michael D Kinney <michael.d.kinney@intel.com>
Acked-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-08-29 11:07:48 +08:00
Feng, Bob C abc0155b03 BaseTools: Incorrect error message for library instance not found
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2099
This is a regression issue introduced by commit e8449e.

This patch is to fix this issue.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-08-23 11:26:24 +08:00
Shi, Steven 94459080c1 BaseTools: Improve the file saving and copying reliability
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2079

The Basetool CopyFileOnChange() and SaveFileOnChange()
functions might raise the IOError occasionally when build
in Windows with multi-process and build cache enabled.
The CopyFileOnChange() and SaveFileOnChange() might be invoked
in multiple sub-processes simultaneously, and this patch adds
global locks to sync these functions invoking which can
harden their reliability.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-08-20 14:06:07 +08:00
Shi, Steven d01a998612 BaseTools: Add GenFds multi-thread support in build cache
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1923

Fix the issue that the GenFds multi-thread will build fail
if enable the build cache together.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-08-20 14:06:07 +08:00
Shi, Steven c340c5bd1b BaseTools: Change the [Arch][Name] module key in Build cache
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1951

Current build cache use the module's [self.Arch][self.Name]
info as the ModuleAutoGen object key in hash list and dictionary.
The [self.Arch][self.Name] is not safe as the module key because
there could be two modules with same module name and arch name in
one platform. E.g. A platform can override a module or library
instance in another different path, the overriding module can has
the same module name and arch name as the original one.
Directly use the ModuleAutoGen obj self as the key, because
the obj __hash__ and __repr__ attributes already contain the
full path and arch name.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-08-20 14:06:06 +08:00
Shi, Steven 56c786b0a9 BaseTools: Print first cache missing file for build cachle
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1925

When a module build cache miss, add support to print the first
cache missing file path and name.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-08-20 14:06:06 +08:00
Shi, Steven 0e7e7a264c BaseTools: Improve the cache hit in the edk2 build cache
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1927

Current cache hash algorithm does not parse and generate
the makefile to get the accurate dependency files for a
module. It instead use the platform and package meta files
to get the module depenedency in a quick but over approximate
way. These meta files are monolithic and involve many redundant
dependency for the module, which cause the module build
cache miss easily.
This patch introduces one more cache checkpoint and a new
hash algorithm besides the current quick one. The new hash
algorithm leverages the module makefile to achieve more
accurate and precise dependency info for a module. When
the build cache miss with the first quick hash, the
Basetool will caculate new one after makefile is generated
and then check again.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-08-20 14:06:05 +08:00
Feng, Bob C 76e12fa334 BaseTools: Fixed issue of incorrect Module Unique Name
https://bugzilla.tianocore.org/show_bug.cgi?id=2088
If there are more than one override instance for a same module,
the Module Unique Name is generated incorrectly.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-08-19 09:06:14 +08:00
Rodriguez, Christian 82407bd129 BaseTools: Fix checking for Sources section in INF file
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1804

The check to see if [Sources] section lists all the header type
files of a module is missing the exclusion of source files that
fall under the scope of Package includes. This change adds the
exclusions.

Signed-off-by: Christian Rodriguez <christian.rodriguez@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Tested-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-08-14 10:59:08 +08:00