Commit Graph

2969 Commits

Author SHA1 Message Date
Sergii Dmytruk eb21879679 BaseTools/GenerateCapsule.py: Fix checking for DepExp presence
struct.unpack() returns a tuple even for a single-element pack,
resulting in signature verification being evaluated to false even when
the signature is there.

This fixes --decode and --dump-info actions incorrectly reporting issues
with parsing capsule dependencies when there are none.

Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
2024-07-30 00:37:37 +03:00
Sergii Dmytruk 2374aa6bf9 BaseTools/GenerateCapsule.py: Fix inconsistent error formatting
Just add a space between colon and a more detailed error message in two
places.

Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
2024-07-30 00:37:37 +03:00
Sergii Dmytruk 3a42552fc6 BaseTools/GenerateCapsule.py: Require --output for --decode
--decode unconditionally uses args.OutputFile.name as a prefix for
output files that it creates and fails in a non-pretty way without
--output option.

This doesn't address creation/truncation of the file specified via
--output, but at least you're able to decode a capsule.

Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
2024-07-30 00:37:37 +03:00
Sergii Dmytruk 0f28589389 BaseTools/GenerateCapsule.py: Better error message on --decode failure
Print error text from the exception.

Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
2024-07-30 00:37:37 +03:00
Sergii Dmytruk 55936941e9 BaseTools/GenerateCapsule.py: Disallow UpdateImageIndex == 0 on --encode
This field seems to be one-based according UEFI specification, default
value is 1 and --decode of GenerateCapsule.py errors upon seeing
UpdateImageIndex less than 1.  So align --encode behaviour to enforce a
value within the 1..255 range.

Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
2024-07-30 00:37:37 +03:00
Sergii Dmytruk 2b9d5d09b8 BaseTools/GenerateCapsule.py: Fix --decode operation
Commit b68d566439 added support for
input subject name with signtool and broke --decode operation by using
incorrect identifier in one place (could be an incomplete rename during
review).

It's `args.SignToolSubjectName`, not `args.SignSubjectName`.

Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
2024-07-30 00:37:37 +03:00
Sergii Dmytruk 68769c619d BaseTools/FmpCapsuleHeader.py: Explain error when throwing exceptions
This gives a caller a chance to report a meaningful error to the user.

Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
2024-07-30 00:37:37 +03:00
Michał Kopeć 6d34a3843e Fix compilation with coreboot-sdk:2023-11-24_2731fa619b
Signed-off-by: Michał Kopeć <michal.kopec@3mdeb.com>
2024-06-15 20:33:54 +03:00
Ashraf Ali S e32b58ab5a BaseTools: Remove Duplicate sets of SkuName and SkuId from allskuset
Currently when the platform has many SKUs then allskuset will be having
so many duplicate. and while parsing the allskuset will take longer
time while assigning Pcd.SkuInfoList.
This patch is to eliminate those duplicate entries to reduce the
build time

Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Amy Chan <amy.chan@intel.com>
Cc: Sai Chaganty <rangasai.v.chaganty@intel.com>
Signed-off-by: Ashraf Ali S <ashraf.ali.s@intel.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Amy Chan <amy.chan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2024-02-08 19:27:18 +00:00
devel@edk2.groups.io 8f316e99ec BaseTools: Optimize GenerateByteArrayValue and CollectPlatformGuids APIs
During the Incremental build GenerateByteArrayValue used to generate the
ByteArrayValue even when there is no change in the PCD/VPDs. which is
time consuming API based on the number of PCD/VPDs and SKU IDs.

The optimization is that GenerateByteArrayValue is used to store the
StructuredPcdsData in a JSON file for each of the arch. and during the
Incremental build this API will check, if there is any change in the
Structured PCD/VPDs then rest of the flow remains the same.
if there is no change then it will return the provious build data.

Flow:
during the 1st build StructuredPcdsData.json is not exists,
StructuredPcdsData will be dumped to json file. and it will copy the
output.txt as well.
Note: as the output.txt are different for different Arch, so it will be
stored in the Arch folder.
During the Incremental build check if there is any change in Structured
PCD/VPD. if there is a change in Structured VPD/PCD then recreate the
StructuredPcdsData.json, and rest of the flow remains same.
if there is no change in VPD/PCD read the output.txt and return the data

Unit Test:
Test1: Modified the Structured Pcds default from DEC file. current flow
is executing.
Test2: Override the default value of the PCD from DEC file. current flow
is executing.
Test3: Modified/Override the PCD from DSC file. current flow executing
Test4: Modified/Override the FDF from DSC file. current flow executing
Test5: update the default value from Command Line.current flow executing
Test6: Build without change in PCD in DSC, FDF, DEC and Command Line the
proposed changes will be executing, and the return data remains the same
with and without the changes.
Test7: Build with and without modified the include headers of Structured
PCDs. if there is any change in those Structured PCD header then
current flow will be executed.

With these changes it's helping to save around ~2.5min to ~3.5min of
Incremental build time in my build environment.

Sample PR: https://github.com/tianocore/edk2-basetools/pull/113

Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Amy Chan <amy.chan@intel.com>
Cc: Sai Chaganty <rangasai.v.chaganty@intel.com>
Cc: Digant H Solanki <digant.h.solanki@intel.com>
Signed-off-by: Ashraf Ali S <ashraf.ali.s@intel.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
2024-02-08 04:08:38 +00:00
PierreGondois 056b4bf74b BaseTools/Scripts/PatchCheck.py: Check for Change-id
Code review tools like gerrit might use a 'Change-id' tag to track
the evolution of patches. This tag should be removed before
submitting a patch to the mailing-list.
It has been observed that contributors sometimes forget to remove
this tag. Add a check in PatchCheck.py to automate this.

Also add a '--ignore-change-id' command line parameter to ignore
the above check.

Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Acked-by: Pedro Falcato <pedro.falcato@gmail.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2024-02-04 13:29:15 +00:00
levi.yun 40a45b5a2b Basetools: Include PCD declarations from Library Instance
The patch "[PATCH v3 1/2] StandaloneMmPkg: Make StandaloneMmCpu driver
architecture independent" (https://edk2.groups.io/g/devel/message/109178)
removed ArmPkg/ArmPkg.dec from the Packages section in the
INF file: StandaloneMmPkg/Drivers/StandaloneMmCpu/StandaloneMmCpu.inf

This change was done as part of making the StandaloneMmCpu driver
architecture independent.

Although this change is correct, it results in a side effect
here some platforms that utilise PCDs declared in ArmPkg.dec are
no longer declared.

An example of this issue can be seen when building
edk2-platforms/Platform/ARM/SgiPkg/PlatformStandaloneMm.dsc

$ build -a AARCH64 -t GCC -p Platform/ARM/SgiPkg/PlatformStandaloneMm.dsc
build.py...
/mnt/source/edk2-platforms/Platform/ARM/SgiPkg/PlatformStandaloneMm.fdf(23):
  error F001: PCD (gArmTokenSpaceGuid.PcdFdBaseAddress) used in
  FDF is not declared in DEC files.

As seen above, removing ArmPkg.dec from the Packages section in the
StandAloneMmCpu Driver Inf file triggers build failure.
Although, ArmPkg.dec is included in other Library Instances,
the build system does not include the declarations from
.dec files defined in Library instances.

The build system only includes the PCD declarations from DEC files
that are specified in INF files for Modules (components).

Therefore, extend the build system to include the Packages from
Library Instances so that the PCD declarations from the respective package
DEC files are included.

This patch can be seen on
    https://github.com/LeviYeoReum/edk2/tree/levi/2848_dec_check_on_library

Signed-off-by: levi.yun <yeoreum.yun@arm.com>
Tested-by: Pierre Gondois <pierre.gondois@arm.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
Reviewed-by: Sami Mujawar <sami.mujawar@arm.com>
2024-01-31 10:58:05 +00:00
Joey Vagedes 6c488a2f39 BaseTools: Fix raw strings containing valid escape characters
Fixes raw regex strings that contain valid (and purposeful) escape
characters as they are being treated as individual characters rather
than the single escaped character they represent (i.e. '\t' is being
treated as a '\' and a 't' rather than a single tab character).

Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2024-01-10 13:54:01 +00:00
Jake Garver 5d533bbc27 BaseTools/GenFw: Correct offset when relocating an ADR
When converting ELF to PE/COFF for the AArch64 target, we may encounter
an R_AARCH64_ADR_GOT_PAGE relocation that refers to an ADR instruction
instead of an ADRP instruction. This can happen when the toolchain is
working around Cortex-A53 erratum #843419.  If that's the case, be sure
to calculate the offset appropriately.

This resolves an issue experienced when building a StandaloneMm image
(which is built with -fpie) with stack protection enabled on GCC
compiled with "--enable-fix-cortex-a53-843419". In this case, the linker
may convert an ADRP instruction appearing at an offset of 0xff8 or 0xffc
modulo 4KiB into an ADR instruction, but will leave the original
R_AARCH64_ADR_GOT_PAGE relocation in place. (This is not a bug in the
linker, given that there is no other relocation type that it could
reasonably convert it into)

In this scenario, the following code is being generated by the
toolchain:

    # Load to set the stack canary
    2ffc:	10028020 	adr	x0, 8000 <mErrorString+0x1bc>
    3008:	f940d400 	ldr	x0, [x0, #424]

    # Load to check the stack canary
    30cc:	b0000020 	adrp	x0, 8000 <mErrorString+0x1bc>
    30d0:	f940d400 	ldr	x0, [x0, #424]

GenFw rewrote that to:

    # Load to set the stack canary
    2ffc:	10000480 	adr	x0, 0x308c
    3008:	912ec000 	add	x0, x0, #0xbb0

    # Load to check the stack canary
    30cc:	f0000460 	adrp	x0, 0x92000
    30d0:	912ec000 	add	x0, x0, #0xbb0

Note that we're now setting the stack canary from the wrong address,
resulting in an erroneous stack fault.

After this fix, the offset will be calculated correctly for an ADR and
the stack canary is set correctly. Note that there is a corner case
where this may cause the conversion to fail: if the original GOT entry
is just within -/+ 1 MiB of the reference, but the actual variable it
refers to is not, the resulting offset cannot be represented by the
immediate offset field in a ADR instruction. Given that this issue only
affects PIE executables, which are rare and usually tiny, this is
unlikely to cause problems in practice.

Ref: https://edk2.groups.io/g/devel/topic/102202314

[ardb: expand commit log, add reference]

Signed-off-by: Jake Garver <jake@nvidia.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-12-21 10:08:47 +00:00
Joey Vagedes via groups.io 9f0061a03b BaseTools: Resolve regex syntax warnings
Switches regex patterns to raw text to resolve python 3.12 syntax
warnings in regards to invalid escape sequences, as is suggested by the
re (regex) module in python.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-12-21 00:33:31 +00:00
Rebecca Cran 89705ad6c6 BaseTools: FMMT GuidTool Auto Select Config file Enabling
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4624

Currently, Python FMMT tool does not support automatically
select FMMTConf.ini file which saves GuidTool settings.
This patch supports this features.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>

Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-12-20 22:59:03 +00:00
Yuwei Chen 5b5481526f BaseTools: fixing FMMT ShrinkFv issue
1. FvLength not change issue;
2. FileSystemGuid align with File Size;

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2023-12-11 10:12:52 +00:00
Yuwei Chen 59f0d350a9 BaseTools: FMMT support ELF UPLD parser
FMMT add new function to support the .elf file parsing.
Using '-v' option, the UPLD info will be printed out.

'''
- UNIVERSAL_PAYLOAD_INFO
  - 4 bytes align (BOOLEAN)
    - Identifier
    - SpecRevision
    - Attribute
    - Revision
    - Capability
    - ProducerId
    - ImageId
UPLD Buffer
'''

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2023-12-11 10:12:52 +00:00
Yuwei Chen 9627447625 BaseTools: FMMT replace output file is not generated successfully
For replace function, when target Ffs and new ffs are with
same size, the output file can not be generated successfully.
This patch fixes this issue.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2023-12-11 10:12:52 +00:00
Yuwei Chen b5f5106c1e BaseTools: FMMT replace new free space fixing in replace
In FMMT replace function, when newffs size <= targetffs size,
the new free space is calculated wrong as loss the pad data delta size.
That will cause invalid binary generated.
This patch fixes this issue.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2023-12-11 10:12:52 +00:00
Michael Kubacki 4f99b5fb93 BaseTools/Conf/target.template: Use VS2019 as default tool chain
Updates the default tool chain from VS2015x86 to VS2019.

This is the VS tool chain used in CI and more likely to be installed
on developer's systems. This is used in stuart commands when a
toolchain is not explicitly specified.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
2023-11-29 20:44:12 +00:00
Michael D Kinney 33deaa3b84 BaseTools/Scripts/GetMaintainer: Sort output addresses
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4593

Sort the list of output addresses alphabetically so this
script produces the same output even if the order of patches
in a patch series is modified such that that order of files
processed by this script changes.

Use set() logic instead of OrderedDict to accumulate the
list of unique addresses that are sorted alphabetically.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Leif Lindholm <quic_llindhol@quicinc.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Acked-by: Rebecca Cran <rebecca@bsdio.com>
Reviewed-by: Leif Lindholm <quic_llindhol@quicinc.com>
2023-11-11 02:31:13 +00:00
Michael D Kinney 706811819d BaseTools/Scripts/GetMaintainer: Handle reviewer only case
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4593

If a package only has reviewers and no maintainers, then also
return the <default> maintainers.

In order to detect this case, get_maintainers() is updated to
return maintainers, reviews, and lists separately instead of
a single merged list.  This also allows this module to be used
by other scripts that need to distinguish between maintainers,
reviewers, and lists.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Leif Lindholm <quic_llindhol@quicinc.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Acked-by: Rebecca Cran <rebecca@bsdio.com>
Reviewed-by: Leif Lindholm <quic_llindhol@quicinc.com>
2023-11-11 02:31:13 +00:00
Leif Lindholm 05f3c3f3d0 BaseTools/Scripts/GetMaintainer: refactor internal returns as dicts
To clean up interfaces, change the lookup functions to return dictionaries
rather than multiple values.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Signed-off-by: Leif Lindholm <quic_llindhol@quicinc.com>
Acked-by: Rebecca Cran <rebecca@bsdio.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
2023-11-11 02:31:13 +00:00
Michael D Kinney 1cb580be85 BaseTools/Scripts/GetMaintainer: Simplify logic
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4593

get_section_maintainers() either returns a list with
valid entries or an empty list.  It never returns None.
Simplify logic that accumulates maintainers and lists by
unconditionally appending lists returned from
get_section_maintainers().

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Leif Lindholm <quic_llindhol@quicinc.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Acked-by: Rebecca Cran <rebecca@bsdio.com>
Reviewed-by: Leif Lindholm <quic_llindhol@quicinc.com>
2023-11-11 02:31:13 +00:00