The CParser4 Python parser files (CLexer.py, CParser.py, CListener.py)
were generated 7 years ago with ANTLR 4.7.1.
Meanwhile, pip-requirements.txt pins antlr4-python3-runtime to version
4.9 in commit 4a7dd50, but the files were patched, not fully
regenerated. This version mismatch could result in failures when
running against non-trivial C code.
This change regenerates the CParser4 files with ANTLR 4.9 to resolve
the version mismatch. It also updates import statements to correctly
reference Eot instead of Ecc.
Steps used to regenerate the files:
1. Download the ANTLR 4.9 complete tool JAR:
- `https://www.antlr.org/download/antlr-4.9-complete.jar`
2. Generate Python3 parser files
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
The CParser4 Python parser files (CLexer.py, CParser.py, CListener.py)
were generated 7 years ago with ANTLR 4.7.1.
Meanwhile, pip-requirements.txt pins antlr4-python3-runtime to version
4.9 in commit 4a7dd50, but the files were patched, not fully
regenerated. This version mismatch produced two failures when running
EccMain.py against non-trivial C code:
1. A runtime warning on every file parsed:
"ANTLR runtime and generated code versions disagree: 4.9!=4.7.1"
2. A crash when parsing complex C constructs that exercise the
struct/union definition rule in CParser.py:
TypeError: '<' not supported between instances of 'tuple' and 'int'
This occurs in antlr4/BufferedTokenStream.py getText() because the
4.9 runtime changed the expected argument types for that method,
and the 4.7.1-generated parser was passing a tuple where an int is
now required.
This change regenerates the CParser4 files with ANTLR 4.9 to resolve
the version mismatch.
Steps used to regenerate the files:
1. Download the ANTLR 4.9 complete tool JAR:
- `https://www.antlr.org/download/antlr-4.9-complete.jar`
2. Generate Python3 parser files from the grammar:
```
java -jar antlr-4.9-complete.jar `
-Dlanguage=Python3 -visitor `
-o BaseTools/Source/Python/Ecc/CParser4_new `
BaseTools/Source/Python/Ecc/CParser4/C.g4
```
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Add include guards back to include files that use the same
include guard macro in BaseTools/Source/C/Include/Common
and MdePkg or MdeModulePkg.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Update makefile rules to run antlr and dlg to completion
before compiling any of the generated cpp files.
Without this change, parallel make may start compiling some
of the cpp files before both antlr and dlg have finished
which produces syntax errors from compilation with partially
generated files.
Also use &: so the targets are treated as a group and the
rule is only executed once for the entire group. Without
this change, parallel make may run the rule actions more
than once and modify the output while it is being used by
another rule.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Since
ae83c6b7fd
-Wno-unused-but-set-variable
-Wno-unused-const-variable
-Wno-unused-variable
warning suppression is no longer needed in any builds, and the
warnings can be re-enabled to catch real errors.
Signed-off-by: Mike Beaton <mjsbeaton@gmail.com>
Replace traditional `#ifndef`/`#define`/`#endif` include guards with
`#pragma` once.
`#pragma once` is a widely supported preprocessor directive that
prevents header files from being included multiple times. It is
supported by all toolchains used to build edk2: GCC, Clang/LLVM, and
MSVC.
Note: Some files in BaseTools are excluded from the change if they
are autogenerated or direcly related to a header from a subproject,
etc. In particular, headers in these directories were ignored:
- BaseTools/Source/C/LzmaCompress/Sdk/
- BaseTools/Source/C/VfrCompile/Pccts/
Compared to macro-based include guards, `#pragma once`:
- Eliminates the risk of macro name collisions or copy/paste errors
where two headers inadvertently use the same guard macro.
- Eliminate inconsistency in the way include guard macros are named
(e.g., some files use `__FILE_H__`, others use `FILE_H_`, etc.).
- Reduces boilerplate (three lines replaced by one).
- Avoids polluting the macro namespace with guard symbols.
- Can improve build times as the preprocessor can skip re-opening the
file entirely, rather than re-reading it to find the matching
`#endif` ("multiple-include optimization").
- Note that some compilers may already optimize traditional include
guards, by recognzining the idiomatic pattern.
This change is made acknowledging that overall portability of the
code will technically be reduced, as `#pragma once` is not part of the
C/C++ standards.
However, this is considered acceptable given:
1. edk2 already defines a subset of supported compilers in
BaseTools/Conf/tools_def.template, all of which have supported
`#pragma once` for over two decades.
2. There have been concerns raised to the project about inconsistent
include guard naming and potential macro collisions.
Approximate compiler support dates:
- MSVC: Supported since Visual C++ 4.2 (1996)
- GCC: Supported since 3.4 (2004)
(http://gnu.ist.utl.pt/software/gcc/gcc-3.4/changes.html)
- Clang (LLVM based): Since initial release in 2007
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
The codebase has moved from traditional `#ifndef` include guards to
`#pragma once`. Remove the ECC checks that validated include guard
presence and naming conventions since they are no longer applicable.
The following checks are removed:
- IncludeFileCheckIfndefStatement: Verified all header file contents
were guarded by a `#ifndef` statement, that the `#ifndef` was the
first line of code after the file header comment, and that the
`#endif` appeared on the last line.
- NamingConventionCheckIfndefStatement: Verified that the `#ifndef`
guard name at the start of an include file used a postfix underscore
and no prefix underscore character.
Also removed related error codes and configuration settings that were
specific to these checks.
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
The C preprocessor turns each .vfr file into a pre-processed .i
file. At this step, the C preprocessor processes `#pragma once`.
Then, VfrCompile is called (with `-n` to prevent preprocessing)
to parse the pre-processed .i files.
The .i files may still contain `#pragma once` lines. Currently,
VfrCompile treats `once` as an unknown token, causing parse failures.
Originally, this change was going to add a `PragmaOnce` token rule
to the VFR lexer grammar (in VfrSyntax.g) that matched `#pragma once`
lines and silently skipped them using `skip()` and `newline()`. The
`newline()` call would keep line numbers stable for error reporting.
This was consistent with how other preprocessor artifacts were already
handled like `#line` directives (`LineDefinition` and
`GccLineDefinition` tokens) and `extern` declarations (skipped with
`mode(CPP_COMMENT)`).
Writing a regular expression to match `#pragma once` was simple
enough, but it makes overall pragma token recognition more fragile
at the lexer level. When the lexer is walking the DFA state table,
it could begin to match a `#pragma ` line but then not be able to
match remaining characters to recognize tokens other than `once`.
Instead, this change handles `#pragma once` lines in the VFR parser
grammar in `vfrPragmaDefinition` alongside where `pack` is already
handled.
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
This reverts commit 3fe1d56cc9.
PR https://github.com/tianocore/edk2/pull/11757 introduced a
"Breaking Change" feature for out of tree builds of tools.
This breaking change is blocking testing of edk2-stable202602
due to side effects on building FitGen tool in edk2-platforms.
Revert this feature for the edk2-stable202602 release and
work on this feature after the release.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
This reverts commit f0542ae07d.
PR https://github.com/tianocore/edk2/pull/11757 introduced a
"Breaking Change" feature for out of tree builds of tools.
This breaking change is blocking testing of edk2-stable202602
due to side effects on building FitGen tool in edk2-platforms.
Revert this feature for the edk2-stable202602 release and
work on this feature after the release.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Use $(SEP) with addprefix of $(OBJDIR) to support Windows MINGW
CLANG builds that use Windows path separators with GNU makefiles.
This fixes Windows MINGW CLANG builds of the PcdValueInit
application that is required for structured PCDs.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Main EDK2 build supports out-of-tree builds but BaseTools make process
still creates tools and object files in-tree. In order to make
out-of-tree build support complete move the generated tools and
interim obj files to $WORKSPACE location as well.
This patch also changes the location of BaseTools for in-tree builds
(default behavior when WORKSPACE is not provided before calling
edksetup) to $WORKSPACE/BaseTools/Build/... It may potentially break
external workflows that invoke tools from the default location outside
of the build tool.
Signed-off-by: Oleksandr Tymoshenko <ovt@google.com>
- Move common wrapper logic for multiple tools to GenericShellWrapper
file
- Drop search for BaseToolsCBinaries from the list of checked
directories. This case has been broken for quite a while since the
exec clause never passed command arguments to the supposed binary.
- Remove a suggestion to run make in $EDK_TOOLS_PATH/Source/C and let
the wrapper fail if the directory exists but no binary is present.
Signed-off-by: Oleksandr Tymoshenko <ovt@google.com>
VfrLexer.h is built as a part of VfrCompile build and
shouldn't be present at the BaseTools/Source/C level.
Signed-off-by: Oleksandr Tymoshenko <ovt@google.com>
As a preparation for out-of-tree build support make sure all interim
files for tests are created in tmp directory.
Signed-off-by: Oleksandr Tymoshenko <ovt@google.com>
SEP variable is only set if PYTHON_COMMAND is undefined but referred
regardless. It breaks the clean target for the following scenario:
. edksetup.sh # Sets PYTHON_COMMAND
make -C BaseTools
make -C BaseTools clean
This change fixes the use case mentioned above by settings SEP
for all build configurations.
[ardb: Drop redundant assignment for Windows]
Signed-off-by: Oleksandr Tymoshenko <ovt@google.com>
New warning after updating gcc:
EfiRom.c: In function ‘main’:
EfiRom.c:78:17: error: assignment discards ‘const’ qualifier from pointer target type [-Werror=discarded-qualifiers]
The assigned value is not used, so fix the warning by just removing it.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
StringFuncs.c: In function ‘SplitStringByWhitespace’:
StringFuncs.c:113:15: error: variable ‘Item’ set but not used [-Werror=unused-but-set-variable=]
113 | UINTN Item;
| ^~~~
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
