Commit Graph

879 Commits

Author SHA1 Message Date
Eric V. Smith
0275e0452a Fix trivial typo in the PEG string parser (GH-21508) 2020-07-16 17:10:23 +01:00
Benjamin Peterson
2ad7e9c011 Fix possibly-unitialized warning in string_parser.c. (GH-21503)
GCC says
```
../cpython/Parser/string_parser.c: In function ‘fstring_find_expr’:
../cpython/Parser/string_parser.c:404:93: warning: ‘cols’ may be used uninitialized in this function [-Wmaybe-uninitialized]
  404 |     p2->starting_col_offset = p->tok->first_lineno == p->tok->lineno ? t->col_offset + cols : cols;
      |                               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~
../cpython/Parser/string_parser.c:384:16: note: ‘cols’ was declared here
  384 |     int lines, cols;
      |                ^~~~
../cpython/Parser/string_parser.c:403:45: warning: ‘lines’ may be used uninitialized in this function [-Wmaybe-uninitialized]
  403 |     p2->starting_lineno = t->lineno + lines - 1;
      |                           ~~~~~~~~~~~~~~~~~~^~~
../cpython/Parser/string_parser.c:384:9: note: ‘lines’ was declared here
  384 |     int lines, cols;
      |         ^~~~~
```

and, indeed, if `PyBytes_AsString` somehow fails, lines & cols will not be initialized.
2020-07-16 08:07:29 -05:00
Lysandros Nikolaou
782f44b8fb bpo-41215: Make assertion in the new parser more strict (GH-21364) 2020-07-07 01:42:21 +03:00
Pablo Galindo
1ac0cbca36 bpo-41215: Don't use NULL by default in the PEG parser keyword list (GH-21355)
Automerge-Triggered-By: @lysnikolaou
2020-07-06 12:31:16 -07:00
Victor Stinner
1f76453173 bpo-41204: Fix compiler warning in ast_type_init() (GH-21307) 2020-07-04 23:18:15 +02:00
Victor Stinner
b1cc6ba73a bpo-41194: Convert _ast extension to PEP 489 (GH-21293)
Convert the _ast extension module to PEP 489 "Multiphase
initialization". Replace the global _ast state with a module state.
2020-07-03 20:01:46 +02:00
Victor Stinner
91e1bc18bd bpo-41194: The _ast module cannot be loaded more than once (GH-21290)
Fix a crash in the _ast module: it can no longer be loaded more than
once. It now uses a global state rather than a module state.

* Move _ast module state: use a global state instead.
* Set _astmodule.m_size to -1, so the extension cannot be loaded more
  than once.
2020-07-03 14:15:53 +02:00
Victor Stinner
74419f0c64 bpo-41194: Pass module state in Python-ast.c (GH-21284)
Rework asdl_c.py to pass the module state to functions in
Python-ast.c, instead of using astmodulestate_global.

Handle also PyState_AddModule() failure in init_types().
2020-07-03 11:35:37 +02:00
Guido van Rossum
9d197c7d48 bpo-35975: Only use cf_feature_version if PyCF_ONLY_AST in cf_flags (#21021) 2020-06-27 17:33:49 -07:00
Lysandros Nikolaou
1f0f4abb11 bpo-41076: Pre-feed the parser with the f-string expression location (GH-21054)
This commit changes the parsing of f-string expressions with the new parser. The parser gets pre-fed with the location of the expression itself (not the f-string, which was what we were doing before). This allows us to completely skip the shifting of the AST nodes after the parsing is completed.
2020-06-28 00:41:48 +01:00
Batuhan Taskaya
c8f29ad986 bpo-40769: Allow extra surrounding parentheses for invalid annotated assignment rule (GH-20387) 2020-06-27 19:33:08 +01:00
Lysandros Nikolaou
6dcbc2422d bpo-41132: Use pymalloc allocator in the f-string parser (GH-21173) 2020-06-27 18:47:00 +01:00
Lysandros Nikolaou
2e0a920e9e bpo-41084: Adjust message when an f-string expression causes a SyntaxError (GH-21084)
Prefix the error message with `fstring: `, when parsing an f-string expression throws a `SyntaxError`.
2020-06-26 12:24:05 +01:00
Lysandros Nikolaou
4b85e60601 bpo-41119: Output correct error message for list/tuple followed by colon (GH-21160) 2020-06-26 00:22:36 +01:00
Lysandros Nikolaou
564cd18767 bpo-40939: Rename PyPegen* functions to PyParser* (GH-21016)
Rename PyPegen* functions to PyParser*, so that we can remove the
old set of PyParser* functions that were using the old parser.
2020-06-22 00:47:46 +01:00
Lysandros Nikolaou
6c4e0bd974 bpo-41060: Avoid SEGFAULT when calling GET_INVALID_TARGET in the grammar (GH-21020)
`GET_INVALID_TARGET` might unexpectedly return `NULL`, which if not
caught will cause a SEGFAULT. Therefore, this commit introduces a new
inline function `RAISE_SYNTAX_ERROR_INVALID_TARGET` that always
checks for `GET_INVALID_TARGET` returning NULL and can be used in
the grammar, replacing the long C ternary operation used till now.
2020-06-21 03:18:01 +01:00
Lysandros Nikolaou
314858e276 bpo-40939: Remove the old parser (Part 2) (GH-21005)
Remove some remaining files and Makefile targets for the old parser
2020-06-20 19:07:25 +01:00
Lysandros Nikolaou
861efc6e8f bpo-40958: Avoid 'possible loss of data' warning on Windows (GH-20970) 2020-06-20 05:57:27 -07:00
Lysandros Nikolaou
01ece63d42 bpo-40334: Produce better error messages on invalid targets (GH-20106)
The following error messages get produced:
- `cannot delete ...` for invalid `del` targets
- `... is an illegal 'for' target` for invalid targets in for
  statements
- `... is an illegal 'with' target` for invalid targets in
  with statements

Additionally, a few `cut`s were added in various places before the
invocation of the `invalid_*` rule, in order to speed things
up.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-06-19 00:10:43 +01:00
Pablo Galindo
51c5896b62 bpo-40958: Avoid buffer overflow in the parser when indexing the current line (GH-20875) 2020-06-16 16:49:43 +01:00
Pablo Galindo
e0bec69854 Remove old comment in string_parser.c (GH-20906) 2020-06-16 02:13:33 +01:00
Victor Stinner
e822e37946 bpo-36020: Remove snprintf macro in pyerrors.h (GH-20889)
On Windows, #include "pyerrors.h" no longer defines "snprintf" and
"vsnprintf" macros.

PyOS_snprintf() and PyOS_vsnprintf() should be used to get portable
behavior.

Replace snprintf() calls with PyOS_snprintf() and replace vsnprintf()
calls with PyOS_vsnprintf().
2020-06-15 21:59:47 +02:00
Pablo Galindo
fb61c42361 Improve readability and style in parser files (GH-20884) 2020-06-15 14:23:43 +01:00
Pablo Galindo
1ed83adb0e bpo-40939: Remove the old parser (GH-20768)
This commit removes the old parser, the deprecated parser module, the old parser compatibility flags and environment variables and all associated support code and documentation.
2020-06-11 17:30:46 +01:00
Lysandros Nikolaou
bcd7deed91 bpo-40939: Remove PEG parser easter egg (__new_parser__) (#20802)
It no longer serves a purpose (there's only one parser) and having "new" in any name will eventually look odd. Also, it impinges on a potential sub-namespace, `__new_...__`.
2020-06-11 09:09:21 -07:00