Commit Graph

852 Commits

Author SHA1 Message Date
Pablo Galindo
3782497cc2 [3.9] bpo-40939: Fix test_keyword for the old parser (GH-20814) 2020-06-11 19:29:13 +01:00
Miss Islington (bot)
e3ce3bba92 bpo-40847: Consider a line with only a LINECONT a blank line (GH-20769)
A line with only a line continuation character should be considered
a blank line at tokenizer level so that only a single NEWLINE token
gets emitted. The old parser was working around the issue, but the
new parser threw a `SyntaxError` for valid input. For example,
an empty line following a line continuation character was interpreted
as a `SyntaxError`.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
(cherry picked from commit 896f4cf63f)

Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
2020-06-10 17:14:16 -07:00
Miss Islington (bot)
d55ed7b107 Raise specialised syntax error for invalid lambda parameters (GH-20776)
(cherry picked from commit c6483c9896)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-06-10 06:24:41 -07:00
Miss Islington (bot)
8df4f3942f bpo-40903: Handle multiple '=' in invalid assignment rules in the PEG parser (GH-20697)
Automerge-Triggered-By: @pablogsal
(cherry picked from commit 9f495908c5)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-06-08 02:22:06 -07:00
Miss Islington (bot)
6440911736 bpo-40904: Fix segfault in the new parser with f-string containing yield statements with no value (GH-20701)
(cherry picked from commit 972ab03276)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-06-07 18:08:53 -07:00
Miss Islington (bot)
15fec5627a bpo-40880: Fix invalid read in newline_in_string in pegen.c (GH-20666)
* bpo-40880: Fix invalid read in newline_in_string in pegen.c

* Update Parser/pegen/pegen.c

Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>

* Add NEWS entry

Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
(cherry picked from commit 2e6593db00)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-06-05 17:13:14 -07:00
Miss Islington (bot)
79e6c15aed bpo-40883: Fix memory leak in fstring_compile_expr in parse_string.c (GH-20667)
(cherry picked from commit a54096e305)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-06-05 17:10:57 -07:00
Victor Stinner
5d2396c8cf [3.9] bpo-40826: Fix GIL usage in PyOS_Readline() (GH-20613)
* bpo-40826: Fix GIL usage in PyOS_Readline() (GH-20579)

Fix GIL usage in PyOS_Readline(): lock the GIL to set an exception.

Pass tstate to my_fgets() and _PyOS_WindowsConsoleReadline(). Cleanup
these functions.

(cherry picked from commit c353764fd5)

* bpo-40826: Add _PyOS_InterruptOccurred(tstate) function (GH-20599)

my_fgets() now calls _PyOS_InterruptOccurred(tstate) to check for
pending signals, rather calling PyOS_InterruptOccurred().

my_fgets() is called with the GIL released, whereas
PyOS_InterruptOccurred() must be called with the GIL held.

test_repl: use text=True and avoid SuppressCrashReport in
test_multiline_string_parsing().

Fix my_fgets() on Windows: fgets(fp) does crash if fileno(fp) is closed.

(cherry picked from commit fa7ab6aa0f)
2020-06-03 17:49:25 +02:00
Miss Islington (bot)
bcbe5c59dd bpo-40217: Ensure Py_VISIT(Py_TYPE(self)) is always called for PyType_FromSpec types (reverts GH-19414) (GH-20264)
Heap types now always visit the type in tp_traverse. See added docs for details.

This reverts commit 0169d3003b.

Automerge-Triggered-By: @encukou
(cherry picked from commit 1cf15af9a6)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-05-28 08:12:23 -07:00
Pablo Galindo
9b83829e7d [3.9] bpo-40614: Respect feature version for f-string debug expressions (GH-20196) (GH-20464)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
(cherry picked from commit c116c94)

Co-authored-by: Shantanu <hauntsaninja@users.noreply.github.com>
2020-05-27 22:01:11 +01:00
Lysandros Nikolaou
c011d1b5be [3.9] Backport GH-20440: Set p->error_indicator in more places (GH-20457) 2020-05-27 21:20:43 +01:00
Lysandros Nikolaou
1bfe659ee5 [3.9] Backport GH-20370 and GH-20436: Soft keywords (GH-20458) 2020-05-27 21:20:07 +01:00
Miss Islington (bot)
fda47659bb Fix peg_generator compiler warnings under MSVC (GH-20405)
(cherry picked from commit a2bbedc8b1)

Co-authored-by: Ammar Askar <ammar@ammaraskar.com>
2020-05-25 21:52:21 -07:00
Lysandros Nikolaou
791a46ed58 [3.9] bpo-38964: Print correct filename on a SyntaxError in an fstring (GH-20399) (GH-20404)
When a `SyntaxError` in the expression part of a fstring is found,
the filename attribute of the `SyntaxError` is always `<fstring>`.
With this commit, it gets changed to always have the name of the file
the fstring resides in.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>.
(cherry picked from commit f7b1e46156)
2020-05-26 02:24:31 +01:00
Miss Islington (bot)
31084be618 bpo-40750: Do not expand the new parser debug flags if Py_BUILD_CORE is not defined (GH-20393)
(cherry picked from commit deb4355a37)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-05-25 12:37:56 -07:00
Miss Islington (bot)
82da2c3eb4 bpo-40750: Support -d flag in the new parser (GH-20340)
(cherry picked from commit 800a35c623)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-05-25 10:58:03 -07:00
Miss Islington (bot)
1a4e9e6f35 bpo-36290: Fix keytword collision handling in AST node constructors (GH-12382)
(cherry picked from commit c73914a562)

Co-authored-by: Rémi Lapeyre <remi.lapeyre@lenstra.fr>
2020-05-24 14:32:32 -07:00
Miss Islington (bot)
11fb605cb8 Use Py_ssize_t for the column number in the PEG support code (GH-20341)
(cherry picked from commit b23d7adfdf)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-05-23 22:20:44 -07:00
Miss Islington (bot)
55c8923524 bpo-40334: Produce better error messages for non-parenthesized genexps (GH-20153)
The error message, generated for a non-parenthesized generator expression
in function calls, was still the generic `invalid syntax`, when the generator expression wasn't appearing as the first argument in the call. With this patch, even on input like `f(a, b, c for c in d, e)`, the correct error message gets produced.
(cherry picked from commit ae14583302)

Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
2020-05-21 18:14:55 -07:00
Miss Islington (bot)
d00aaf306a bpo-40715: Reject dict unpacking on dict comprehensions (GH-20292)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
(cherry picked from commit b8a65ec1d3)

Co-authored-by: Batuhan Taskaya <isidentical@gmail.com>
2020-05-21 15:58:16 -07:00
Pablo Galindo
275d7e1080 [3.9] bpo-40176: Improve error messages for trailing comma on from import (GH-20294) (GH-20302)
(cherry picked from commit 72e0aa2)

Co-authored-by: Batuhan Taskaya <batuhanosmantaskaya@gmail.com>
2020-05-21 22:04:54 +01:00
Pablo Galindo
ced4e5c227 Regenerate the parser (#20195) 2020-05-18 23:47:51 +02:00
Lysandros Nikolaou
75b863aa97 bpo-40334: Reproduce error message for type comments on bare '*' in the new parser (GH-20151) 2020-05-18 20:14:47 +01:00
Batuhan Taskaya
63b8e0cba3 bpo-40528: Improve AST generation script to do builds simultaneously (GH-19968)
- Switch from getopt to argparse.
- Removed the limitation of not being able to produce both C and H simultaneously.

This will make it run faster since it parses the asdl definition once and uses the generated tree to generate both the header and the C source.
2020-05-18 18:42:10 +01:00
Lysandros Nikolaou
7b7a21bc4f bpo-40661: Fix segfault when parsing invalid input (GH-20165)
Fix segfaults when parsing very complex invalid input, like `import äˆ ð£„¯ð¢·žð±‹á”€ð””ð‘©±å®ä±¬ð©¾\nð—¶½`.

Co-authored-by: Guido van Rossum <guido@python.org>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2020-05-18 18:32:03 +01:00