Commit Graph

3064 Commits

Author SHA1 Message Date
Miss Islington (bot)
961edf7979 bpo-40939: Generate keyword.py using the new parser (GH-20800)
(cherry picked from commit 9727694f08)

Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
2020-06-11 06:02:37 -07:00
Miss Islington (bot)
18f1226884 Refactor scripts in Tools/peg_generator/scripts (GH-20401)
(cherry picked from commit ba6fd87e41)

Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
2020-06-05 21:41:12 -07:00
Miss Islington (bot)
24a88b008b bpo-30008: Fix OpenSSL no-deprecated compilation (GH-20397)
Fix :mod:`ssl`` code to be compatible with OpenSSL 1.1.x builds that use
``no-deprecated`` and ``--api=1.1.0``.

Note: Tests assume full OpenSSL API and fail with limited API.

Signed-off-by: Christian Heimes <christian@python.org>
Co-authored-by: Mark Wright <gienah@gentoo.org>
(cherry picked from commit a871f692b4)

Co-authored-by: Christian Heimes <christian@python.org>
2020-06-01 00:18:39 -07:00
Lysandros Nikolaou
1bfe659ee5 [3.9] Backport GH-20370 and GH-20436: Soft keywords (GH-20458) 2020-05-27 21:20:07 +01:00
Miss Islington (bot)
fda47659bb Fix peg_generator compiler warnings under MSVC (GH-20405)
(cherry picked from commit a2bbedc8b1)

Co-authored-by: Ammar Askar <ammar@ammaraskar.com>
2020-05-25 21:52:21 -07:00
Miss Islington (bot)
3c6c86ab77 bpo-40688: Use the correct parser in the peg_generator scripts (GH-20235)
The scripts in `Tools/peg_generator/scripts` mostly assume that
`ast.parse` and `compile` use the old parser, since this was the
state of things, while we were developing them. They need to be
updated to always use the correct parser. `_peg_parser` is being
extended to support both parsing and compiling with both parsers.
(cherry picked from commit 9645930b5b)

Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
2020-05-25 13:11:36 -07:00
Miss Islington (bot)
31084be618 bpo-40750: Do not expand the new parser debug flags if Py_BUILD_CORE is not defined (GH-20393)
(cherry picked from commit deb4355a37)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-05-25 12:37:56 -07:00
Miss Islington (bot)
82da2c3eb4 bpo-40750: Support -d flag in the new parser (GH-20340)
(cherry picked from commit 800a35c623)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-05-25 10:58:03 -07:00
Miss Islington (bot)
82c274e3ba bpo-40334: Support suppressing of multiple optional variables in Pegen (GH-20367)
(cherry picked from commit cba5031510)

Co-authored-by: Batuhan Taskaya <batuhanosmantaskaya@gmail.com>
2020-05-24 15:38:47 -07:00
Miss Islington (bot)
427c696400 Fix debug output in PEG parser generator (GH-20308)
(cherry picked from commit b831129123)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-05-21 19:05:20 -07:00
Pablo Galindo
7178b1bb68 [3.9] Fix typing problems reported by mypy in pegen (GH-20297) (GH-20300)
(cherry picked from commit d10fef35c6)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-05-21 22:09:43 +01:00
Miss Islington (bot)
c06983bc30 bpo-39631: Fix file association MIME type in the Windows installer (GH-20205)
Use text/x-python instead of text/plain to avoid issues with tools assuming that "ShellExecute(script)" is a non-executable operation.
(cherry picked from commit 8c862e5124)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>
2020-05-19 05:39:16 -07:00
Miss Islington (bot)
3d062829de bpo-40669: Use requirements.pip when installing PEG dependencies (GH-20194)
(cherry picked from commit 3764069f3b)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-05-18 19:31:28 -07:00
Lysandros Nikolaou
dc31800f86 bpo-40669: Install PEG benchmarking dependencies in a venv (GH-20183)
Create a `make venv` target, that creates a virtual environment
and installs the dependency in that venv. `make time` and all
the related targets are changed to use the virtual environment
python.

Automerge-Triggered-By: @pablogsal
2020-05-18 11:27:40 -07:00
Lysandros Nikolaou
7b7a21bc4f bpo-40661: Fix segfault when parsing invalid input (GH-20165)
Fix segfaults when parsing very complex invalid input, like `import äˆ ð£„¯ð¢·žð±‹á”€ð””ð‘©±å®ä±¬ð©¾\nð—¶½`.

Co-authored-by: Guido van Rossum <guido@python.org>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2020-05-18 18:32:03 +01:00
Lysandros Nikolaou
2c8cd06afe bpo-40334: Improvements to error-handling code in the PEG parser (GH-20003)
The following improvements are implemented in this commit:
- `p->error_indicator` is set, in case malloc or realloc fail.
- Avoid memory leaks in the case that realloc fails.
- Call `PyErr_NoMemory()` instead of `PyErr_Format()`, because it requires no memory.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-05-17 04:19:23 +01:00
Paul Ganssle
62972d9d73 bpo-40503: PEP 615: Tests and implementation for zoneinfo (GH-19909)
This is the initial implementation of PEP 615, the zoneinfo module,
ported from the standalone reference implementation (see
https://www.python.org/dev/peps/pep-0615/#reference-implementation for a
link, which has a more detailed commit history).

This includes (hopefully) all functional elements described in the PEP,
but documentation is found in a separate PR. This includes:

1. A pure python implementation of the ZoneInfo class
2. A C accelerated implementation of the ZoneInfo class
3. Tests with 100% branch coverage for the Python code (though C code
   coverage is less than 100%).
4. A compile-time configuration option on Linux (though not on Windows)

Differences from the reference implementation:

- The module is arranged slightly differently: the accelerated module is
  `_zoneinfo` rather than `zoneinfo._czoneinfo`, which also necessitates
  some changes in the test support function. (Suggested by Victor
  Stinner and Steve Dower.)
- The tests are arranged slightly differently and do not include the
  property tests. The tests live at test/test_zoneinfo/test_zoneinfo.py
  rather than test/test_zoneinfo.py or test/test_zoneinfo/__init__.py
  because we may do some refactoring in the future that would likely
  require this separation anyway; we may:
        - include the property tests
        - automatically run all the tests against both pure Python and C,
          rather than manually constructing C and Python test classes (similar
          to the way this works with test_datetime.py, which generates C
          and Python test cases from datetimetester.py).
- This includes a compile-time configuration option on Linux (though not
  on Windows); added with much help from Thomas Wouters.
- Integration into the CPython build system is obviously different from
  building a standalone zoneinfo module wheel.
- This includes configuration to install the tzdata package as part of
  CI, though only on the coverage jobs. Introducing a PyPI dependency as
  part of the CI build was controversial, and this is seen as less of a
  major change, since the coverage jobs already depend on pip and PyPI.

Additional changes that were introduced as part of this PR, most / all of
which were backported to the reference implementation:

- Fixed reference and memory leaks

    With much debugging help from Pablo Galindo

- Added smoke tests ensuring that the C and Python modules are built

    The import machinery can be somewhat fragile, and the "seamlessly falls
    back to pure Python" nature of this module makes it so that a problem
    building the C extension or a failure to import the pure Python version
    might easily go unnoticed.

- Adjustments to zoneinfo.__dir__

    Suggested by Petr Viktorin.

- Slight refactorings as suggested by Steve Dower.

- Removed unnecessary if check on std_abbr

    Discovered this because of a missing line in branch coverage.
2020-05-16 10:20:06 +02:00
Christian Heimes
6e8cda91d9 bpo-40457: Support OpenSSL without TLS 1.0/1.1 (GH-19862)
OpenSSL can be build without support for TLS 1.0 and 1.1. The ssl module
now correctly adheres to OPENSSL_NO_TLS1 and OPENSSL_NO_TLS1_1 flags.

Also update multissltest to test with latest OpenSSL and LibreSSL
releases.

Signed-off-by: Christian Heimes <christian@python.org>

Automerge-Triggered-By: @tiran
2020-05-15 18:33:05 -07:00
Christian Heimes
938717fd04 bpo-40163: Fix multissltest download of old OpenSSL (GH-19329) 2020-05-15 22:32:25 +02:00
Christian Heimes
62d618c06b bpo-40479: Test with latest OpenSSL versions (GH-20108)
* 1.0.2u (EOL)
* 1.1.0l (EOL)
* 1.1.1g
* 3.0.0-alpha2 (disabled for now)

Build the FIPS provider and create a FIPS configuration file for OpenSSL
3.0.0.

Signed-off-by: Christian Heimes <christian@python.org>

Automerge-Triggered-By: @tiran
2020-05-15 09:48:25 -07:00
Victor Stinner
97e1568325 bpo-38787: Fix Argument Clinic defining_class_converter (GH-20074)
Don't hardcode defining_class parameter name to "cls":

* Define CConverter.set_template_dict(): do nothing by default
* CLanguage.render_function() now calls set_template_dict() on all
  converters.
2020-05-14 00:31:31 +02:00
Steve Dower
d6b727e2c9 bpo-40501: Replace ctypes code in uuid with native module (GH-19948) 2020-05-12 23:32:32 +01:00
Pablo Galindo
ac7a92cc0a bpo-40334: Avoid collisions between parser variables and grammar variables (GH-19987)
This is for the C generator:
- Disallow rule and variable names starting with `_`
- Rename most local variable names generated by the parser to start with `_`

Exceptions:
- Renaming `p` to `_p` will be a separate PR
- There are still some names that might clash, e.g.
  - anything starting with `Py`
  - C reserved words (`if` etc.)
  - Macros like `EXTRA` and `CHECK`
2020-05-09 21:34:50 -07:00
Pablo Galindo
db9163ceef bpo-40555: Check for p->error_indicator in loop rules after the main loop is done (GH-19986) 2020-05-08 03:38:44 +01:00
Petr Viktorin
e1becf46b4 bpo-38787: C API for module state access from extension methods (PEP 573) (GH-19936)
Module C state is now accessible from C-defined heap type methods (PEP 573).
Patch by Marcel Plch and Petr Viktorin.

Co-authored-by: Marcel Plch <mplch@redhat.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
2020-05-07 15:39:59 +02:00