* bpo-34485: stdout uses surrogateescape on POSIX locale (GH-8986)
Standard streams like sys.stdout now use the "surrogateescape" error
handler, instead of "strict", on the POSIX locale (when the C locale is not
coerced and the UTF-8 Mode is disabled).
Add tests on sys.stdout.errors with LC_ALL=POSIX.
Fix the error handler of standard streams like sys.stdout:
PYTHONIOENCODING=":" is now ignored instead of setting the error handler to
"strict".
(cherry picked from commit 315877dc36)
* _Py_InitializeCore() now sets the LC_CTYPE locale to the user
preferred locale before checking if the C locale should be coerced
or not in _PyCoreConfig_Read().
* Fix pymain_read_conf(): remember if the C locale has been coerced
when the configuration should be read again if the encoding has
changed.
(cherry picked from commit 2c8ddcf4f1)
* The UTF-8 Mode is now also enabled by the "POSIX" locale, not only
by the "C" locale.
* On FreeBSD, Py_DecodeLocale() and Py_EncodeLocale() now also forces
the ASCII encoding if the LC_CTYPE locale is "POSIX", not only if
the LC_CTYPE locale is "C".
* test_utf8_mode.test_cmd_line() checks also that the command line
arguments are decoded from UTF-8 when the the UTF-8 Mode is enabled
with POSIX locale or C locale.
(cherry picked from commit 5cb258950c)
* -X dev: it is now possible to override the memory allocator using
PYTHONMALLOC even if the developer mode is enabled.
* Add _Py_InitializeFromConfig()
* Add _Py_Initialize_ReadEnvVars() to set global configuration
variables from environment variables
* Fix the code to initialize Python: Py_Initialize() now also reads
environment variables
* _Py_InitializeCore() can now be called twice: the second call
only replaces the configuration.
* Write unit tests on Py_Initialize() and the different ways to
configure Python
* The isolated mode now always sets Py_IgnoreEnvironmentFlag and
Py_NoUserSiteDirectory to 1.
* pymain_read_conf() now saves/restores the configuration
if the encoding changed
Py_Main() can again be called after Py_Initialize(), as in Python
3.6. The new configuration is ignored, except of
_PyMainInterpreterConfig.argv which is used to update sys.argv.
(cherry picked from commit fb47bca9ee)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
`_PyUnicode_TransformDecimalAndSpaceToASCII()` missed trailing NUL char.
It caused buffer overflow in `_Py_string_to_number_with_underscores()`.
This bug is introduced in 9b6c60cb.
(cherry picked from commit 16dfca4d82)
Co-authored-by: INADA Naoki <methane@users.noreply.github.com>
This will prevent emitting a resource warning when the execution was
interrupted by Ctrl-C between calling open() and entering a 'with' block
in "with open()".
(cherry picked from commit 3f4d90d4d7)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
PyErr_Print() will not return when the exception is a SystemExit, so
decref the __main__ module object in that case.
(cherry picked from commit d8cba5d16f)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
Directly executed pyc files were being kept open longer than necessary.
(cherry picked from commit ea737751b1)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
Calling Py_Initialize() twice does nothing, instead of failing with a
fatal error: restore the Python 3.6 behaviour.
(cherry picked from commit 209abf7469)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
