It no longer spends much time doing complex calculations and no
longer consumes much memory for creating large constants that will
be dropped later.
This fixes also bpo-21074.
Each interpreter now has its core_config and main_config copy:
* Add _PyCoreConfig_Copy() and _PyMainInterpreterConfig_Copy()
* Move _PyCoreConfig_Read(), _PyCoreConfig_Clear() and
_PyMainInterpreterConfig_Clear() from Python/pylifecycle.c to
Modules/main.c
* Fix _Py_InitializeEx_Private(): call _PyCoreConfig_ReadEnv() before
_Py_InitializeCore()
bpo-32329, bpo-32030:
* The -R option now turns on hash randomization when the
PYTHONHASHSEED environment variable is set to 0 Previously, the
option was ignored.
* sys.flags.hash_randomization is now properly set to 0 when hash
randomization is turned off by PYTHONHASHSEED=0.
* _PyCoreConfig_ReadEnv() now reads the PYTHONHASHSEED environment
variable. _Py_HashRandomization_Init() now only apply the
configuration, it doesn't read PYTHONHASHSEED anymore.
Conversions like `not a is b -> a is not b` are implemented
in AST optimizer in previous commit (7ea143a).
So this commit removes them from peephole optimizer.
_PyMainInterpreterConfig now contains Python objects, whereas
_PyCoreConfig contains wchar_t* strings.
Core config:
* Rename _PyMainInterpreterConfig_ReadEnv() to _PyCoreConfig_ReadEnv()
* Move 3 strings from _PyMainInterpreterConfig to _PyCoreConfig:
module_search_path_env, home, program_name.
* Add _PyCoreConfig_Clear()
* _PyPathConfig_Calculate() now takes core config rather than main
config
* _PyMainInterpreterConfig_Read() now requires also a core config
Main config:
* Add _PyMainInterpreterConfig.module_search_path: sys.path list
* Add _PyMainInterpreterConfig.argv: sys.argv list
* _PyMainInterpreterConfig_Read() now computes module_search_path
Changes:
* Split _PySys_SetArgvWithError() into subfunctions for Py_Main():
* Create the Python list object
* Set sys.argv to the list
* Compute argv0
* Prepend argv0 to sys.path
* Add _PyPathConfig_ComputeArgv0()
* Remove _PySys_SetArgvWithError()
* Py_Main() now splits the code to compute sys.argv/path0 and the
code to update the sys module: add pymain_compute_argv()
subfunction.
bpo-29240, bpo-32030:
* Rename pymain_set_argv() to pymain_set_sys_argv()
* pymain_set_sys_argv() now creates of copy of argv and modify the
copy, rather than modifying pymain->argv
* Call pymain_set_sys_argv() earlier: before pymain_run_python(), but
after pymain_get_importer().
* Add _PySys_SetArgvWithError() to handle errors
* Add -X utf8 command line option, PYTHONUTF8 environment variable
and a new sys.flags.utf8_mode flag.
* If the LC_CTYPE locale is "C" at startup: enable automatically the
UTF-8 mode.
* Add _winapi.GetACP(). encodings._alias_mbcs() now calls
_winapi.GetACP() to get the ANSI code page
* locale.getpreferredencoding() now returns 'UTF-8' in the UTF-8
mode. As a side effect, open() now uses the UTF-8 encoding by
default in this mode.
* Py_DecodeLocale() and Py_EncodeLocale() now use the UTF-8 encoding
in the UTF-8 Mode.
* Update subprocess._args_from_interpreter_flags() to handle -X utf8
* Skip some tests relying on the current locale if the UTF-8 mode is
enabled.
* Add test_utf8mode.py.
* _Py_DecodeUTF8_surrogateescape() gets a new optional parameter to
return also the length (number of wide characters).
* pymain_get_global_config() and pymain_set_global_config() now
always copy flag values, rather than only copying if the new value
is greater than the old value.
Fix the warning:
Python/import.c: warning: comparison between signed and unsigned integer expressions
if ((i + n + 1) <= PY_SSIZE_T_MAX / sizeof(struct _inittab)) {
Rather than supporting dev mode directly in the warnings module, this
instead adjusts the initialisation code to add an extra 'default'
entry to sys.warnoptions when dev mode is enabled.
This ensures that dev mode behaves *exactly* as if `-Wdefault` had
been passed on the command line, including in the way it interacts
with `sys.warnoptions`, and with other command line flags like `-bb`.
Fix also bpo-20361: have -b & -bb options take precedence over any
other warnings options.
Patch written by Nick Coghlan, with minor modifications of Victor Stinner.
Reference siphash takes the keys as a bytes, so it makes sense to byte swap
when reifying the keys as 64-bit integers. However, Python's siphash takes host
integers in to start with.
Python now supports checking bytecode cache up-to-dateness with a hash of the
source contents rather than volatile source metadata. See the PEP for details.
While a fairly straightforward idea, quite a lot of code had to be modified due
to the pervasiveness of pyc implementation details in the codebase. Changes in
this commit include:
- The core changes to importlib to understand how to read, validate, and
regenerate hash-based pycs.
- Support for generating hash-based pycs in py_compile and compileall.
- Modifications to our siphash implementation to support passing a custom
key. We then expose it to importlib through _imp.
- Updates to all places in the interpreter, standard library, and tests that
manually generate or parse pyc files to grok the new format.
- Support in the interpreter command line code for long options like
--check-hash-based-pycs.
- Tests and documentation for all of the above.
