bpo-36443, bpo-36202: Since Python 3.7.0, calling Py_DecodeLocale()
before Py_Initialize() produces mojibake if the LC_CTYPE locale is
coerced and/or if the UTF-8 Mode is enabled by the user
configuration. This change fix the issue by disabling LC_CTYPE
coercion and UTF-8 Mode by default. They must now be enabled
explicitly (opt-in) using the new _Py_PreInitialize() API with
_PyPreConfig.
When embedding Python, set coerce_c_locale and utf8_mode attributes
of _PyPreConfig to -1 to enable automatically these parameters
depending on the LC_CTYPE locale, environment variables and command
line arguments
Alternative: Setting Py_UTF8Mode to 1 always explicitly enables the
UTF-8 Mode.
Changes:
* _PyPreConfig_INIT now sets coerce_c_locale and utf8_mode to 0 by
default.
* _Py_InitializeFromArgs() and _Py_InitializeFromWideArgs() can now
be called with config=NULL.
* _PyCoreConfig_Write() now updates _PyRuntime.preconfig
* Remove _PyPreCmdline_Copy()
* _PyPreCmdline_Read() now accepts _PyPreConfig and _PyCoreConfig
optional configurations.
* Rename _PyPreConfig_ReadFromArgv() to _PyPreConfig_Read(). Simplify
the code.
* Calling _PyCoreConfig_Read() no longer adds the warning options
twice: don't add a warning option if it's already in the list.
* Rename _PyCoreConfig_ReadFromArgv() to _PyCoreConfig_Read().
* Rename config_from_cmdline() to _PyCoreConfig_ReadFromArgv().
* Add more assertions on _PyCoreConfig in _PyCoreConfig_Read().
* Move some functions.
* Make some config functions private.
* Make _PyPreConfig_GetEnv(), _PyCoreConfig_GetEnv() and
_PyCoreConfig_GetEnvDup() private
* _Py_get_env_flag() first parameter becomes "int use_environment"
* Add _Py_GetConfigsAsDict() function to get all configurations as a
dict.
* dump_config() of _testembed.c now dumps preconfig as a separated
key: call _Py_GetConfigsAsDict().
* Make _PyMainInterpreterConfig_AsDict() private.
* Initialize _PyPreConfig.dev_mode to -1.
* _PyPreConfig_Read(): coreconfig has the priority over preconfig.
* _PyCoreConfig_Read() now calls _PyPreCmdline_Read() internally.
* config_from_cmdline() now pass _PyPreCmdline to config_read().
* Add _PyPreCmdline_Copy().
Prepare code to move some _PyPreConfig parameters into _PyPreCmdline.
Changes:
* _PyCoreConfig_ReadFromArgv(): remove preconfig parameter,
use _PyRuntime.preconfig.
* Add _PyPreCmdline_GetPreConfig() (called by _PyPreConfig_Read()).
* Rename _PyPreCmdline_Init() to _PyPreCmdline_SetArgv()
* Factorize _Py_PreInitializeFromPreConfig() code: add
pyinit_preinit().
* _PyPreConfig_Read() now sets coerce_c_locale to 2 if it must be
coerced.
* Remove _PyCoreConfig_ReadPreConfig().
* _PyCoreConfig_Write() now copies updated preconfig into _PyRuntime.
* Add _PyRuntime.pre_initialized: set to 1 when Python
is pre-initialized
* Add _Py_PreInitialize() and _Py_PreInitializeFromPreConfig().
* _PyCoreConfig_Read() now calls _Py_PreInitialize().
* Move _PyPreConfig_GetGlobalConfig() and
_PyCoreConfig_GetGlobalConfig() calls from main.c to preconfig.c
and coreconfig.c.
Calling _PyRuntime_Initialize() after _PyRuntime_Finalize() now re-initializes
_PyRuntime structure. Previously, _PyRuntime_Initialize() did
nothing in that case.
* _PyPathConfig_ComputeSysPath0() now returns 0 if argv is empty.
* Cleanup also _PyPathConfig_ComputeSysPath0() code: move variables
definitions closer to where they are used.
At Python initialization, the current directory is no longer
prepended to sys.path if it has been removed.
Rename _PyPathConfig_ComputeArgv0() to
_PyPathConfig_ComputeSysPath0() to avoid confusion between argv[0]
and sys.path[0].
