Commit Graph

106 Commits

Author SHA1 Message Date
Victor Stinner
cb064fc232 bpo-31900: Fix localeconv() encoding for LC_NUMERIC (#4174)
* Add _Py_GetLocaleconvNumeric() function: decode decimal_point and
  thousands_sep fields of localeconv() from the LC_NUMERIC encoding,
  rather than decoding from the LC_CTYPE encoding.
* Modify locale.localeconv() and "n" formatter of str.format() (for
  int, float and complex to use _Py_GetLocaleconvNumeric()
  internally.
2018-01-15 15:58:02 +01:00
Victor Stinner
7ed7aead95 bpo-29240: Fix locale encodings in UTF-8 Mode (#5170)
Modify locale.localeconv(), time.tzname, os.strerror() and other
functions to ignore the UTF-8 Mode: always use the current locale
encoding.

Changes:

* Add _Py_DecodeLocaleEx() and _Py_EncodeLocaleEx(). On decoding or
  encoding error, they return the position of the error and an error
  message which are used to raise Unicode errors in
  PyUnicode_DecodeLocale() and PyUnicode_EncodeLocale().
* Replace _Py_DecodeCurrentLocale() with _Py_DecodeLocaleEx().
* PyUnicode_DecodeLocale() now uses _Py_DecodeLocaleEx() for all
  cases, especially for the strict error handler.
* Add _Py_DecodeUTF8Ex(): return more information on decoding error
  and supports the strict error handler.
* Rename _Py_EncodeUTF8_surrogateescape() to _Py_EncodeUTF8Ex().
* Replace _Py_EncodeCurrentLocale() with _Py_EncodeLocaleEx().
* Ignore the UTF-8 mode to encode/decode localeconv(), strerror()
  and time zone name.
* Remove PyUnicode_DecodeLocale(), PyUnicode_DecodeLocaleAndSize()
  and PyUnicode_EncodeLocale() now ignore the UTF-8 mode: always use
  the "current" locale.
* Remove _PyUnicode_DecodeCurrentLocale(),
  _PyUnicode_DecodeCurrentLocaleAndSize() and
  _PyUnicode_EncodeCurrentLocale().
2018-01-15 10:45:49 +01:00
Serhiy Storchaka
f7eae0adfc [security] bpo-13617: Reject embedded null characters in wchar* strings. (#2302)
Based on patch by Victor Stinner.

Add private C API function _PyUnicode_AsUnicode() which is similar to
PyUnicode_AsUnicode(), but checks for null characters.
2017-06-28 08:30:06 +03:00
Benjamin Peterson
ad4a0cc519 allow the first call to wcsxfrm to return ERANGE (#536)
If the output buffer provided to wcsxfrm is too small, errno is set to ERANGE. We should not error out in that case.
2017-03-07 22:24:44 -08:00
Serhiy Storchaka
be487a65f1 bpo-15954: Check return code of wcsxfrm(). (#508) 2017-03-06 21:21:41 +02:00
Martin Panter
d508d00919 Issue #28139: Merge indentation fixes from 3.5 into 3.6 2016-09-17 07:59:14 +00:00
Martin Panter
6d57fe1c23 Issue #28139: Fix messed up indentation
Also update the classmethod and staticmethod doc strings and comments to
match the RST documentation.
2016-09-17 03:26:16 +00:00
Christian Heimes
ff4fddde57 Add NULL checks to the initializer of the locale module
The _locale module was using old-style APIs to set numeric module
constants from macros. The new way requires less code and properly
checks for NULL.

CID 1295027
2016-09-09 00:24:12 +02:00
Serhiy Storchaka
ef1585eb9a Issue #25923: Added more const qualifiers to signatures of static and private functions. 2015-12-25 20:01:53 +02:00
Serhiy Storchaka
1a1ff29659 Issue #23446: Use PyMem_New instead of PyMem_Malloc to avoid possible integer
overflows.  Added few missed PyErr_NoMemory().
2015-02-16 13:28:22 +02:00
Victor Stinner
f38a5c28e0 Cleanup locale.localeconv(): move Py_DECREF() closer to the error 2013-10-29 19:28:20 +01:00
Victor Stinner
d594f24fbc Issue #18408: Fix locale.localeconv(), handle PyDict_SetItemString() failure 2013-07-17 00:55:57 +02:00
Kristjan Valur Jonsson
85634d7a2e Issue #14909: A number of places were using PyMem_Realloc() apis and
PyObject_GC_Resize() with incorrect error handling.  In case of errors,
the original object would be leaked.  This checkin fixes those cases.
2012-05-31 09:37:31 +00:00
Victor Stinner
a9c895d497 PyUnicode_DecodeLocale() second argument is now a char*, no more an int 2012-02-14 02:33:38 +01:00
Victor Stinner
af02e1c85a Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
* PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale() decode a string
   from the current locale encoding
 * _Py_char2wchar() writes an "error code" in the size argument to indicate
   if the function failed because of memory allocation failure or because of a
   decoding error. The function doesn't write the error message directly to
   stderr.
 * Fix time.strftime() (if wcsftime() is missing): decode strftime() result
   from the current locale encoding, not from the filesystem encoding.
2011-12-16 23:56:01 +01:00
Victor Stinner
ce0e7daf41 Remove temporary hacks for the issue #13441 2011-11-21 16:01:27 +01:00
Victor Stinner
a697b37300 Another temporary hack to debug the issue #13441
Dump the wchar_t that we are going to decode and dump the locale
2011-11-21 15:41:17 +01:00
Victor Stinner
da29cc36aa Issue #13441: _PyUnicode_CheckConsistency() dumps the string if the maximum
character is bigger than U+10FFFF and locale.localeconv() dumps the string
before decoding it.

Temporary hack to debug the issue #13441.
2011-11-21 14:31:41 +01:00
Victor Stinner
2c5d3cbfb8 Fix a compiler warning in _locale 2011-10-11 22:35:52 +02:00
Victor Stinner
6364927cec PyLocale_strxfrm() uses the new Unicode API 2011-09-29 23:32:06 +02:00
Victor Stinner
beb4135b8c PyUnicode_AsWideCharString() takes a PyObject*, not a PyUnicodeObject*
All unicode functions uses PyObject* except PyUnicode_AsWideChar(). Fix the
prototype for the new function PyUnicode_AsWideCharString().
2010-10-07 01:02:42 +00:00
Victor Stinner
449057f2fa Issue #9979: Use PyUnicode_AsWideCharString() for _locale.strcoll()
It simplifies the code and prepare the surrogates support.
2010-09-29 10:30:43 +00:00
Daniel Stutzbach
8515eaefda Issue 8781: On systems a signed 4-byte wchar_t and a 4-byte Py_UNICODE, use memcpy to convert between the two (as already done when wchar_t is unsigned) 2010-08-24 21:57:33 +00:00
Victor Stinner
9e19ca42d3 locale.bindtextdomain(): use PyUnicode_FSConverter() to parse the filename 2010-06-11 22:09:51 +00:00
Antoine Pitrou
f95a1b3c53 Recorded merge of revisions 81029 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines

  Untabify C files. Will watch buildbots.
........
2010-05-09 15:52:27 +00:00