Commit Graph

205 Commits

Author SHA1 Message Date
Miss Islington (bot)
74735e2a40 bpo-33209: End framing at the end of C implementation of pickle.Pickler.dump(). (GH-6366)
(cherry picked from commit c869529ea9)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-04-03 14:35:11 -07:00
Serhiy Storchaka
1211c9a989 bpo-32503: Avoid creating too small frames in pickles. (#5127) 2018-01-20 16:42:44 +02:00
Serhiy Storchaka
5b76bdba07 bpo-31993: Do not use memoryview when pickle large strings. (#5154)
PyMemoryView_FromMemory() created a memoryview referring to
the internal data of the string.  When the string is destroyed
the memoryview become referring to a freed memory.
2018-01-13 00:28:31 +02:00
Serhiy Storchaka
0a2da50e18 bpo-31993: Do not create frames for large bytes and str objects (#5114)
when serialize into memory buffer with C pickle implementations.

This optimization already is performed when serialize into memory
with Python pickle implementations or into a file with both
implementations.
2018-01-11 13:03:20 +02:00
Olivier Grisel
3cd7c6e6eb bpo-31993: Do not allocate large temporary buffers in pickle dump. (#4353)
The picklers do no longer allocate temporary memory when dumping large
bytes and str objects into a file object. Instead the data is
directly streamed into the underlying file object.

Previously the C implementation would buffer all content and issue a
single call to file.write() at the end of the dump. With protocol 4
this behavior has changed to issue one call to file.write() per frame.

The Python pickler with protocol 4 now dumps each frame content as a
memoryview to an IOBytes instance that is never reused and the
memoryview is no longer released after the call to write. This makes it
possible for the file object to delay access to the memoryview of
previous frames without forcing any additional memory copy as was
already possible with the C pickler.
2018-01-06 17:18:54 +02:00
Serhiy Storchaka
3daaafb700 bpo-32037: Use the INT opcode for 32-bit integers in protocol 0 pickles. (#4407) 2017-11-16 09:44:43 +02:00
Serhiy Storchaka
6545256df9 bpo-32032: Test both implementations of module-level pickle API. (#4401) 2017-11-15 14:01:08 +02:00
Victor Stinner
8c663fd60e Replace KB unit with KiB (#4293)
kB (*kilo* byte) unit means 1000 bytes, whereas KiB ("kibibyte")
means 1024 bytes. KB was misused: replace kB or KB with KiB when
appropriate.

Same change for MB and GB which become MiB and GiB.

Change the output of Tools/iobench/iobench.py.

Round also the size of the documentation from 5.5 MB to 5 MiB.
2017-11-08 14:44:44 -08:00
Serhiy Storchaka
3410c01d83 Issue #17711: Fixed unpickling by the persistent ID with protocol 0.
Original patch by Alexandre Vassalotti.
2016-07-17 11:35:35 +03:00
Serhiy Storchaka
dec25afab1 Issue #17711: Fixed unpickling by the persistent ID with protocol 0.
Original patch by Alexandre Vassalotti.
2016-07-17 11:24:17 +03:00
Serhiy Storchaka
96c058b4de Fixed typo in pickle tests. 2016-03-04 09:40:03 +02:00
Serhiy Storchaka
be70002866 Fixed typo in pickle tests. 2016-03-04 09:39:47 +02:00
Serhiy Storchaka
58f8833e4c Make catched exception more specific and correct a comment. 2015-12-30 21:00:29 +02:00
Serhiy Storchaka
beb652ccc1 Make catched exception more specific and correct a comment. 2015-12-30 21:00:08 +02:00
Serhiy Storchaka
59fb6342a4 Issue #25761: Improved detecting errors in broken pickle data. 2015-12-06 22:01:35 +02:00
Serhiy Storchaka
92d5fbaf8f Issue #25761: Added more test cases for testing unpickling broken data.
Output raised exception at verbose level 2 (-vv).
2015-11-29 13:14:00 +02:00
Serhiy Storchaka
3c49710e9f Issue #25761: Added more test cases for testing unpickling broken data.
Output raised exception at verbose level 2 (-vv).
2015-11-29 13:13:24 +02:00
Serhiy Storchaka
7279befccb Issue #25761: Added more test cases for testing unpickling broken data.
Output raised exception at verbose level 2 (-vv).
2015-11-29 13:12:10 +02:00
Serhiy Storchaka
b1ce30281d Issue #23914: Fixed SystemError raised by unpickler on broken pickle data. 2015-11-23 15:20:21 +02:00
Serhiy Storchaka
4409c6cfae Issue #23914: Fixed SystemError raised by unpickler on broken pickle data. 2015-11-23 15:19:23 +02:00
Serhiy Storchaka
e9b3074cf9 Issue #23914: Fixed SystemError raised by unpickler on broken pickle data. 2015-11-23 15:17:43 +02:00
Serhiy Storchaka
41a87637c0 Issue #892902: Added new tests for pickling recursive collections. 2015-11-07 11:17:15 +02:00
Serhiy Storchaka
838b7cc0e7 Issue #892902: Added new tests for pickling recursive collections. 2015-11-07 11:17:02 +02:00
Serhiy Storchaka
608c213530 Issue #892902: Added new tests for pickling recursive collections. 2015-11-07 11:16:10 +02:00
Serhiy Storchaka
0d554d7ef1 Issue #24164: Objects that need calling `__new__` with keyword arguments,
can now be pickled using pickle protocols older than protocol version 4.
2015-10-10 22:42:18 +03:00