Miss Islington (bot)
b571958f54
Improve highlighting of some code blocks. (GH-6401)
...
(cherry picked from commit 46936d5a71storchaka@gmail.com >
2018-04-09 07:56:44 -07:00
Mariatta
ea0f7c26ce
bpo-31394: Make tokenize.rst PEP 8-compliant (GH-3526)
...
The last commit contained lines longer than 80 characters.
2017-09-12 21:00:00 -07:00
Aivar Annamaa
5f8fbf917e
bpo-31394: Clarify documentation about token type attribute (GH-3469)
...
Make it clear that Ellipsis tokens also have type attribute set to token.OP.
2017-09-12 20:24:03 -07:00
Serhiy Storchaka
5cefb6cfdd
bpo-25324: Move the description of tokenize tokens to token.rst. ( #1911 )
2017-06-06 18:43:35 +03:00
Terry Jan Reedy
fa089b9b0b
Issue #22558 : Add remaining doc links to source code for Python-coded modules.
...
Reformat header above separator line (added if missing) to a common format.
Patch by Yoni Lavi.
2016-06-11 15:02:54 -04:00
Serhiy Storchaka
dba903993a
Issue #23921 : Standardized documentation whitespace formatting.
...
Original patch by James Edwards.
2016-05-10 12:01:23 +03:00
Martin Panter
20b1bfa6fb
Issue #26127 : Fix links in tokenize documentation; patch by Silent Ghost
2016-01-16 04:32:52 +00:00
Serhiy Storchaka
d65c9496da
Issue #25523 : Further a-to-an corrections.
2015-11-02 14:10:23 +02:00
Benjamin Peterson
96e0430dfb
document TokenError and unclosed expression behavior ( closes #12063 )
...
Patch by Amandine Lee.
2014-06-07 17:47:41 -07:00
Meador Inge
00c7f85298
Issue #2134 : Add support for tokenize.TokenInfo.exact_type.
2012-01-19 00:44:45 -06:00
Meador Inge
14c0f03b58
Issue #12943 : python -m tokenize support has been added to tokenize.
2011-10-07 08:53:38 -05:00
Raymond Hettinger
4f707fd316
Separate source link from main text.
2011-01-10 19:54:11 +00:00
Raymond Hettinger
1048094037
Move source links to consistent location and remove wordy, big yellow boxes.
2011-01-10 03:26:08 +00:00
Éric Araujo
6e6cb8e0e4
Provide links to Python source where the code is short, readable and
...
informative adjunct to the docs. Forward-port of Raymond's r86225 and
r86245 using the new source reST role added in #10334 .
2010-11-16 19:13:50 +00:00
Victor Stinner
58c0752a33
Issue #10335 : Add tokenize.open(), detect the file encoding using
...
tokenize.detect_encoding() and open it in read only mode.
2010-11-09 01:08:59 +00:00
Raymond Hettinger
6c60d099e5
Improve the repr for the TokenInfo named tuple.
2010-09-09 04:32:39 +00:00
Benjamin Peterson
b3a482962d
show a common usage of detect_encoding
2010-03-18 22:43:41 +00:00
Benjamin Peterson
689a558098
in tokenize.detect_encoding(), return utf-8-sig when a BOM is found
2010-03-18 22:29:52 +00:00
Antoine Pitrou
4adb288f4e
Issue #7631 : Fix undefined references to the "built-in file object", which
...
has ceased to be.
2010-01-04 18:50:53 +00:00
Ezio Melotti
a8f6f1e2d6
Fixed capitalization and markup; added imports in the example
2009-12-20 12:24:57 +00:00
Raymond Hettinger
a48db39992
Issue #5857 : tokenize.tokenize() now returns named tuples.
2009-04-29 00:34:27 +00:00
Georg Brandl
ae2dbe2543
#5486 : typos.
2009-03-13 19:04:40 +00:00
Georg Brandl
48310cd3f2
Remove trailing whitespace.
2009-01-03 21:18:54 +00:00
Georg Brandl
c28e1fa71f
Merged revisions 64002-64003,64012,64036-64037,64047,64050-64052,64054-64055,64066,64071 via svnmerge from
...
svn+ssh://pythondev@svn.python.org/python/trunk
........
r64002 | travis.oliphant | 2008-06-07 00:33:21 +0200 (Sat, 07 Jun 2008) | 1 line
Add long double check support to configure test.
........
r64003 | travis.oliphant | 2008-06-07 00:39:47 +0200 (Sat, 07 Jun 2008) | 1 line
Remove locking part of new buffer protocol.
........
r64012 | facundo.batista | 2008-06-07 15:36:36 +0200 (Sat, 07 Jun 2008) | 4 lines
Finished bug #2451 . Fixed the retrying part to make it
more robust.
........
r64036 | georg.brandl | 2008-06-08 10:54:40 +0200 (Sun, 08 Jun 2008) | 2 lines
#3028 : tokenize passes the physical line.
........
r64037 | georg.brandl | 2008-06-08 10:59:38 +0200 (Sun, 08 Jun 2008) | 2 lines
Argh, I read it wrong. Reverted 64036 and added a clarifying remark.
........
r64047 | raymond.hettinger | 2008-06-09 03:28:30 +0200 (Mon, 09 Jun 2008) | 1 line
Issue3065: Fixed pickling of named tuples. Added tests.
........
r64050 | raymond.hettinger | 2008-06-09 08:54:45 +0200 (Mon, 09 Jun 2008) | 1 line
Issue #2138 : Add math.factorial().
........
r64051 | raymond.hettinger | 2008-06-09 10:33:37 +0200 (Mon, 09 Jun 2008) | 1 line
Let set.union() and set.update() accept multiple inputs.
........
r64052 | raymond.hettinger | 2008-06-09 11:29:17 +0200 (Mon, 09 Jun 2008) | 1 line
Address double-rounding scenarios by setting all variables to long doubles.
........
r64054 | raymond.hettinger | 2008-06-09 13:24:47 +0200 (Mon, 09 Jun 2008) | 1 line
Unhappy buildbots. Revert 64052. Long doubles have unexpected effects on some builds.
........
r64055 | raymond.hettinger | 2008-06-09 15:07:27 +0200 (Mon, 09 Jun 2008) | 1 line
Let set.intersection() and set.intersection_update() take multiple input arguments.
........
r64066 | robert.schuppenies | 2008-06-10 12:10:31 +0200 (Tue, 10 Jun 2008) | 2 lines
Issue 3048: Fixed sys.getsizeof for unicode objects.
........
r64071 | thomas.heller | 2008-06-10 16:07:12 +0200 (Tue, 10 Jun 2008) | 3 lines
NEWS entry for:
Add an optional 'offset' parameter to byref, defaulting to zero.
........
2008-06-10 19:20:26 +00:00
Trent Nelson
428de65ca9
- Issue #719888 : Updated tokenize to use a bytes API. generate_tokens has been
...
renamed tokenize and now works with bytes rather than strings. A new
detect_encoding function has been added for determining source file encoding
according to PEP-0263. Token sequences returned by tokenize always start
with an ENCODING token which specifies the encoding used to decode the file.
This token is used to encode the output of untokenize back to bytes.
Credit goes to Michael "I'm-going-to-name-my-first-child-unittest" Foord from Resolver Systems for this work.
2008-03-18 22:41:35 +00:00
