Victor Stinner
f2ddc6ac93
tokenizer: Remove unused tabs options ( #4422 )
...
Remove the following fields from tok_state structure which are now
used unused:
* altwarning: "Issue warning if alternate tabs don't match"
* alterror: "Issue error if alternate tabs don't match"
* alttabsize: "Alternate tab spacing"
Replace alttabsize variable with ALTTABSIZE define.
2017-11-17 01:25:47 -08:00
Jelle Zijlstra
ac317700ce
bpo-30406: Make async and await proper keywords ( #1669 )
...
Per PEP 492, 'async' and 'await' should become proper keywords in 3.7.
2017-10-05 23:24:46 -04:00
Albert-Jan Nijburg
c9ccacea3f
bpo-25324: add missing comma in Parser/tokenizer.c (GH-1910)
2017-06-01 13:51:27 -07:00
Albert-Jan Nijburg
fc354f0785
bpo-25324: copy tok_name before changing it ( #1608 )
...
* add test to check if were modifying token
* copy list so import tokenize doesnt have side effects on token
* shorten line
* add tokenize tokens to token.h to get them to show up in token
* move ERRORTOKEN back to its previous location, and fix nitpick
* copy comments from token.h automatically
* fix whitespace and make more pythonic
* change to fix comments from @haypo
* update token.rst and Misc/NEWS
* change wording
* some more wording changes
2017-05-31 16:00:21 +02:00
Berker Peksag
d2f4404bbb
Issue #28489 : Merge from 3.6
2017-02-05 04:33:11 +03:00
Berker Peksag
6f80562862
Issue #28489 : Fix comment in tokenizer.c
...
Patch by Ryan Gonzalez.
2017-02-05 04:32:39 +03:00
Victor Stinner
a5ed5f000a
Use _PyObject_CallNoArg()
...
Replace:
PyObject_CallObject(callable, NULL)
with:
_PyObject_CallNoArg(callable)
2016-12-06 18:45:50 +01:00
Serhiy Storchaka
06515833fe
Replaced outdated macros _PyUnicode_AsString and _PyUnicode_AsStringAndSize
...
with PyUnicode_AsUTF8 and PyUnicode_AsUTF8AndSize.
2016-11-20 09:13:07 +02:00
Benjamin Peterson
f5e8e8fc2b
merge 3.5 ( #24022 )
2016-09-18 23:44:02 -07:00
Benjamin Peterson
57bda335e1
merge 3.4
2016-09-18 23:43:18 -07:00
Benjamin Peterson
26d998cfdd
properly handle the single null-byte file ( closes #24022 )
2016-09-18 23:41:11 -07:00
Benjamin Peterson
5a715cfc57
merge 3.5 ( #27981 )
2016-09-12 22:07:14 -07:00
Benjamin Peterson
35ee948fa5
restructure fp_setreadl so as to avoid refleaks ( closes #27981 )
2016-09-12 22:06:58 -07:00
Brett Cannon
a721abac29
Issue #26331 : Implement the parsing part of PEP 515.
...
Thanks to Georg Brandl for the patch.
2016-09-09 14:57:09 -07:00
Christian Heimes
c6cc23d0b9
Skip unused value in tokenizer code
...
In the case of an escape character, c is never read. tok_next() is
used to advance the pointer.
CID 1225097
2016-09-09 00:09:45 +02:00
Serhiy Storchaka
ec39756960
Issue #22570 : Renamed Py_SETREF to Py_XSETREF.
2016-04-06 09:50:03 +03:00
Serhiy Storchaka
48842714b9
Issue #22570 : Renamed Py_SETREF to Py_XSETREF.
2016-04-06 09:45:48 +03:00
Benjamin Peterson
7285d520e0
remove duplicated check for fractions and complex numbers ( closes #26076 )
...
Patch by Oren Milman.
2016-03-24 22:43:23 -07:00
Serhiy Storchaka
a051bf3afb
Issue #26581 : Use the first coding cookie on a line, not the last one.
2016-03-20 23:47:48 +02:00
Serhiy Storchaka
e431d3c9aa
Issue #26581 : Use the first coding cookie on a line, not the last one.
2016-03-20 23:36:29 +02:00
Serhiy Storchaka
ef1585eb9a
Issue #25923 : Added more const qualifiers to signatures of static and private functions.
2015-12-25 20:01:53 +02:00
Serhiy Storchaka
f006940351
Issue #20440 : Massive replacing unsafe attribute setting code with special
...
macro Py_SETREF.
2015-12-24 10:39:57 +02:00
Serhiy Storchaka
5a57ade58e
Issue #20440 : Massive replacing unsafe attribute setting code with special
...
macro Py_SETREF.
2015-12-24 10:35:59 +02:00
Serhiy Storchaka
0304729ec4
Issue #25388 : Fixed tokenizer crash when processing undecodable source code
...
with a null byte.
2015-11-14 15:12:04 +02:00
Serhiy Storchaka
7e2b870b85
Issue #25388 : Fixed tokenizer crash when processing undecodable source code
...
with a null byte.
2015-11-14 15:11:17 +02:00
