Berker Peksag
6f80562862
Issue #28489 : Fix comment in tokenizer.c
...
Patch by Ryan Gonzalez.
2017-02-05 04:32:39 +03:00
Serhiy Storchaka
06515833fe
Replaced outdated macros _PyUnicode_AsString and _PyUnicode_AsStringAndSize
...
with PyUnicode_AsUTF8 and PyUnicode_AsUTF8AndSize.
2016-11-20 09:13:07 +02:00
Benjamin Peterson
f5e8e8fc2b
merge 3.5 ( #24022 )
2016-09-18 23:44:02 -07:00
Benjamin Peterson
57bda335e1
merge 3.4
2016-09-18 23:43:18 -07:00
Benjamin Peterson
26d998cfdd
properly handle the single null-byte file ( closes #24022 )
2016-09-18 23:41:11 -07:00
Benjamin Peterson
5a715cfc57
merge 3.5 ( #27981 )
2016-09-12 22:07:14 -07:00
Benjamin Peterson
35ee948fa5
restructure fp_setreadl so as to avoid refleaks ( closes #27981 )
2016-09-12 22:06:58 -07:00
Brett Cannon
a721abac29
Issue #26331 : Implement the parsing part of PEP 515.
...
Thanks to Georg Brandl for the patch.
2016-09-09 14:57:09 -07:00
Christian Heimes
c6cc23d0b9
Skip unused value in tokenizer code
...
In the case of an escape character, c is never read. tok_next() is
used to advance the pointer.
CID 1225097
2016-09-09 00:09:45 +02:00
Serhiy Storchaka
ec39756960
Issue #22570 : Renamed Py_SETREF to Py_XSETREF.
2016-04-06 09:50:03 +03:00
Serhiy Storchaka
48842714b9
Issue #22570 : Renamed Py_SETREF to Py_XSETREF.
2016-04-06 09:45:48 +03:00
Benjamin Peterson
7285d520e0
remove duplicated check for fractions and complex numbers ( closes #26076 )
...
Patch by Oren Milman.
2016-03-24 22:43:23 -07:00
Serhiy Storchaka
a051bf3afb
Issue #26581 : Use the first coding cookie on a line, not the last one.
2016-03-20 23:47:48 +02:00
Serhiy Storchaka
e431d3c9aa
Issue #26581 : Use the first coding cookie on a line, not the last one.
2016-03-20 23:36:29 +02:00
Serhiy Storchaka
ef1585eb9a
Issue #25923 : Added more const qualifiers to signatures of static and private functions.
2015-12-25 20:01:53 +02:00
Serhiy Storchaka
f006940351
Issue #20440 : Massive replacing unsafe attribute setting code with special
...
macro Py_SETREF.
2015-12-24 10:39:57 +02:00
Serhiy Storchaka
5a57ade58e
Issue #20440 : Massive replacing unsafe attribute setting code with special
...
macro Py_SETREF.
2015-12-24 10:35:59 +02:00
Serhiy Storchaka
0304729ec4
Issue #25388 : Fixed tokenizer crash when processing undecodable source code
...
with a null byte.
2015-11-14 15:12:04 +02:00
Serhiy Storchaka
7e2b870b85
Issue #25388 : Fixed tokenizer crash when processing undecodable source code
...
with a null byte.
2015-11-14 15:11:17 +02:00
Serhiy Storchaka
0d441119f5
Issue #25388 : Fixed tokenizer crash when processing undecodable source code
...
with a null byte.
2015-11-14 15:10:35 +02:00
Eric V. Smith
235a6f0984
Issue #24965 : Implement PEP 498 "Literal String Interpolation". Documentation is still needed, I'll open an issue for that.
2015-09-19 14:51:32 -04:00
Eric V. Smith
6408dc82fa
Fixed indentation.
2015-09-12 18:53:36 -04:00
Yury Selivanov
96ec934e75
Issue #24619 : Simplify async/await tokenization.
...
This commit simplifies async/await tokenization in tokenizer.c,
tokenize.py & lib2to3/tokenize.py. Previous solution was to keep
a stack of async-def & def blocks, whereas the new approach is just
to remember position of the outermost async-def block.
This change won't bring any parsing performance improvements, but
it makes the code much easier to read and validate.
2015-07-23 15:01:58 +03:00
Yury Selivanov
8fb307cd65
Issue #24619 : New approach for tokenizing async/await.
...
This commit fixes how one-line async-defs and defs are tracked
by tokenizer. It allows to correctly parse invalid code such
as:
>>> async def f():
... def g(): pass
... async = 10
and valid code such as:
>>> async def f():
... async def g(): pass
... await z
As a consequence, is is now possible to have one-line
'async def foo(): await ..' functions:
>>> async def foo(): return await bar()
2015-07-22 13:33:45 +03:00
Yury Selivanov
8085b80c18
Issue 24226: Fix parsing of many sequential one-line 'def' statements.
2015-05-18 12:50:52 -04:00
