* bpo-36520: reset the encoded word offset when starting a new
line during an email header folding operation
* 📜🤖 Added by blurb_it.
* bpo-36520: add an additional test case, and provide descriptive
comments for the test_folding_with_utf8_encoding_* tests
* bpo-36520: fix whitespace issue
* bpo-36520: changes per reviewer request -- remove extraneous
backslashes; add whitespace between terminating quotes and
line-continuation backslashes; use "bpo-" instead of
"issue #" in comments
* bpo-21315: Fix parsing of encoded words with missing leading ws.
Because of missing leading whitespace, encoded word would get parsed as
unstructured token. This patch fixes that by looking for encoded words when
splitting tokens with whitespace.
Missing trailing whitespace around encoded word now register a defect
instead.
Original patch suggestion by David R. Murray on bpo-21315.
* bpo-30835: email: Fix AttributeError when parsing invalid Content-Transfer-Encoding
Parsing an email containing a multipart Content-Type, along with a
Content-Transfer-Encoding containing an invalid (non-ASCII-decodable) byte
will fail. email.feedparser.FeedParser._parsegen() gets the header and
attempts to convert it to lowercase before comparing it with the accepted
encodings, but as the header contains an invalid byte, it's returned as a
Header object rather than a str.
Cast the Content-Transfer-Encoding header to a str to avoid this.
Found using the AFL fuzzer.
Reported-by: Daniel Axtens <dja@axtens.net>
Signed-off-by: Andrew Donnellan <andrew@donnellan.id.au>
* Add email and NEWS entry for the bugfix.
* bpo-35805: Add parser for Message-ID header.
This parser is based on the definition of Identification Fields from RFC 5322
Sec 3.6.4.
This should also prevent folding of Message-ID header using RFC 2047 encoded
words and hence fix bpo-35805.
* Prevent folding of non-ascii message-id headers.
* Add fold method to MsgID token to prevent folding.
Two kind of mistakes:
1. Missed space. After concatenating there is no space between words.
2. Missed comma. Causes unintentional concatenating in a list of strings.
When attempting to base64-decode a payload of invalid length (1 mod 4),
properly recognize and handle it. The given data will be returned as-is,
i.e. not decoded, along with a new defect, InvalidBase64LengthDefect.
* Fix multiple typos in code comments
* Add spacing in comments (test_logging.py, test_math.py)
* Fix spaces at the beginning of comments in test_logging.py
The original algorithm tried to delegate the folding to the tokens so
that those tokens whose folding rules differed could specify the
differences. However, this resulted in a lot of duplicated code because
most of the rules were the same.
The new algorithm moves all folding logic into a set of functions
external to the token classes, but puts the information about which
tokens can be folded in which ways on the tokens...with the exception of
mime-parameters, which are a special case (which was not even
implemented in the old folder).
This algorithm can still probably be improved and hopefully simplified
somewhat.
Note that some of the test expectations are changed. I believe the
changes are toward more desirable and consistent behavior: in general
when (re) folding a line the canonical version of the tokens is
generated, rather than preserving errors or extra whitespace.
While there is not real bug in this case, using re.IGNORECASE without re.ASCII
leads unexpected behavior.
Instead of adding re.ASCII, this commit removes re.IGNORECASE flag because
it's easier and simpler.
This commit removes dead copy of the pattern in email.util module too.
While the pattern is same, it is compiled separately because it had different flags.
Leading whitespace was incorrectly dropped during folding of certain lines in the _header_value_parser's folding algorithm. This makes the whitespace handling code consistent.
