This is another preparation patch for the transition to python 3.x.
With Python 3.x, the output we get from the logger is going to be
a byte string, which we need to then decode into a string.
Change-Id: I2307b90f2e2cccaed8a93fa589f82fba0064c28b
TN: U530-006
This is another preparation patch for the transition to Python 3.x.
With Python 3.x, we need to make sure that the input used when
calling sendmail is converted to a byte string. We also then need
to make sure that the script's output is decoded into a string
when printing it.
Change-Id: I1b792638fb77c8d1b4ee2197b29b63922e0fe211
TN: U530-006
This is another preparation patch for the transition to Python 3.x.
With Python 3.x, we need to make sure that the input used when
calling the filer cmd is converted to a byte string. We also
then need to make sure that the script's output is decoded into
a string.
Change-Id: I324410dd5c9b1e811252803b854d0f06ca65435d
TN: U530-006
This commit is part of the prep work for the transition to Python 3.x,
where the input of the hooks need to be encoded before send it,
and where the output needs to be decoded.
Change-Id: I68fa9de5b4c8b932f931725174a6424716855c2a
TN: U530-006
This is another preparation patch for the transition to Python 3.x,
where the input first needs to be encoded before it is passed to
the git command to be executed.
Change-Id: I2d8aac02a17b5d5765ab5e2c357bc15ead4f2c64
TN: U530-006
This is another preparation patch for the transition to Python 3.x.
The script's input needs to be encoded when called, and its output
needs to be decoded into a string for us to process it.
For input encoding, the same approach as for decoding is taken:
In order to make progress towards Python 3.x support while at
the same time preserving support for Python 2.x, we introduce
a new function "encode_utf8" which only performs the encoding
on Python 3.x. With Python 2.x, the function just returns the string
unmodified.
Change-Id: Ieb47d32c756405cdd0d300254e8cd7c8c3db50b5
TN: U530-006
This commit is preparation work for the transition to Python 3.x,
where the output obtained by running Git commands will become
bytes as opposed to a string. In the vast majority of cases,
we'll want to decode that output into a string. Ideally, we would
want to do this in a way that is both compatible with Python 2.x
and Python 3.x, but we have found that this requires a lot of
work with many changes spread all over the code. So, instead,
what this commit does is introduce the concept of decoding
the output, but with the decoding only occurring when running
under Python 3.x.
That way, we can make progress towards Python 3.x while preserving
the behavior under Python 2.x intact.
Change-Id: I189577798ee96cba1fa55c7356babf102575642f
TN: U530-006
This is another commit to prepare for the transition to Python 3.x,
where the revlog of the given Commit object will be (unicode) strings.
In this situation decoding before checking the string against
the ISO-8859-15 is not necessary.
This commit therefore enhances the function to handle the case
where the revlog is already unicode by checking whether it offers
the "decode" method or not. The code is written in such a way that
the Python 3.x case doesn't use a temporary variable. That way, when
Python 2.x support is removed, we can simply remove the corresponding
half of the if-else block, and just keep the other half untouched.
TN: U530-006
Change-Id: I674c639eb7af656cdc084cc510b9eca90751f18f
This commit changes the guess_encoding function, in the Python 3.x
case, to return UTF-8 instead of iso-8859-15 for strings with content
which is compatible with iso-8859-15. The reason for this change is
to standardize a little more towards UTF-8 as our encoding of choice
when generating textual data.
Using iso-8859-15 was not wrong as far as I can tell, but I believe
the majority of users and applications have switched to UTF-8 now,
so this commit simply follows that trend.
This commit won't have any effect until we switch the testsuite over
to Python 3.x, where some email header fields will end up being
encoded using UTF-8 instead of iso-8859-15. For the moment, no visible
change within the current testing, as it only supports being run
with Python 2.x.
TN: U530-006
Change-Id: I6b008cd5c2e12a4dbb97a567fb35a76f40e9782a
The goal of this commit is to include the updates.sendmail module
in our testing strategy, in order to make sure that the hooks are
passing email data down to the sendmail program without issues.
This will become particularly important when we switch over to
using Python 3.x, because of the strong distinction between bytes
and strings with newer versions of Python which can cause a lot
problems. Hence the need to use this code during our testing.
The main strategy introduced by this commit to achieve this is
fairly simple: The testsuite framework introduces a new minimal
script to be called in place of the standard sendmail. A new
environment variable called GIT_HOOKS_SENDMAIL is introduced
allowing the testsuite to tell the hooks to use its own (fake)
sendmail instead of the system one. With that in place,
the old code bypassing the use of updates.sendmail can be removed,
thus allowing the testsuite to include it as part of the testing.
The testsuite's (fake) sendmail script was written in a way to
mimick the old bypassing code, so there is no change in output.
Parallel to that, the hooks are enhanced to check that we can
indeed find sendmail, and otherwise return immediately with
an error if not. This way, we avoid emails silently being
dropped due to the missing sendmail.
A couple of testcases are also added to double-check some
specific error situations.
Note that I tried to think of ways to split this patch into
smaller individual parts, but couldn't really find a way to
do so in a meaningful way, while at the same time producing
a commit where the coverage report stays clean (0 lines missed).
TN: U530-006 (transition to Python 3.x)
TN: U924-032 (test for sendmail not found)
TN: U924-034 (test for sendmail override when in testsuite mode)
Change-Id: I74b993592ec6d701347bbca5283a42e037411f1c
The implementation of this module was originally inherited from
gnatpython, where it was trying first to call sendmail, and if
not available, then fallback on using Python's smtplib instead.
This commit removes support for using smtplib, and instead assumes
that sendmail is always available.
The reasons for this change are two-fold:
- For all the users of these scripts I know of, sendmail is always
available, so we haven't really used the smtplib fallback.
- While this code is currently excluded during testing (to avoid
sending emails while running the testsuite), I'd like to enhance
our testing strategy to start including this code as part of
the testing. In particular, one thing we can do is for the testuite
to eventually provide its own version of a sendmail program that
would dump the traces to stdout rather than actually send an email.
On the other hand, if we were to keep smtplib support as a fallback,
I do not see how we could test that part without actually having it
send email, something we absolutely do not want.
This is related to the effort of moving to Python 3.x, where Python
now makes a strong distinction between bytes and strings when
passing data between processes. With Python 3.x, it's much more
important to always test that data is passed correctly.
TN: U530-006
Change-Id: Ic2153be62a80906dce709fb3d622e1194ca7c869
This pragma allows us to exclude this block when doing coverage analysis
when testing the git-hooks using a Python 3.x interpreter.
Change-Id: Id2f61c2a1cbf965c93693771b6dcb9d55d6a2708
TN: U530-006
This is another commit to prepare for the transition to Python 3.x,
where text will be converted early to unicode strings, instead of
being kept as byte strings. When passed to "guess_encoding" in
Python 3.x, unicode strings don't have a "decode" method, as
the strings are already decoded. As a result, the current
implementation always returns None (no encoding found), because
we get an exception calling the non-existent method, promptly
trapped and wrongly interpreted as being a decoding error.
To prepare the transition to Python 3.x, this commit adds
a check to see if we have a byte-string. If we do, then do
the same as before. Otherwise, we must have a unicode string,
and so check the encodings by trying to encoding rather than
decode the string.
TN: U530-006
Change-Id: I50cf689fec8c205a6e48b42fac3a95a6bb9886b4
This commit adds a couple of "# pragma: py2-only" comments to
a couple of code blocks which are only expected to be run when
the hooks are tested with Python 2.x (these two code blocks are
conditioned on the version of Python being less than 3).
This will help us manage the transition to Python 3.x until we are
able to drop support for Python 2.x. That way, we can run coverage
analysis with both Python 2.x and Python 3.x, and get the Python 3.x
coverage analyzer to ignore those blocks we know we cannot cover
with Python 3.x.
Once the transition to Python 3.x is over, we will remove those code
blocks.
TN: U530-006
Change-Id: I44f1cd883c3fdf4e487e1e553158517e721416df
This is another Python 3.x preparation patch, which fixes a situation
where we change a dictionary size while iterating over it, something
that became apparent while exercizing the hooks with Python 3.x.
This commit avoids this by rewriting this part of the function
in a way that aboves the dictionary iteration entirely.
TN: U530-006
Change-Id: Ia22b28919ac2dabc3e22ed1b3652e14d799e566b
While testing the behavior of the git-hooks under Python 3.x,
I noticed that the order of some output was not the same as
when running it with Python 2.x. When investigating further,
I found that the order with Python 2.x made better sense, and
that the different order was caused by stdout/stderr buffering.
This commit turns buffering off entirely in an effort to make
sure that output sent to stdout & stderr gets seen in the order
that it was sent.
As it happens, while this commit was aimed at Python 3.x,
running the testsuite showed that we had one testcase where
the order when using Python 2.x was also incorrect, and
therefore misleading, for a couple of tests. The tests'
expected output was double-checked, and adjusted accordingly,
with additional comments explaining what each part was about.
TN: U530-006
Change-Id: Iaf2c5266e13a645dab006e1f7f4cb553cbd5704f
This commit changes some code which currently only works with
Python 2.x, to make it compatible with both Python 2.x and
Python 3.x.
TN: U530-006
Change-Id: I0c2d2b90b445c9310a74850d81c1de42ed6f648f
This commit enhances the error message being generated when
filename collisions are detected to list the collisions in
alphabetical order. While this doesn't make a big difference
for the users, it ensures that the error message is deterministic.
Found while working on the transition to Python 3.x (U530-006)
Change-Id: Ie6787022a1833ea7f0db23a17c79d3bb9ce2ba96
For some reason that I was unable to determine, trying to cover
the case failure to fork during the second fork causes an error
when running the testsuite with coverage, such as:
$ ./coverage.sh -j16 --enable-color tests/LC28-010__daemon_utest
We get a crash during the execution of the at_exit handling when
the coverage framework tries to create a file in the root ("/")
directory, presumably to save the coverage information. I couldn't
figure out why this was happening, nor could I find a fix. Since
the code we're trying is not expected to change much over time,
and the second fork error handling is only about damage control,
I gave up on the idea. Instead, I commented out the corresponding
testing code, so as for it to be available should we try to solve
that mystery again, and I added a "pragma nocover" in run_in_daemon.
This had the side-effect of dropping one call to syslog that was
contributing to full coverage. This commit simply extends the unit
test to include a call to that function as well.
TN: U627-007
Change-Id: Iaacfd174747ff43a921f5d17e3a74900ad96a78e
Support for this specific file is purely AdaCore-specific and
historical. Since then, we have introduced (much!) better ways
to support users who want to suppress checks for a given commit.
We know this feature hasn't been used for many years, so it is
time to remove support for it. Note that it wasn't even documented.
TN: U627-004
Change-Id: Ie67532158abc1d302a3d98e57835f15dadfe0817
This commit makes sure that the stdout and stderr output has been
flushed so that no output is left buffered by the time we call os.fork.
Otherwise, if there was any output still buffered, we would be getting
the same output printed multiple times, once by the parent process,
and once by each child process.
Found while working on the transition to Python 3.x (U530-006).
TN: U619-004
Change-Id: I9c38bfc410041e58de2a8f5d7d340f7bff599df4
I noticed while working on the transition to Python 3.x that
this argument is not actually used, so this commit removes it.
It makes the API for this function simpler and also helps avoid
some pitfalls (as shown by one of the limitations we documented
that this commit removes).
TN: U530-006
Change-Id: I5ced7c6a979c5381d35498b06f1af8c569a3f9ed
This is a preparation patch for the transition to Python 3.x.
Currently, we have the following code:
| files_to_check = filter(needs_style_check_p, files_to_check)
| if not files_to_check:
| debug("style_check_commit: no files to style-check")
| return
With Python 2.x, the "filter" function constructs a list, so
we can indeed check for emptiness using the logical operator "not".
With Python 3.x, however, filter returns an iterator, and using
the logical operator "not" no longer has the same semantics at all.
The easiest way to handle this is to simply convert the result to
a tuple.
TN: U530-006
Change-Id: I334f2657728405226d7da577e893b7c64a3220fb
This commit updates the pre-commit hook to black version 21.5b1.
The hooks where then re-run on all files to update their formatting
to this new version of black.
Change-Id: Ib0866745ef8432cf93380a4d83fa23a479eb4a49
The "filename_list" of the style_check_files function is expected
to be a list. However, as it happens, we have a call to this function
where the filename_list parameter was computed using the "filter"
function.
files_to_check = filter(needs_style_check_p, files_to_check)
style_check_files(files_to_check, new_rev, project_name)
This is no problem when trying to iterate on the list alone, but
at one point inside the "style_check_files" funtion, we try to
add to it like so:
for filename in filename_list + aux_files:
With Python 3.x, this causes a crash because the "filter" function
now returns a generator rather than a list. Seems it seems desirable
to allow "style_check_files" to accept any iterable as "filename_list",
this commit modifies the code to allow it.
TN: U530-006
Change-Id: I8e5caaf09d177096788c522d7bff8168eb158ec7
