pipenv is a tool used for managing virtual environments with pinned,
explicit dependencies. It is used for precisely recreating python
virtual environments.
pipenv uses two files to do this:
(1) Pipfile, which is similar in purpose and scope to what setup.cfg
lists. It specifies the requisite minimum to get a functional
environment for using this package.
(2) Pipfile.lock, which is similar in purpose to `pip freeze >
requirements.txt`. It specifies a canonical virtual environment used for
deployment or testing. This ensures that all users have repeatable
results.
The primary benefit of using this tool is to ensure *rock solid*
repeatable CI results with a known set of packages. Although I endeavor
to support as many versions as I can, the fluid nature of the Python
toolchain often means tailoring code for fairly specific versions.
Note that pipenv is *not* required to install or use this module; this is
purely for the sake of repeatable testing by CI or developers.
Here, a "blank" pipfile is added with no dependencies, but specifies
Python 3.6 for the virtual environment.
Pipfile will specify our version minimums, while Pipfile.lock specifies
an exact loadout of packages that were known to operate correctly. This
latter file provides the real value for easy setup of container images
and CI environments.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-id: 20210527211715.394144-15-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
When creating a source or binary distribution via 'python3 setup.py
<sdist|bdist>', the VERSION and PACKAGE.rst files aren't bundled by
default. Create a MANIFEST.in file that instructs the build tools to
include these so that installation from these files won't fail.
This is required by 'tox', as well as by the tooling needed to upload
packages to PyPI.
Exclude the 'README.rst' file -- that's intended as a guidebook to our
source tree, not a file that needs to be distributed.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-id: 20210527211715.394144-14-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Add short readmes to python/, python/qemu/, python/qemu/machine,
python/qemu/qmp, and python/qemu/utils that explain the directory
hierarchy. These readmes are visible when browsing the source on
e.g. gitlab/github and are designed to help new developers/users quickly
make sense of the source tree.
They are not designed for inclusion in a published manual.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-id: 20210527211715.394144-13-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Python infrastructure as it exists today is not capable reliably of
single-sourcing a package version from a parent directory. The authors
of pip are working to correct this, but as of today this is not possible.
The problem is that when using pip to build and install a python
package, it copies files over to a temporary directory and performs its
build there. This loses access to any information in the parent
directory, including git itself.
Further, Python versions have a standard (PEP 440) that may or may not
follow QEMU's versioning. In general, it does; but naturally QEMU does
not follow PEP 440. To avoid any automatically-generated conflict, a
manual version file is preferred.
I am proposing:
- Python tooling follows the QEMU version, indirectly, but with a major
version of 0 to indicate that the API is not expected to be
stable. This would mean version 0.5.2.0, 0.5.1.1, 0.5.3.0, etc.
- In the event that a Python package needs to be updated independently
of the QEMU version, a pre-release alpha version should be preferred,
but *only* after inclusion to the qemu development or stable branches.
e.g. 0.5.2.0a1, 0.5.2.0a2, and so on should be preferred prior to
5.2.0's release.
- The Python core tooling makes absolutely no version compatibility
checks or constraints. It *may* work with releases of QEMU from the
past or future, but it is not required to.
i.e., "qemu.machine" will, for now, remain in lock-step with QEMU.
- We reserve the right to split the qemu package into independently
versioned subpackages at a later date. This might allow for us to
begin versioning QMP independently from QEMU at a later date, if
we so choose.
Implement this versioning scheme by adding a VERSION file and setting it
to 0.6.0.0a1.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-id: 20210527211715.394144-12-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Add setup.cfg and setup.py, necessary for installing a package via
pip. Add a ReST document (PACKAGE.rst) explaining the basics of what
this package is for and who to contact for more information. This
document will be used as the landing page for the package on PyPI.
List the subpackages we intend to package by name instead of using
find_namespace because find_namespace will naively also packages tests,
things it finds in the dist/ folder, etc. I could not figure out how to
modify this behavior; adding allow/deny lists to setuptools kept
changing the packaged hierarchy. This works, roll with it.
I am not yet using a pyproject.toml style package manifest, because
"editable" installs are not defined (yet?) by PEP-517/518.
I consider editable installs crucial for development, though they have
(apparently) always been somewhat poorly defined.
Pip now (19.2 and later) now supports editable installs for projects
using pyproject.toml manifests, but might require the use of the
--no-use-pep517 flag, which somewhat defeats the point. Full support for
setup.py-less editable installs was not introduced until pip 21.1.1:
https://github.com/pypa/pip/pull/9547/commits/7a95720e796a5e56481c1cc20b6ce6249c50f357
For now, while the dust settles, stick with the de-facto
setup.py/setup.cfg combination supported by setuptools. It will be worth
re-evaluating this point again in the future when our supported build
platforms all ship a fairly modern pip.
Additional reading on this matter:
https://github.com/pypa/packaging-problems/issues/256https://github.com/pypa/pip/issues/6334https://github.com/pypa/pip/issues/6375https://github.com/pypa/pip/issues/6434https://github.com/pypa/pip/issues/6438
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-id: 20210527211715.394144-11-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
move python/qemu/*.py to python/qemu/[machine, qmp, utils]/*.py and
update import directives across the tree.
This is done to create a PEP420 namespace package, in which we may
create subpackages. To do this, the namespace directory ("qemu") should
not have any modules in it. Those files will go into new 'machine',
'qmp' and 'utils' subpackages instead.
Implement machine/__init__.py making the top-level classes and functions
from its various modules available directly inside the package. Change
qmp.py to qmp/__init__.py similarly, such that all of the useful QMP
library classes are available directly from "qemu.qmp" instead of
"qemu.qmp.qmp".
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-id: 20210527211715.394144-10-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Shift the open() call later so that the pylint pragma applies *only* to
that one open() call. Add a note that suggests why this is safe: the
resource is unconditionally cleaned up in _post_shutdown().
_post_shutdown is called after failed launches (see launch()), and
unconditionally after every call to shutdown(), and therefore also on
__exit__.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-id: 20210527211715.394144-6-jsnow@redhat.com
Message-id: 20210517184808.3562549-6-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Each instance of qemu.machine.QEMUMachine currently has a "test
directory", which may not have any relation to a "test", and it's
really a temporary directory.
Users instantiating the QEMUMachine class will be able to set the
location of the directory that will *contain* the QEMUMachine unique
temporary directory, so that parameter name has been changed from
test_dir to base_temp_dir.
A property has been added to allow users to access it without using
private attributes, and with that, the directory is created on first
use of the property.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210211220146.2525771-3-crosa@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
We enabled callers to interface directly with settimeout, but this
reacts poorly with blocking/nonblocking operation; as they are using the
same internal mechanism.
1. Whenever we change the blocking mechanism temporarily, always set it
back to what it was afterwards.
2. Disallow callers from setting a timeout of "0", which means
Non-blocking mode. This is going to create more weird problems than
anybody wants, so just forbid it.
I opt not to coerce '0' to 'None' to maintain the principal of least
surprise in mirroring the semantics of Python's interface.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20201009175123.249009-4-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Formalize the options used for checking the python library. You can run
mypy from the directory that mypy.ini is in by typing `mypy qemu/`.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20201009175123.249009-2-jsnow@redhat.com
[Edit: Added newline; thanks Bin Meng --js]
Signed-off-by: John Snow <jsnow@redhat.com>
John Snow
Cleber RosaandJohn Snow
Alex BennéeandPaolo Bonzini