Compare commits

..
11 Commits
Author SHA1 Message Date
Mikaël Capelle ee3faa0c87 Update for pybind11. 2022-05-05 12:59:28 +02:00
Mikaël Capelle c5ee7f8a43 Pypi-publish on tag. 2022-04-22 13:17:01 +02:00
Mikaël Capelle 33b7222b13 Fix enabledByDefault(). 2022-04-22 12:19:47 +02:00
Mikaël Capelle e083e0832e Linting. 2022-04-21 22:03:21 +02:00
Mikaël Capelle 478af17c58 Fix PyQt5 -> PyQt6. 2022-04-21 21:40:18 +02:00
Mikaël Capelle 6ca51655aa Fix version in config. 2022-04-21 21:19:37 +02:00
Mikaël Capelle 0ec04f4e9e Fix setup.py 2022-04-21 21:10:14 +02:00
Mikaël Capelle d43d067c84 Update for Qt6. 2022-04-21 21:03:48 +02:00
Mikaël Capelle 06e0fe1cca Update documentation. 2021-02-26 19:25:23 +01:00
Mikaël Capelle c57280578d Update PyQt5-stubs version. 2021-02-26 19:25:15 +01:00
Mikaël Capelle 178b1d111d Fix some argument names in the configuration file. 2021-02-07 20:20:02 +01:00
16 changed files with 4667 additions and 1723 deletions
+6 -5
View File
@@ -4,8 +4,8 @@
name: Upload Python Package
on:
release:
types: [published]
push:
tags: ["*.dev[0-9]+"]
jobs:
deploy:
@@ -18,13 +18,13 @@ jobs:
uses: frabert/replace-string-action@v1.1
id: version
with:
string: ${{ github.event.release.tag_name }}
string: ${{ github.ref_name }}
pattern: "v?([0-9][.][0-9][.][0-9]).*"
replace-with: "$1"
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: '3.8'
python-version: '3.10'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
@@ -36,5 +36,6 @@ jobs:
run: |
cd stubs/setup
cp ../${{ steps.version.outputs.replaced }}/mobase.pyi mobase-stubs/__init__.pyi
sed -i 's/__version__ = ".*"/__version__ = "${{ github.ref_name }}"/' mobase-stubs/__init__.pyi
python setup.py sdist bdist_wheel
twine upload dist/*
twine upload dist/*
+3 -3
View File
@@ -642,7 +642,7 @@ mobase:
The path to copy the entry to. If the path ends with / or \\, the entry will
be copied in the corresponding directory instead of replacing it. If the
given path is empty (`""`), the entry is copied directly under this tree.
policy: Policy to use to resolve conflicts.
insert_policy: Policy to use to resolve conflicts.
returns: The new entry (copy of the specified entry).
raises:
RuntimeError: If the entry could not be copied.
@@ -2947,7 +2947,7 @@ mobase:
getMissingAssets:
__doc__: Retrieve missing assets from the save.
args:
filepath: The save to find missing assets for.
save: The save to find missing assets for.
returns: |
A collection of missing assets and the modules that can supply those assets.
@@ -3095,7 +3095,7 @@ mobase:
args:
value: String to parse.
scheme: Scheme to use to parse the string.
manual_input: True if the given string should be treated as user input.
is_manual: True if the given string should be treated as user input.
scheme:
returns: The version scheme in effect for this VersionInfo.
File diff suppressed because it is too large Load Diff
+22 -23
View File
@@ -15,7 +15,7 @@ This guide assumes that:
`Python extension <https://marketplace.visualstudio.com/items?itemName=ms-python.python>`_.
- You have Python installed: https://www.python.org/downloads/.
- It is recommended but not mandatory to use the Python version that is used by MO2.
- You must use the Python version that used by MO2.
You can check the ``pythonXX.dll`` in the MO2 installation folder to find the Python version used by MO2 (``python38.dll`` means Python 3.8).
- You obviously need a valid MO2 installation: https://github.com/modorganizer2/modorganizer/releases
@@ -28,8 +28,8 @@ Preparation
-----------
**Note:** This part is optional but highly recommended if you want a proper environment to work with.
Everything here is written to be as simple as possible but you can of course adapt it to your preferences: use a python virtual
environment, use workspace settings instead of global ones, etc.
Everything here is written to be as simple as possible but you can of course adapt it to your preferences:
use a python virtual environment, use workspace settings instead of global ones, etc.
1. Get the ``mobase`` stubs
...........................
@@ -39,11 +39,14 @@ as ``flake8`` or ``mypy``.
Instead, we provide `stubs <https://stackoverflow.com/questions/59051631/what-is-the-use-of-stub-files-pyi-in-python>`_
which can be used for auto-completion or type-checking.
The stubs for ``mobase`` are available at https://github.com/ModOrganizer2/pystubs-generation/tree/master/stubs.
You want to download the ``mobase.pyi`` file in the folder corresponding to your MO2 version and put it under ``$MO2DIR/plugins/data``.
You can install the stubs for ``mobase`` using ``pip``:
.. code::
pip install mobase-stubs
This will install the stubs for ``mobase`` but also for PyQt5, which is heavily used by MO2.
**Note:** It is possible to put the stubs in a different location, but we are going to use ``$MO2DIR/plugins/data`` for PyQt5,
so we might as well use it for the stubs.
2. Configure Visual Studio Code for ``mobase``
..............................................
@@ -57,32 +60,28 @@ Open ``settings.json`` (Ctrl+Shift+P, then "Open Settings (JSON)"), and add the
"python.linting.enabled": true,
"python.linting.mypyEnabled": true,
"python.linting.flake8Enabled": true,
"python.autoComplete.extraPaths": [
"$MO2DIR\\plugins\\data",
]
3. Configure ``mypy`` to find the ``mobase`` stubs
..................................................
3. [Optional] Configure ``black`` to auto-format your source files
..................................................................
There are multiply way to configure ``mypy``:
This step is optional for your own plugin but recent MO2 plugins use ``black``
to get consistent formatting.
1. You can create a ``mypy.ini`` file somewhere containing:
You can install ``black`` with ``pip``:
.. code-block:: ini
.. code::
[mypy]
mypy_path = $MO2DIR\plugins\data
pip install black flake8-black
And then add the following to ``settings.json`` (with the correct path):
To configure Visual Studio Code to auto-format your code with ``black`` when saving, open ``settings.json``
(Ctrl+Shift+P, then "Open Settings (JSON)"), and add the following entries:
.. code-block:: json-object
"python.linting.mypyArgs": [
"--config-file=path-to-mypy.ini",
]
"editor.formatOnSave": true,
"editor.formatOnPaste": true,
"python.formatting.provider": "black",
2. You can set the ``MYPYPATH`` environment variable to ``$MO2DIR\plugins\data`` (this requires
restarting VS code).
4. [Optional] Automatically reload plugins during development
.............................................................
+1 -5
View File
@@ -1,9 +1,5 @@
# -*- encoding: utf-8 -*-
import logging
import sys
logging.basicConfig(stream=sys.stderr, format="%(levelname)s: %(message)s")
logger = logging.getLogger(__name__)
logger.setLevel(logging.WARNING)
LOGGER = logging.getLogger(__name__)
+8 -45
View File
@@ -1,41 +1,17 @@
# -*- encoding: utf-8 -*-
import importlib.machinery
import importlib.util
import os
import sys
from pathlib import Path
def load_module(name: str, path: Path):
# Create the loader:
loader = importlib.machinery.ExtensionFileLoader( # type: ignore
name, path.as_posix()
)
# Extract the spec:
spec = importlib.util.spec_from_loader(name, loader)
# Create the module and execute it?
module = importlib.util.module_from_spec(spec)
if module is None:
raise ImportError(f"Failed to import module {name} from {path}.")
loader.exec_module(module)
return module
def load_mobase(path: Path, moprivate: bool = False):
def load_mobase(path: Path):
"""
Load the mobase from the given MO2 installation path and
returns it.
Args:
path: Path to the MO2 installation (folder containing the ModOrganizer.exe).
moprivate: If True, the moprivate module will also be loaded and returned
alongside mobase.
Returns: The mobase module.
"""
@@ -51,22 +27,15 @@ def load_mobase(path: Path, moprivate: bool = False):
[str(path), str(path.joinpath("dlls")), os.environ.get("PATH", "")]
)
else:
os.add_dll_directory(str(path)) # type: ignore[attr-defined]
os.add_dll_directory(str(path.joinpath("dlls"))) # type: ignore[attr-defined]
os.add_dll_directory(str(path))
os.add_dll_directory(str(path.joinpath("dlls")))
# We need to add plugins/data to sys.path, mainly for PyQt5
sys.path.insert(1, path.joinpath("plugins", "data").as_posix())
# We need to add plugins/data to sys.path, mainly for PyQt6
sys.path.insert(1, path.joinpath("plugins", "plugin_python", "libs").as_posix())
mobase = load_module("mobase", path.joinpath("plugins", "data", "pythonrunner.dll"))
import mobase
if not moprivate:
return mobase
moprivate = load_module(
"moprivate", path.joinpath("plugins", "data", "pythonrunner.dll")
)
return mobase, moprivate
return mobase
if __name__ == "__main__":
@@ -83,13 +52,7 @@ if __name__ == "__main__":
default=None,
help="installation directory of Mod Organizer 2",
)
parser.add_argument(
"-p", "--private", action="store_true", help="also load the moprivate module"
)
args = parser.parse_args()
if args.private:
mobase, moprivate = load_mobase(args.install_dir, moprivate=True)
else:
mobase = load_mobase(args.install_dir)
mobase = load_mobase(args.install_dir)
+73 -369
View File
File diff suppressed because it is too large Load Diff
+177 -398
View File
File diff suppressed because it is too large Load Diff
+9 -53
View File
@@ -1,10 +1,10 @@
# -*- encoding: utf-8 -*-
from collections import OrderedDict
from typing import Optional, Dict, Union, List
from __future__ import annotations
from . import logger
from .mtypes import Class, Type, CType, Function
from collections import OrderedDict
from .mtypes import Class, Function
class MobaseRegister:
@@ -12,21 +12,16 @@ class MobaseRegister:
Class that register classes.
"""
objects: Dict[str, Union[Class, List[Function]]]
objects: dict[str, Class | list[Function]]
def __init__(self):
self.raw_objects: Dict[str, Union[type]] = OrderedDict()
self.raw_objects: dict[str, type] = OrderedDict()
self.objects = {}
self._cpptypes = {}
self.cpp2py = {}
def add_object(self, name, object):
self.raw_objects[name] = object
def make_object(
self, name: str, e: Optional[type] = None
) -> Union["Class", List["Function"]]:
def make_object(self, name: str, e: type | None = None) -> Class | list[Function]:
"""
Construct a Function, Class or Enum for the given object.
@@ -37,7 +32,7 @@ class MobaseRegister:
Returns:
A Class object for the given type, or a list of function overloads.
"""
from .parser import make_enum, make_class, is_enum, make_functions
from .parser import make_class, make_functions
if e is None:
e = self.raw_objects[name]
@@ -46,9 +41,7 @@ class MobaseRegister:
self.raw_objects[name] = e
if name not in self.objects:
if is_enum(e):
self.objects[name] = make_enum(name, e)
elif isinstance(e, type):
if isinstance(e, type):
self.objects[name] = make_class(name, e, self)
elif callable(e):
self.objects[name] = make_functions(name, e)
@@ -68,42 +61,5 @@ class MobaseRegister:
"""
return self.objects[name]
def register_type(self, ptype: "Type", ctype: "CType"):
"""Register an equivalence between a python name and a C++ name.
Args:
python_name: Name of the Python class.
cpp_name: Name of the C++ class.
"""
# Register the const equivalent for smart pointers:
cname = ctype.name
if ctype.is_smart_pointer():
if cname.find(" const >") != -1:
c2name = cname.replace(" const >", ">")
if c2name in self._cpptypes:
ptype = self.cpp2py[c2name]
# Not the const, replace the const one:
else:
c2name = cname.replace(">", " const >")
if c2name in self._cpptypes and self.cpp2py[c2name].is_object():
self._cpptypes[c2name] = ctype
self.cpp2py[c2name] = ptype
logger.warning(
"Replace registration {} [c++] with {} [python] using {}"
" information.".format(c2name, ptype.name, cname) # noqa: E501
)
if cname not in MOBASE_REGISTER.cpp2py:
self._cpptypes[cname] = ctype
self.cpp2py[cname] = ptype
logger.info("Registered {} [c++] as {} [python].".format(cname, ptype.name))
@property
def py2cpp(self):
result = {v.name: [] for v in self.cpp2py.values()}
for k in self.cpp2py:
result[self.cpp2py[k].name].append(self._cpptypes[k])
return result
MOBASE_REGISTER = MobaseRegister()
+266 -194
View File
File diff suppressed because it is too large Load Diff
+30 -34
View File
@@ -1,9 +1,9 @@
# -*- encoding: utf-8 -*-
from typing import TextIO, List, Union, Tuple
from typing import TextIO
from . import logger
from .mtypes import Function, Class, Method, Property, Enum
from . import LOGGER
from .mtypes import Class, Enum, Function, Method, Property
from .utils import Settings
@@ -36,7 +36,7 @@ class Writer:
self._print('__version__ = "{}"'.format(version))
self._print()
def print_imports(self, imports: List[Union[str, Tuple[str, List[str]]]]):
def print_imports(self, imports: list[str | tuple[str, list[str]]]):
"""
Print the given imports.
"""
@@ -55,12 +55,10 @@ class Writer:
if fn.has_overloads():
self._print("{}@overload".format(indent))
srtype = ""
sig_return_type = ""
if not fn.ret.type.is_none():
srtype = " -> " + fn.ret.type.typing(self._settings)
sig_return_type = " -> " + fn.ret.type.typing()
fargs = fn.args
largs: List[str] = []
if isinstance(fn, Method):
if fn.is_static():
self._print("{}@staticmethod".format(indent))
@@ -68,17 +66,19 @@ class Writer:
if fn.is_abstract():
self._print("{}@abc.abstractmethod".format(indent))
largs.insert(0, "self")
fargs = fargs[1:]
for i, arg in enumerate(fargs):
tmp = "{}: {}".format(arg.name, arg.type.typing(self._settings))
python_args: list[str] = []
for i, arg in enumerate(fn.args):
tmp = "{}: {}".format(arg.name, arg.type.typing())
if arg.has_default_value():
tmp += " = {}".format(arg.value)
largs.append(tmp)
sargs = ", ".join(largs)
python_args.append(tmp)
self._print("{}def {}({}){}:".format(indent, fn.name, sargs, srtype), end="")
self._print(
"{}def {}({}){}:".format(
indent, fn.name, ", ".join(python_args), sig_return_type
),
end="",
)
# Add the documentation, if any:
doc = ""
@@ -93,9 +93,12 @@ class Writer:
if any(arg.doc for arg in args):
doc += "\nArgs:\n"
for arg in args:
adocl = arg.doc.strip().split("\n")
adoc = "\n".join([adocl[0]] + [" " + ldoc for ldoc in adocl[1:]])
doc += " " + arg.name + ": " + adoc + "\n"
arg_doc_list = arg.doc.strip().split("\n")
arg_doc = "\n".join(
[arg_doc_list[0]]
+ [" " + line_doc for line_doc in arg_doc_list[1:]]
)
doc += " " + arg.name + ": " + arg_doc + "\n"
if not fn.ret.type.is_none() and fn.ret.doc:
doc += "\nReturns:\n " + fn.ret.doc.strip() + "\n"
@@ -103,13 +106,7 @@ class Writer:
if fn.raises:
doc += "\nRaises:\n"
for rai in fn.raises:
doc += (
" "
+ rai.type.typing(self._settings)
+ ": "
+ rai.doc.strip()
+ "\n"
)
doc += " " + rai.type.typing() + ": " + rai.doc.strip() + "\n"
if doc:
self._print()
@@ -127,7 +124,7 @@ class Writer:
"""
if prop.type.is_object() or prop.type.is_any():
logger.warning(
LOGGER.warning(
"Property {}.{} does not have a specified type.".format(
cls.name, prop.name
)
@@ -135,15 +132,13 @@ class Writer:
self._print("{}@property".format(indent))
self._print(
"{}def {}(self) -> {}: ...".format(
indent, prop.name, prop.type.typing(self._settings)
)
"{}def {}(self) -> {}: ...".format(indent, prop.name, prop.type.typing())
)
if not prop.is_read_only():
self._print("{}@{}.setter".format(indent, prop.name))
self._print(
"{}def {}(self, arg0: {}): ...".format(
indent, prop.name, prop.type.typing(self._settings)
indent, prop.name, prop.type.typing()
)
)
self._print()
@@ -179,8 +174,8 @@ class Writer:
self._print()
# Inner classes:
for iclass in cls.inner_classes:
self.print_class(iclass, indent=indent + " ")
for inner_class in cls.inner_classes:
self.print_class(inner_class, indent=indent + " ")
self._print()
# Constants:
@@ -191,7 +186,7 @@ class Writer:
typing = ""
if constant.type is not None:
typing = ": {}".format(constant.type.typing(self._settings))
typing = ": {}".format(constant.type.typing())
# Note: We do not print the value, we use ...
self._print(
@@ -217,6 +212,7 @@ class Writer:
cls.methods,
key=lambda m: (m.name != "__init__", not m.is_special(), m.name),
)
for method in methods:
self.print_function(method, indent=indent + " ")
+38 -19
View File
@@ -2,20 +2,20 @@
import argparse
import logging
from pathlib import Path
from typing import cast
import black
import isort
from generator import logger
from generator import LOGGER
from generator.loader import load_mobase
from generator.register import MOBASE_REGISTER
from generator.mtypes import Class, Function, PyType
from generator.parser import is_enum
from generator.mtypes import Type, Class, Function
from generator.register import MOBASE_REGISTER
from generator.utils import Settings, clean_class
from generator.writer import Writer
parser = argparse.ArgumentParser("Stubs generator for the MO2 python interface")
parser.add_argument(
"install_dir",
@@ -44,8 +44,13 @@ parser.add_argument(
args = parser.parse_args()
logging.basicConfig()
LOGGER.setLevel(logging.WARNING)
if args.verbose:
logger.setLevel(logging.INFO)
LOGGER.setLevel(logging.INFO)
output_path = cast(Path, args.output)
# Load settings from the configuration:
settings: Settings = Settings(register=MOBASE_REGISTER)
@@ -67,21 +72,26 @@ for name in dir(mobase):
if name in settings.ignore_names:
continue
# We do not want the real MoVariant.
# we do not want the real MoVariant
if name == "MoVariant":
continue
# For now, ignore this since it is a submodule and we
# not handle them.
# ignore the private module
if name == "private":
continue
# for now, ignore this since it is a submodule and we
# not handle them
if name == "widgets":
continue
# IPlugin is not the real object
if name == "IPlugin":
continue
objects.append((name, getattr(mobase, name)))
# Enum first, and then alphabetical. Might cause issue with base classes, so
# maybe create a kind of dependency...
# For argument or return types, this should not be an issue since we quote
# everything from mobase.
# enum first, and then alphabetical, should be fine with the __future__ import
objects = sorted(
objects, key=lambda e: (isinstance(e[1], type), not is_enum(e[1]), e[0])
)
@@ -107,7 +117,7 @@ for n, o in objects:
settings.patch_functions(c)
else:
logger.critical(
LOGGER.critical(
"Cannot generated stubs for {}, unsupported object type.".format(n)
)
@@ -115,15 +125,20 @@ for n, o in objects:
with open(args.output, "w") as output:
writer = Writer(output, settings)
# the __future__ import must be at the beginning
writer.print_imports([("__future__", ["annotations"])])
writer.print_version(settings.mobase["__version__"]) # type: ignore
writer.print_imports(
[
"abc",
("enum", ["Enum"]),
("pathlib", ["Path"]),
(
"typing",
[
"Dict",
"Iterable",
"Iterator",
"List",
"Tuple",
@@ -132,18 +147,21 @@ with open(args.output, "w") as output:
"Optional",
"Callable",
"overload",
"Set",
"TypeVar",
"Type",
],
),
"PyQt5.QtCore",
"PyQt5.QtGui",
"PyQt5.QtWidgets",
"PyQt6.QtCore",
"PyQt6.QtGui",
"PyQt6.QtWidgets",
]
)
# Needs to define the MVariant and GameFeatureType type:
writer._print("MoVariant = {}".format(Type.MO_VARIANT))
writer._print(f"MoVariant = {PyType.MO_VARIANT}")
writer._print(f"FileWrapper = {PyType.FILE_WRAPPER}")
writer._print(f"DirectoryWrapper = {PyType.DIRECTORY_WRAPPER}")
writer._print('GameFeatureType = TypeVar("GameFeatureType")')
writer._print()
@@ -164,8 +182,9 @@ with open(args.output, "w") as output:
writer.print_function(fn)
black.format_file_in_place(
args.output,
output_path,
fast=False,
mode=black.Mode(is_pyi=args.output.name.endswith("pyi")),
write_back=black.WriteBack.YES,
)
isort.api.sort_file(output_path)
+10 -3
View File
@@ -15,19 +15,26 @@ warn_return_any = True
warn_unused_configs = True
namespace_packages = True
[isort]
profile = black
multi_line_output = 3
[tox:tox]
skipsdist = true
envlist = py38-lint
envlist = py310-lint
[testenv:py38-lint]
[testenv:py310-lint]
skip_install = true
deps =
black
mypy
flake8
flake8-black
PyQt5-stubs
git+https://github.com/TilmanK/PyQt6-stubs.git
types-PyYAML
isort
commands =
black generator main.py --check --diff
flake8 generator main.py
mypy generator main.py
isort -c generator main.py
+6 -6
View File
@@ -795,7 +795,7 @@ class IFileTree(FileTreeEntry):
self,
entry: "FileTreeEntry",
path: str = "",
policy: "IFileTree.InsertPolicy" = InsertPolicy.FAIL_IF_EXISTS,
insert_policy: "IFileTree.InsertPolicy" = InsertPolicy.FAIL_IF_EXISTS,
) -> "FileTreeEntry":
"""
Move the given entry to the given path under this tree.
@@ -816,7 +816,7 @@ class IFileTree(FileTreeEntry):
path: The path to copy the entry to. If the path ends with / or \\, the entry will
be copied in the corresponding directory instead of replacing it. If the
given path is empty (`""`), the entry is copied directly under this tree.
policy: Policy to use to resolve conflicts.
insert_policy: Policy to use to resolve conflicts.
Returns:
The new entry (copy of the specified entry).
@@ -3900,12 +3900,12 @@ class SaveGameInfo(abc.ABC):
def __init__(self): ...
@abc.abstractmethod
def getMissingAssets(self, filepath: "ISaveGame") -> Dict[str, List[str]]:
def getMissingAssets(self, save: "ISaveGame") -> Dict[str, List[str]]:
"""
Retrieve missing assets from the save.
Args:
filepath: The save to find missing assets for.
save: The save to find missing assets for.
Returns:
A collection of missing assets and the modules that can supply those assets.
@@ -4155,7 +4155,7 @@ class VersionInfo:
self,
value: str,
scheme: "VersionScheme" = VersionScheme.DISCOVER,
manual_input: bool = False,
is_manual: bool = False,
):
"""
Update this VersionInfo by parsing the given string using the given scheme.
@@ -4163,7 +4163,7 @@ class VersionInfo:
Args:
value: String to parse.
scheme: Scheme to use to parse the string.
manual_input: True if the given string should be treated as user input.
is_manual: True if the given string should be treated as user input.
"""
...
def scheme(self) -> "VersionScheme":
File diff suppressed because it is too large Load Diff
+3 -2
View File
@@ -11,6 +11,7 @@
import io
import os
import re
from setuptools import setup
@@ -46,8 +47,8 @@ setup(
version=find_version("mobase-stubs", "__init__.pyi"),
package_data={"mobase-stubs": ["*.pyi"]},
packages=["mobase-stubs"],
install_requires=["PyQt5-stubs==5.14.2"],
python_requires="==3.8.*",
install_requires=[],
python_requires="==3.10.*",
classifiers=[
"Intended Audience :: Developers",
"Programming Language :: Python :: 3.8",