Compare commits

..
24 changed files with 2717 additions and 2871 deletions
+2 -2
View File
@@ -5,7 +5,7 @@ name: Upload Python Package
on:
push:
tags: ["*.dev[0-9]+"]
tags: ["*"]
jobs:
deploy:
@@ -35,7 +35,7 @@ jobs:
TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
run: |
cd stubs/setup
cp ../${{ steps.version.outputs.replaced }}/mobase.pyi mobase-stubs/__init__.pyi
cp -r ../${{ steps.version.outputs.replaced }}/mobase-stubs/* mobase-stubs/
sed -i 's/__version__ = ".*"/__version__ = "${{ github.ref_name }}"/' mobase-stubs/__init__.pyi
python setup.py sdist bdist_wheel
twine upload dist/*
+2 -1
View File
@@ -2,9 +2,10 @@
.mypy_cache
__pycache__
.vscode
**/*.egg-info
# The 'bin/' directory:
bin
docs/build
docs/mobase.py
docs/source/api
docs/source/api
+30 -48
View File
@@ -19,20 +19,9 @@ You can install stubs for a specific version of MO2:
pip install mobase-stubs==2.3.2.*
```
If you want development stubs, you can install them this way:
```bash
# Clone this repository:
git clone https://github.com/ModOrganizer2/pystubs-generation.git
# Install the stubs:
cd pystubs-generation/stubs/setup
pip install .
```
Some words of warning:
- The stubs are as correct as possible, but some errors are expected.
- If you see a `InterfaceNotImplemented` class anywhere in the stubs, it means that
a proper interface is currently not available.
- Some classes are said (in the stubs) to inherit `QWidget` or `QObject`. This is true
on the C++ side but NOT on the python side. The inheritance is only added to help with
auto-completion since these classes also override `__getattr__` to dispatch to the
@@ -44,35 +33,39 @@ Some words of warning:
The stubs are generated using python by parsing the `mobase` module.
You need the version of python that matches your current MO2 installation: e.g., if you
have a `python38.dll` in your MO2 installation path, then you need **Python 3.8**.
have a `python310.dll` in your MO2 installation path, then you need **Python 3.10**.
To generate the stubs, you can run:
```
# Change the output folder to whatever you want:
python main.py -c configs\config-2.4.yml ${MO2_INSTALL_PATH}
```bash
# install the package (-e if you want editable mode)
pip install [-e] .
# change the output folder to whatever you want
mo2-stubs-generator -c configs/config-2.4.yml -o mobase-stubs ${MO2_INSTALL_PATH}
```
Where `${MO2_INSTALL_PATH}` is the path to your MO2 installation (the one containing `ModOrganizer.exe`).
Where `${MO2_INSTALL_PATH}` is the path to your MO2 installation (the one
containing `ModOrganizer.exe`).
The stubs are generated under `stubs/setup/mobase-stubs/__init__.pyi`, you
can change the output file by using the `-o` option
The latest stubs are kept under `stubs/setup/mobase-stubs/__init__.pyi`,
and when a new version is released, the stubs are backed-up under
`stubs/x.y.z/mobase.pyi`.
The stubs are generated under `stubs/setup/mobase-stubs` by default, you
can change the output file by using the `-o` option.
The stubs under `stubs/setup/mobase-stubs` should not be committed as these are
generated from the version stubs under `stubs/${VERSION}/mobase-stubs`.
A few options are available for `main.py`:
A few options are available for `mo2-stubs-generator`:
```
usage: Stubs generator for the MO2 python interface [-h] [-o OUTPUT] [-v] [-c CONFIG] INSTALL_DIR
```bash
$ mo2-stubs-generator --help
usage: stubs generator for the MO2 python interface [-h] [-o OUTPUT] [-v] [-c CONFIG] INSTALL_DIR
positional arguments:
INSTALL_DIR installation directory of Mod Organizer 2
optional arguments:
options:
-h, --help show this help message and exit
-o OUTPUT, --output OUTPUT
output file (default stubs/setup/mobase-stubs/__init__.pyi)
output folder (default stubs/setup/mobase-stubs)
-v, --verbose verbose mode (all logs go to stderr)
-c CONFIG, --config CONFIG
configuration file
@@ -81,19 +74,7 @@ optional arguments:
The stubs generator will try hard to find a valid stubs for all classes
and methods of `mobase`.
A lot of information is available through the `-v` options. Without it,
only conversions or fixes
considered "strange" will be shown.
For instance, here is the output with the current `config-2.4.yml` file:
```
WARNING: Replacing IOrganizer::FileInfo with FileInfo.
WARNING: Replacing IOrganizer::FileInfo with FileInfo.
WARNING: Replacing IPluginInstaller::EInstallResult with InstallResult.
WARNING: Replacing IPluginInstaller::EInstallResult with InstallResult.
```
As you can see, only a few types were manually fixed (specified in
`config-2.4.yml`).
only conversions or fixes considered "strange" will be shown.
## Configuration file
@@ -102,20 +83,21 @@ deduced by `main` (or are too complex to deduce), and the documentation for ever
## Uploading the stubs to pypi
The upload of the stubs to https://pypi.org/project/mobase-stubs/ should be
done automatically when a new Github release is made.
The upload of the stubs to [https://pypi.org/project/mobase-stubs/](https://pypi.org/project/mobase-stubs/)
should be done automatically when a new Github tag is pushed.
## Extras — Starts a python interpreter with `mobase`
## Extras — Using `mobase` in a Python interpreter
It is possible to start a (i)python interpret with `mobase` imported by running:
It is possible to start a (i)python interpreter with `mobase` imported by running
```
python -im generator.loader ${MO2_INSTALL_PATH}
```bash
python -i -m mo2.stubs.generator.loader ${MO2_INSTALL_PATH}
```
This has no real usage except for MO2 developers since most classes from the `mobase` module cannot be instantiated.
**Note:** Most classes in `mobase` cannot be instantiated, so this is mostly intended
for MO2 developers.
# License
## License
The MIT License (MIT)
+26 -35
View File
@@ -1,24 +1,14 @@
---
version: 1
# version of the configuration
version: 2
# This is the list of type to replace:
replacements:
Organizer::FileInfo: FileInfo
IOrganizer::FileInfo: FileInfo
IPluginInstaller::EInstallResult: InstallResult
GuessedValue< QString>: GuessedString
# List of names to ignores:
ignores:
- toPyQt
# version of the stubs - this is overridden when publishing
__version__: "2.5.0"
# This is the root of the mobase module and will contain everything
# related to functions / classes, including their documentation.
mobase:
# Version of the stubs.
__version__: 2.5.0.dev2
getFileVersion:
__doc__: Retrieve the file version of the given executable.
args:
@@ -329,7 +319,7 @@ mobase:
__doc__: |
The parent tree containing this entry, or a `None` if this entry is the root
or the parent tree is unreachable.
type: Optional["IFileTree"]
type: Optional[IFileTree]
path:
__doc__: |
@@ -673,7 +663,7 @@ mobase:
__doc__: |
The entry at the given location, or `None` if the entry was not found or
was not of the correct type.
type: Optional[Union["IFileTree", "FileTreeEntry"]]
type: Optional[Union[IFileTree, FileTreeEntry]]
insert:
__doc__: |
@@ -1208,7 +1198,7 @@ mobase:
IModRepositoryBridge:
__bases__:
- PyQt5.QtCore.QObject
- PyQt6.QtCore.QObject
signals[]:
descriptionAvailable:
@@ -1372,7 +1362,7 @@ mobase:
type: MoVariant
desc: The data that was included in the request.
error:
type: PyQt5.QtNetwork.QNetworkReply.NetworkError
type: PyQt6.QtNetwork.QNetworkReply.NetworkError
desc: The actual error.
message:
type: str
@@ -2034,6 +2024,9 @@ mobase:
The name of the launcher executable to run (relative to the game folder), or an
empty string if there is no launcher.
getSupportURL:
returns: An URL for the support page of this game.
iniFiles:
returns: |
The list of INI files this game uses. The first file in the list should be the
@@ -2223,13 +2216,6 @@ mobase:
args:
parent: The parent widget.
IPluginInstallerCustom:
__doc__: |
Custom installer for mods. Custom installers receive the archive name and have to go
from there. They have to be able to extract the archive themselves.
Example of such installers are the external NCC installer or the OMOD installer.
_manager:
abstract: false
returns: The installation manager.
@@ -2238,6 +2224,13 @@ mobase:
abstract: false
returns: The parent widget.
IPluginInstallerCustom:
__doc__: |
Custom installer for mods. Custom installers receive the archive name and have to go
from there. They have to be able to extract the archive themselves.
Example of such installers are the external NCC installer or the OMOD installer.
install:
__doc__: |
Install the given archive.
@@ -2278,14 +2271,6 @@ mobase:
representing the archive and can modify what to install and where by editing this structure.
Actually extracting the archive is handled by the manager.
_manager:
abstract: false
returns: The installation manager.
_parentWidget:
abstract: false
returns: The parent widget.
install:
__doc__: |
Install a mod from an archive filetree.
@@ -2920,7 +2905,7 @@ mobase:
type: int
desc:
fileTime:
type: PyQt5.QtCore.QDateTime
type: PyQt6.QtCore.QDateTime
desc:
gameName:
type: str
@@ -3013,7 +2998,7 @@ mobase:
parent: The parent widget.
returns:
__doc__: A SaveGameInfoWidget to display information about save game.
type: Optional["ISaveGameInfoWidget"]
type: Optional[ISaveGameInfoWidget]
ScriptExtender:
__doc__:
@@ -3152,3 +3137,9 @@ mobase:
scheme:
returns: The version scheme in effect for this VersionInfo.
mobase.widgets:
TaskDialog: {}
TaskDialogButton: {}
-8
View File
@@ -1,8 +0,0 @@
# -*- encoding: utf-8 -*-
import logging
import sys
logging.basicConfig(stream=sys.stderr, format="%(levelname)s: %(message)s")
logger = logging.getLogger(__name__)
logger.setLevel(logging.WARNING)
-95
View File
@@ -1,95 +0,0 @@
# -*- encoding: utf-8 -*-
import importlib.machinery
import importlib.util
import os
import sys
from pathlib import Path
def load_module(name: str, path: Path):
# Create the loader:
loader = importlib.machinery.ExtensionFileLoader( # type: ignore
name, path.as_posix()
)
# Extract the spec:
spec = importlib.util.spec_from_loader(name, loader)
assert spec is not None
# Create the module and execute it?
module = importlib.util.module_from_spec(spec)
if module is None:
raise ImportError(f"Failed to import module {name} from {path}.")
loader.exec_module(module)
return module
def load_mobase(path: Path, moprivate: bool = False):
"""
Load the mobase from the given MO2 installation path and
returns it.
Args:
path: Path to the MO2 installation (folder containing the ModOrganizer.exe).
moprivate: If True, the moprivate module will also be loaded and returned
alongside mobase.
Returns: The mobase module.
"""
# We need absolute path for loading DLL and modules:
path = path.resolve()
# Adding to PATH environment variable for python < 3.8 and
# via os.add_dll_directory (python >= 3.8).
# See: https://stackoverflow.com/a/58632354/2666289
if sys.version_info < (3, 8):
os.environ["PATH"] = os.pathsep.join(
[str(path), str(path.joinpath("dlls")), os.environ.get("PATH", "")]
)
else:
os.add_dll_directory(str(path)) # type: ignore[attr-defined]
os.add_dll_directory(str(path.joinpath("dlls"))) # type: ignore[attr-defined]
# We need to add plugins/data to sys.path, mainly for PyQt6
sys.path.insert(1, path.joinpath("plugins", "data").as_posix())
mobase = load_module("mobase", path.joinpath("plugins", "data", "pythonrunner.dll"))
if not moprivate:
return mobase
moprivate = load_module(
"moprivate", path.joinpath("plugins", "data", "pythonrunner.dll")
)
return mobase, moprivate
if __name__ == "__main__":
import argparse
parser = argparse.ArgumentParser(
"Load mobase python module from MO2 installation directory"
)
parser.add_argument(
"install_dir",
metavar="INSTALL_DIR",
type=Path,
default=None,
help="installation directory of Mod Organizer 2",
)
parser.add_argument(
"-p", "--private", action="store_true", help="also load the moprivate module"
)
args = parser.parse_args()
if args.private:
mobase, moprivate = load_mobase(args.install_dir, moprivate=True)
else:
mobase = load_mobase(args.install_dir)
-713
View File
File diff suppressed because it is too large Load Diff
-598
View File
File diff suppressed because it is too large Load Diff
-109
View File
@@ -1,109 +0,0 @@
# -*- encoding: utf-8 -*-
from collections import OrderedDict
from typing import Dict, List, Optional, Union
from . import logger
from .mtypes import Class, CType, Function, Type
class MobaseRegister:
"""
Class that register classes.
"""
objects: Dict[str, Union[Class, List[Function]]]
def __init__(self):
self.raw_objects: Dict[str, Union[type]] = OrderedDict()
self.objects = {}
self._cpptypes = {}
self.cpp2py = {}
def add_object(self, name, object):
self.raw_objects[name] = object
def make_object(
self, name: str, e: Optional[type] = None
) -> Union["Class", List["Function"]]:
"""
Construct a Function, Class or Enum for the given object.
Args:
name: The name of the object to inspect.
e: The object to inspect, or None to fetch it from the underlying list.
Returns:
A Class object for the given type, or a list of function overloads.
"""
from .parser import is_enum, make_class, make_enum, make_functions
if e is None:
e = self.raw_objects[name]
if name not in self.raw_objects:
self.raw_objects[name] = e
if name not in self.objects:
if is_enum(e):
self.objects[name] = make_enum(name, e)
elif isinstance(e, type):
self.objects[name] = make_class(name, e, self)
elif callable(e):
self.objects[name] = make_functions(name, e)
return self.objects[name]
def get_object(self, name: str):
"""
Retrieve the object if the given name. Fails if no object with this
name exists (if `make_object(name, ...)` has never been called).
Args:
name: Name of the object to retrieve.
Returns:
The object with the given name.
"""
return self.objects[name]
def register_type(self, ptype: "Type", ctype: "CType"):
"""Register an equivalence between a python name and a C++ name.
Args:
python_name: Name of the Python class.
cpp_name: Name of the C++ class.
"""
# Register the const equivalent for smart pointers:
cname = ctype.name
if ctype.is_smart_pointer():
if cname.find(" const >") != -1:
c2name = cname.replace(" const >", ">")
if c2name in self._cpptypes:
ptype = self.cpp2py[c2name]
# Not the const, replace the const one:
else:
c2name = cname.replace(">", " const >")
if c2name in self._cpptypes and self.cpp2py[c2name].is_object():
self._cpptypes[c2name] = ctype
self.cpp2py[c2name] = ptype
logger.warning(
"Replace registration {} [c++] with {} [python] using {}"
" information.".format(c2name, ptype.name, cname) # noqa: E501
)
if cname not in MOBASE_REGISTER.cpp2py:
self._cpptypes[cname] = ctype
self.cpp2py[cname] = ptype
logger.info("Registered {} [c++] as {} [python].".format(cname, ptype.name))
@property
def py2cpp(self):
result = {v.name: [] for v in self.cpp2py.values()}
for k in self.cpp2py:
result[self.cpp2py[k].name].append(self._cpptypes[k])
return result
MOBASE_REGISTER = MobaseRegister()
-514
View File
File diff suppressed because it is too large Load Diff
-169
View File
@@ -1,169 +0,0 @@
# -*- encoding: utf-8 -*-
import argparse
import logging
from pathlib import Path
import black
from generator import logger
from generator.loader import load_mobase
from generator.mtypes import Class, Function, Type
from generator.parser import is_enum
from generator.register import MOBASE_REGISTER
from generator.utils import Settings, clean_class
from generator.writer import Writer
parser = argparse.ArgumentParser("Stubs generator for the MO2 python interface")
parser.add_argument(
"install_dir",
metavar="INSTALL_DIR",
type=Path,
default=None,
help="installation directory of Mod Organizer 2",
)
parser.add_argument(
"-o",
"--output",
type=Path,
default="stubs/setup/mobase-stubs/__init__.pyi",
help="output file (default stubs/setup/mobase-stubs/__init__.pyi)",
)
parser.add_argument(
"-v", "--verbose", action="store_true", help="verbose mode (all logs go to stderr)"
)
parser.add_argument(
"-c",
"--config",
type=argparse.FileType("r"),
default=None,
help="configuration file",
)
args = parser.parse_args()
if args.verbose:
logger.setLevel(logging.INFO)
# Load settings from the configuration:
settings: Settings = Settings(register=MOBASE_REGISTER)
if args.config is not None:
settings = Settings(MOBASE_REGISTER, args.config)
# Parse mobase:
# Load mobase (cannot simply do "import mobase"):
mobase = load_mobase(Path(args.install_dir))
# List of objects:
objects = []
for name in dir(mobase):
if name.startswith("__"):
continue
if name in settings.ignore_names:
continue
# We do not want the real MoVariant.
if name == "MoVariant":
continue
# For now, ignore this since it is a submodule and we
# not handle them.
if name == "widgets":
continue
objects.append((name, getattr(mobase, name)))
# Enum first, and then alphabetical. Might cause issue with base classes, so
# maybe create a kind of dependency...
# For argument or return types, this should not be an issue since we quote
# everything from mobase.
objects = sorted(
objects, key=lambda e: (isinstance(e[1], type), not is_enum(e[1]), e[0])
)
for n, o in objects:
MOBASE_REGISTER.add_object(n, o)
# Process everything:
for n, o in objects:
# Create the corresponding object:
c = MOBASE_REGISTER.make_object(n, o)
if isinstance(c, Class):
# Clean the class (e.g., remove duplicates methods due to wrappers):
clean_class(c, settings)
# Path the class using the configuration:
settings.patch_class(c)
elif isinstance(c, list) and isinstance(c[0], Function):
settings.patch_functions(c)
else:
logger.critical(
"Cannot generated stubs for {}, unsupported object type.".format(n)
)
# Write everything:
with open(args.output, "w") as output:
writer = Writer(output, settings)
writer.print_version(settings.mobase["__version__"]) # type: ignore
writer.print_imports(
[
"abc",
("enum", ["Enum"]),
(
"typing",
[
"Dict",
"Iterator",
"List",
"Tuple",
"Union",
"Any",
"Optional",
"Callable",
"overload",
"TypeVar",
"Type",
],
),
"PyQt6.QtCore",
"PyQt6.QtGui",
"PyQt6.QtWidgets",
]
)
# Needs to define the MVariant and GameFeatureType type:
writer._print("MoVariant = {}".format(Type.MO_VARIANT))
writer._print('GameFeatureType = TypeVar("GameFeatureType")')
writer._print()
# This is a class to represent interface not implemented:
writer.print_class(Class("InterfaceNotImplemented", [], []))
writer._print()
for n, o in objects:
# Get the corresponding object:
c = MOBASE_REGISTER.get_object(n)
if isinstance(c, Class):
writer.print_class(c)
elif isinstance(c, list) and isinstance(c[0], Function):
for fn in c:
writer.print_function(fn)
black.format_file_in_place(
args.output,
fast=False,
mode=black.Mode(is_pyi=args.output.name.endswith("pyi")),
write_back=black.WriteBack.YES,
)
+4 -4
View File
@@ -34,7 +34,7 @@ deps =
types-PyYAML
isort
commands =
black generator main.py --check --diff
flake8 generator main.py
mypy generator main.py
isort -c generator main.py
black src --check --diff
flake8 src
mypy src
isort -c src
+35
View File
@@ -0,0 +1,35 @@
# -*- encoding: utf-8 -*-
from setuptools import find_namespace_packages, setup
install_requires = ["black", "isort"]
dev_requires = [
"black",
"flake8-black",
"flake8",
"types-chardet",
]
setup(
name="mo2-stubs-generator",
version="1.0.0",
package_dir={"": "src"},
packages=find_namespace_packages(where="src", include=["mo2.*"]),
author="Holt59",
author_email="capelle.mikael@gmail",
description="Python stubs generator for mobase (MO2 Python API).",
long_description=open("README.md").read(),
url="https://github.com/ModOrganizer2/pystubs-generation",
classifiers=[
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
],
license="MIT",
install_requires=install_requires,
extras_require={"dev": dev_requires},
entry_points={
"console_scripts": ["mo2-stubs-generator=mo2.stubs.generator.__main__:main"],
},
)
+5
View File
@@ -0,0 +1,5 @@
# -*- encoding: utf-8 -*-
import logging
LOGGER = logging.getLogger(__name__)
+240
View File
@@ -0,0 +1,240 @@
# -*- encoding: utf-8 -*-
import argparse
import inspect
import logging
from pathlib import Path
from typing import Callable, TextIO, cast
import black
import isort
from . import LOGGER
from .loader import load_mobase
from .mtypes import Class, Function, PyType
from .parser import is_enum
from .register import MobaseRegister
from .utils import Settings, clean_class
from .writer import Writer
def extract_objects(module: object, skips: list[str] = []) -> list[tuple[str, object]]:
objects: list[tuple[str, object]] = []
for name in dir(module):
if name.startswith("__") or name in skips:
continue
obj = getattr(module, name)
# skip submodules
if inspect.ismodule(obj):
continue
objects.append((name, obj))
return objects
def add_mobase_header(writer: Writer):
writer.print_imports(
[
"abc",
("enum", ["Enum"]),
("pathlib", ["Path"]),
(
"typing",
[
"Dict",
"Iterator",
"List",
"Tuple",
"Union",
"Any",
"Optional",
"Callable",
"overload",
"Sequence",
"Set",
"TypeVar",
"Type",
],
),
"PyQt6.QtCore",
"PyQt6.QtGui",
"PyQt6.QtWidgets",
]
)
# Needs to define the MVariant and GameFeatureType type:
writer._print(f"MoVariant = {PyType.MO_VARIANT}")
writer._print(f"FileWrapper = {PyType.FILE_WRAPPER}")
writer._print(f"DirectoryWrapper = {PyType.DIRECTORY_WRAPPER}")
writer._print('GameFeatureType = TypeVar("GameFeatureType")')
writer._print()
def add_mobase_widgets_header(writer: Writer):
writer.print_imports(
[
(
"typing",
["List", "Tuple", "Union", "overload"],
),
"PyQt6.QtCore",
"PyQt6.QtGui",
"PyQt6.QtWidgets",
]
)
def main():
parser = argparse.ArgumentParser("stubs generator for the MO2 python interface")
parser.add_argument(
"install_dir",
metavar="INSTALL_DIR",
type=Path,
default=None,
help="installation directory of Mod Organizer 2",
)
parser.add_argument(
"-o",
"--output",
type=Path,
default=Path("stubs/setup/mobase-stubs"),
help="output folder (default stubs/setup/mobase-stubs)",
)
parser.add_argument(
"-v",
"--verbose",
action="store_true",
help="verbose mode (all logs go to stderr)",
)
parser.add_argument(
"-c",
"--config",
type=Path,
default=None,
help="configuration file",
)
args = parser.parse_args()
logging.basicConfig()
LOGGER.setLevel(logging.WARNING)
if args.verbose:
LOGGER.setLevel(logging.INFO)
output_path: Path = args.output
config_path: Path | None = args.config
# create the register
register = MobaseRegister()
# load mobase (cannot simply do "import mobase")
mobase = load_mobase(Path(args.install_dir))
# headers
module_headers: dict[str, Callable[[Writer], None]] = {
"mobase": add_mobase_header,
"mobase.widgets": add_mobase_widgets_header,
}
# list of objects directly in mobase
module_objects: dict[str, list[tuple[str, object]]] = {
"mobase": extract_objects(
mobase,
[
# we do not want the real MoVariant
"MoVariant",
# the "real" IPlugin is IPluginBase
"IPlugin",
],
),
"mobase.widgets": extract_objects(mobase.widgets),
}
for name, objects in module_objects.items():
# load settings from the configuration
settings: Settings = Settings(register)
if config_path is not None:
with open(config_path, "r") as fp:
settings = Settings(register, fp, module=name)
for n, o in objects:
register.add_object(n, o)
# enum first, and then alphabetical, should be fine with the __future__ import
objects = sorted(
objects, key=lambda e: (isinstance(e[1], type), not is_enum(e[1]), e[0])
)
# Process everything:
for n, o in objects:
# Create the corresponding object:
c = register.make_object(n, o)
if isinstance(c, Class):
# Clean the class (e.g., remove duplicates methods due to wrappers):
clean_class(c)
# Path the class using the configuration:
settings.patch_class(c)
elif isinstance(c, list) and isinstance(c[0], Function):
settings.patch_functions(c)
else:
LOGGER.critical(
"Cannot generated stubs for {}, unsupported object type.".format(n)
)
output_folder = output_path
if name != "mobase":
output_folder = output_path.joinpath(
name.replace("mobase.", "").replace(".", "/")
)
# create directory if required
output_folder.mkdir(parents=True, exist_ok=True)
# write everything
with open(output_folder.joinpath("__init__.pyi"), "w") as output:
writer = Writer(package=name, output=output, settings=settings)
# the __future__ import must be at the beginning
writer.print_imports([("__future__", ["annotations"])])
writer.print_version(settings.version)
module_headers[name](writer)
for n, o in objects:
# Get the corresponding object:
c = register.get_object(n)
if isinstance(c, Class):
writer.print_class(c)
elif isinstance(c, list) and isinstance(c[0], Function):
for fn in c:
writer.print_function(fn)
black.format_file_in_place(
output_folder.joinpath("__init__.pyi"),
fast=False,
mode=black.Mode(is_pyi=True),
write_back=black.WriteBack.YES,
)
isort.api.sort_file(output_folder.joinpath("__init__.pyi"))
if __name__ == "__main__":
main()
+58
View File
@@ -0,0 +1,58 @@
# -*- encoding: utf-8 -*-
import os
import sys
from pathlib import Path
def load_mobase(path: Path):
"""
Load the mobase from the given MO2 installation path and
returns it.
Args:
path: Path to the MO2 installation (folder containing the ModOrganizer.exe).
Returns: The mobase module.
"""
# We need absolute path for loading DLL and modules:
path = path.resolve()
# Adding to PATH environment variable for python < 3.8 and
# via os.add_dll_directory (python >= 3.8).
# See: https://stackoverflow.com/a/58632354/2666289
if sys.version_info < (3, 8):
os.environ["PATH"] = os.pathsep.join(
[str(path), str(path.joinpath("dlls")), os.environ.get("PATH", "")]
)
else:
os.add_dll_directory(str(path))
os.add_dll_directory(str(path.joinpath("dlls")))
# We need to add plugins/data to sys.path, mainly for PyQt6
sys.path.insert(1, path.joinpath("plugins", "plugin_python", "libs").as_posix())
import mobase # type: ignore
return mobase
if __name__ == "__main__":
import argparse
parser = argparse.ArgumentParser(
"Load mobase python module from MO2 installation directory"
)
parser.add_argument(
"install_dir",
metavar="INSTALL_DIR",
type=Path,
default=None,
help="installation directory of Mod Organizer 2",
)
args = parser.parse_args()
mobase = load_mobase(args.install_dir)
+432
View File
@@ -0,0 +1,432 @@
# -*- encoding: utf-8 -*-
from __future__ import annotations
import re
class PyType:
"""
Class representing a python type.
"""
# The `MoVariant` actual type - This should be list["MoVariant"] and
# Dict[str, "MoVariant"], but mypy (and other type checkers) do not
# handle recursive definition yet:
MO_VARIANT = """Union[None, bool, int, str, list[Any], dict[str, Any]]"""
# File/Directory wrappers
FILE_WRAPPER = """Union[str, PyQt6.QtCore.QFileInfo, Path]"""
DIRECTORY_WRAPPER = """Union[str, PyQt6.QtCore.QDir, Path]"""
name: str
def __init__(self, name: str | type):
# import only here since we change the path to find them
from PyQt6 import QtCore, QtGui, QtWidgets
if isinstance(name, type):
name = name.__name__
self.name = name.strip()
# replace QFlags[xxx] with xxx
self.name = re.sub(r"QFlags\[([^]]*)\]", r"\1", self.name)
# find PyQt types
for m in (QtCore, QtGui, QtWidgets):
if self.name in dir(m):
self.name = "{}.{}".format(m.__name__, self.name)
def typing(self) -> str:
"""
Returns:
A valid typing representation for this type.
"""
# IPluginBase -> IPlugin
if self.name == "mobase.IPluginBase":
return "IPlugin"
return self.name
def is_none(self) -> bool:
"""
Check if this type represent None.
Returns:
True if this type represents None.
"""
return self.name.lower() in ("none", "nonetype")
def is_object(self) -> bool:
"""
Check if this type represent the generic "object" type.
Returns:
True if this type represent the generic object type.
"""
return self.name.lower() == "object"
def is_any(self) -> bool:
"""
Check if this type represent the typing "Any".
Returns:
True if this type represent the typing "Any".
"""
return self.name == "Any"
def __str__(self):
return "Type({})".format(self.name)
def __repr__(self):
return str(self)
def __hash__(self):
return hash(self.name)
def __eq__(self, other: object) -> bool:
if not isinstance(other, PyType):
return NotImplemented
return self.name == other.name
class Return:
"""
Class representing the return value of a function (type and documentation).
"""
type: PyType
doc: str
def __init__(self, type: PyType, doc: str = ""):
self.type = type
self.doc = doc
class Argument:
"""
Class representing a function argument (type and eventual default value).
"""
# Constant representing None since None indicates no default value:
DEFAULT_NONE = "None"
name: str
type: PyType
_value: str | None
doc: str
def __init__(
self, name: str, type: PyType, value: str | None = None, doc: str = ""
):
self.name = name
self.type = type
self._value = value
self.doc = doc
@property
def value(self) -> str | None:
value = self._value
if value is None:
return None
# pybind11 puts enum in <> so we need to fix
m = re.match(r"<([^:]+):\s*[0-9]+>", value)
if m:
# if this is a mobase enum, we eed to use the upper case version
if self.type.name.startswith("mobase"):
value = m.group(1)
parts = value.split(".")
value = ".".join(parts[:-1] + [parts[-1].upper()])
# PyQt -> need to fix
elif self.type.name.startswith("PyQt"):
parts = m.group(1).split(".")
value = f"{self.type.name}.{parts[-1]}"
else:
value = m.group(1)
return value
def has_default_value(self) -> bool:
return self.value is not None
def __str__(self):
if self.has_default_value():
return "Arg({}={})".format(self.type, self.value)
return "Arg({})".format(self.type)
def __repr__(self):
return str(self)
def __hash__(self):
return hash(self.type)
def __eq__(self, other: object) -> bool:
if not isinstance(other, Argument):
return NotImplemented
return self.type == other.type
class Exception:
"""
Small class representing exception that can be raised from functions.
"""
type: PyType
doc: str
def __init__(self, type: PyType, doc: str = ""):
self.type = type
self.doc = doc
class Function:
"""
Class representing a function.
"""
name: str
ret: Return
args: list[Argument]
overloads: bool
raises: list[Exception]
doc: str
deprecated: bool
def __init__(
self,
name: str,
ret: Return,
args: list[Argument],
has_overloads: bool = False,
doc: str = "",
):
self.name = name
self.ret = ret
self.args = args
self.overloads = has_overloads
self.raises = []
self.doc = ""
self.deprecated = False
def has_overloads(self):
return self.overloads
def is_deprecated(self):
return self.deprecated
class Method(Function):
"""
Class representing a method.
"""
cls: Class
abstract: str | bool
static: bool
def __init__(
self,
name: str,
ret: Return,
args: list[Argument],
static: bool,
has_overloads: bool = False,
doc: str = "",
):
super().__init__(name, ret, args, has_overloads, doc)
self.static = static
self.abstract = "auto"
def is_abstract(self):
if self.name.startswith("__"):
return False
if self.abstract == "auto":
return self.cls.is_abstract()
return self.abstract
def is_static(self):
return self.static
def is_special(self):
return self.name.startswith("__")
def is_constructor(self):
return self.name == "__init__"
class Constant:
"""
Class representing a constant.
"""
name: str
type: PyType | None
value: object
doc: str | None
def __init__(
self, name: str, type: PyType | None, value: object, doc: str | None = None
):
self.name = name
self.type = type
# Note: The value is not used actually since we can hide it using `...`.
self.value = value
self.doc = doc
class Property:
"""
Class representing a property.
"""
name: str
type: PyType
doc: str
read_only: bool
def __init__(self, name: str, type: PyType, read_only: bool, doc: str = ""):
self.name = name
self.type = type
self.read_only = read_only
self.doc = doc
def is_read_only(self):
return self.read_only
class Class:
"""
Class representing a class.
"""
name: str
bases: list[Class]
methods: list[Method]
constants: list[Constant]
properties: list[Property]
inner_classes: list[Class]
outer_class: Class | None
doc: str
abstract: bool
deprecated: bool
def __init__(
self,
package: str,
name: str,
bases: list[Class],
methods: list[Method],
constants: list[Constant] = [],
properties: list[Property] = [],
inner_classes: list[Class] = [],
doc: str = "",
):
self.package = package
self.name = name
self.bases = bases
self.methods = methods
self.properties = properties
self.constants = constants
self.inner_classes = inner_classes
self.doc = ""
self.abstract = False
self.outer_class = None
self.deprecated = False
# Update class in method:
for m in self.methods:
m.cls = self
for ic in self.inner_classes:
ic.outer_class = self
def is_abstract(self):
"""
Returns:
True if this class is abstract, False otherwise.
"""
return self.abstract or any(bc.is_abstract() for bc in self.bases)
@property
def canonical_name(self):
"""
Returns:
The canonical name of this class.
"""
name = self.name
oc = self.outer_class
while oc is not None:
name = "{}.{}".format(oc.name, name)
oc = oc.outer_class
return name
@property
def full_name(self):
"""
Returns:
The full name of this class, i.e., package.canonical_name.
"""
if self.package:
return f"{self.package}.{self.canonical_name}"
return self.canonical_name
@property
def all_bases(self) -> set[Class]:
"""
Returns:
All the bases of this class, including bases of bases and so on.
"""
bases = set(self.bases)
for b in self.bases:
bases = bases.union(b.all_bases)
return bases
def is_deprecated(self):
return self.deprecated
class PyClass(Class):
"""
Class use to wrap Python class to be used as parent class for some classes
in mobase.
"""
def __init__(
self,
package: str,
name: str,
):
super().__init__(package, name, [], [])
self.abstract = False
class Enum(Class):
"""
Class representing an enum.
"""
def __init__(
self, package: str, name: str, values: dict[str, int], methods: list[Method]
):
# Note: Boost.Python.enum inherits int() not enum.Enum() but for the sake
# of stubs, I think making them inherit enum.Enum is more appropriate:
super().__init__(
package,
name,
[PyClass("", "Enum")],
methods,
inner_classes=[],
constants=[Constant(k, None, v) for k, v in values.items()],
)
def is_abstract(self):
return False
+379
View File
@@ -0,0 +1,379 @@
# -*- encoding: utf-8 -*-
import inspect
import re
import types
from collections import OrderedDict, defaultdict
from itertools import chain
from typing import Iterable, cast
from . import LOGGER
from .mtypes import (
Argument,
Class,
Constant,
Enum,
Function,
Method,
Property,
PyClass,
PyType,
Return,
)
from .register import MobaseRegister
def magic_split(value: str, sep=",", open="(<[", close=")>]"):
"""
Split the value according to the given separator, but keeps together elements
within the given separator. Useful to split C++ signature function since type names
can contain special characters...
Examples:
- magic_split("a,b,c", sep=",") -> ["a", "b", "c"]
- magic_split("a<b,c>,d(e,<k,c>),p) -> ["a<b,c>", "d(e,<k,c>)", "p"]
Args:
value: String to split.
sep: Separator to use.
open: List of opening characters.
close: List of closing characters. Order must match open.
Returns: The list of split parts from value.
"""
i, j = 0, 0
s: list[str] = []
r = []
while i < len(value):
j = i + 1
while j < len(value):
c = value[j]
# Separator found and the stack is empty:
if c == sep and not s:
break
# Check close/open:
if c in open:
s.append(open.index(c))
elif c in close:
# The stack might be empty if the separator is also an opening element:
if not s and sep in open and j + 1 == len(value):
pass
else:
t = s.pop()
if t != close.index(c):
raise ValueError(
"Found closing element {} for opening element {}.".format(
c, open[t]
)
)
j += 1
r.append(value[i:j])
i = j + 1
assert not s
return r
def parse_python_signature(s: str, name: str) -> tuple[PyType, list[Argument]]:
"""
Parse a pybind11 python signature.
Args:
s: The signature to parse.
name: Name of the function.
Returns: (RType, Args) where RType is a Type object, and Args is a list of Arg
objects containing Type.
"""
m = re.search(rf"{name}\((.*)\)\s*->\s*([^:]+)\s*", s)
if not m:
raise ValueError(f"invalid signature: {s}")
args = magic_split(m.group(1).strip(), ",", open="[", close="]")
return_type = m.group(2)
arguments = []
for i, pa in enumerate(args):
m = re.search(
r"(?P<name>[^:]+)\s*:\s*(?P<type>[^=]+)\s*(=\s*(?P<value>[^,]+))?",
pa.strip(),
)
if not m:
raise ValueError(f"invalid argument: {pa}, {s}")
matches = m.groupdict()
arguments.append(
Argument(matches["name"], PyType(matches["type"]), matches["value"])
)
return PyType(return_type), arguments
def is_enum(e: type) -> bool:
"""Check if the given class is an enumeration.
Args:
e: The class object to check.
Returns: True if the object is an enumeration (boost::python enumeration, not
python) False otherwise.
"""
# Yet to find a better way...
if not isinstance(e, type):
return False
return hasattr(e, "__entries")
class Overload:
"""Small class to avoid mypy issues..."""
return_type: PyType
arguments: list[Argument]
def __init__(self, return_type: PyType, arguments: list[Argument]):
self.return_type = return_type
self.arguments = arguments
def parse_pybind11_function_docstring(e) -> list[Overload]:
"""
Parse the docstring of the given element.
Args:
e: The function to "parse".
Returns:
A list of overloads for the given function.
"""
lines = e.__doc__.strip().split("\n")
signatures: list[str]
if len(lines) == 1:
signatures = lines
else:
signatures = []
for line in lines:
m = re.match(rf"^[0-9]+[.]\s+({e.__name__}.*)$", line)
if m:
signatures.append(m.group(1).strip())
# We are going to parse the python and C++ signature, and try to merge
# them...
overloads: list[Overload] = []
for signature in signatures:
# fix MOBase:: in some places to get proper Python types
signature = signature.replace("MOBase::", "mobase.").replace("::", ".")
try:
return_type, arguments = parse_python_signature(signature, e.__name__)
except ValueError:
raise ValueError(f"invalid signature: {e.__name__}, {e.__doc__}")
overloads.append(Overload(return_type=return_type, arguments=arguments))
return overloads
def make_functions(e) -> list[Function]:
overloads = parse_pybind11_function_docstring(e)
return [
Function(
e.__name__,
Return(overload.return_type),
overload.arguments,
has_overloads=len(overloads) > 1,
)
for overload in overloads
]
def make_class(e: type, register: MobaseRegister) -> Class:
"""
Constructs a Class object from the given python class.
Args:
e: The python class (created from boost) to construct an object for.
class_register:
Returns: A Class object corresponding to the given class.
"""
base_classes_s: list[str] = []
# Kind of ugly, but...:
for c in inspect.getmro(e):
if c != e and c.__module__ == "mobase":
base_classes_s.append(c.__name__)
if c.__module__ == "pybind11_builtins":
break
first_base = inspect.getmro(e)[1]
# This contains ALL the parent classes, not the direct ones:
base_classes: list[Class] = [
register.make_object(name) for name in base_classes_s # type: ignore
]
# retrieve all the attributes that are not in a base class
all_attrs = [
(n, getattr(e, n))
for n in dir(e)
if not hasattr(first_base, n) or getattr(first_base, n) is not getattr(e, n)
]
# members to exclude
EXCLUDED_MEMBERS = [
"__init_subclass__",
"__module__",
"__subclasshook__",
"__hash__",
"__getstate__",
"__setstate__",
"__index__",
"__repr__",
]
all_attrs = [a for a in all_attrs if a[0] not in EXCLUDED_MEMBERS]
# fetch all attributes from the base classes
base_attrs: dict[str, list[Constant | Property | Method | Class]] = defaultdict(
list
)
for bc in base_classes:
for a in cast(
Iterable[Constant | Property | Method | Class],
chain(bc.constants, bc.methods, bc.properties, bc.inner_classes),
):
base_attrs[a.name].append(a)
# retrieve the enumerations and classes
inner_classes = [ic[1] for ic in all_attrs if isinstance(ic[1], type)]
pinner_classes: list[Class] = [
cast(Class, register.make_object(f"{e.__qualname__}.{ic.__name__}", ic))
for ic in inner_classes
]
# find the methods
raw_methods = [
m[1] for m in all_attrs if callable(m[1]) and m[1] not in inner_classes
]
raw_methods = sorted(raw_methods, key=lambda m: str(m.__name__))
raw_methods = [m for m in raw_methods if m.__doc__ is not None]
# remove __init__
raw_methods = [
m for m in raw_methods if not isinstance(m, types.WrapperDescriptorType)
]
methods: list[Method] = []
for method in raw_methods:
if method.__doc__ is None:
continue
# __eq__ must accept an object in python (and it does with pybind11), so we
# force the overload
if method.__name__ in ["__eq__", "__ne__"]:
overloads = [
Overload(
return_type=PyType("bool"),
arguments=[
Argument("self", PyType(e.__module__ + "." + e.__qualname__)),
Argument("other", PyType("object")),
],
)
]
# otherwise we parse the docstring
else:
overloads = parse_pybind11_function_docstring(method)
for overload in overloads:
args = overload.arguments
# pybind11 seems to be consistent with the naming of "self", so we can
# mostly rely on it
static = len(args) == 0 or args[0].name != "self"
# we need to fix some default values (basically default values that
# comes from inner enum) and argument
for arg in overload.arguments:
if arg.has_default_value():
value: str = arg.value # type: ignore
base_name = value.split(".")[0]
for base_class in base_classes:
for biclass in base_class.inner_classes:
if isinstance(biclass, Enum) and biclass.name == base_name:
arg._value = base_class.name + "." + value
methods.append(
Method(
method.__name__,
Return(overload.return_type),
overload.arguments,
static=static,
has_overloads=len(overloads) > 1,
)
)
# Retrieve the attributes:
constants = []
properties = []
for name, attr in all_attrs:
if callable(attr) or isinstance(attr, type):
continue
# Maybe we should check an override here (e.g., different value for a constant):
if name in base_attrs:
continue
if isinstance(attr, property):
properties.append(Property(name, PyType("Any"), attr.fset is None))
elif not hasattr(attr, "__name__"):
constants.append(Constant(name, PyType(type(attr).__name__), attr))
direct_bases: list[Class] = []
for c in e.__bases__:
if c.__module__ != "pybind11_builtins":
direct_bases.append(register.get_object(c.__name__))
# Forcing QWidget base for XWidget classes since these do not show up
# and we use a trick:
if e.__name__.endswith("Widget"):
LOGGER.info(
"Forcing base {} for class {}.".format(
"PyQt6.QtWidgets.QWidget", e.__name__
)
)
direct_bases.append(PyClass("PyQt6.QtWidgets", "QWidget"))
# check if it an enum
if is_enum(e):
# all pybind11 enums have a .__entries attribute
values = e.__entries # type: ignore
# drop the __init__
methods = [m for m in methods if m.name != "__init__"]
return Enum(
e.__module__,
e.__name__,
OrderedDict((name, value) for name, (value, _) in values.items()),
methods=methods,
)
return Class(
e.__module__,
e.__name__,
direct_bases,
methods,
inner_classes=pinner_classes,
properties=properties,
constants=constants,
)
+62
View File
@@ -0,0 +1,62 @@
# -*- encoding: utf-8 -*-
from __future__ import annotations
from collections import OrderedDict
from .mtypes import Class, Function
class MobaseRegister:
"""
Class that register classes.
"""
objects: dict[str, Class | list[Function]]
def __init__(self):
self.raw_objects: dict[str, type] = OrderedDict()
self.objects = {}
def add_object(self, name, object):
self.raw_objects[name] = object
def make_object(self, name: str, e: type | None = None) -> Class | list[Function]:
"""
Construct a Function, Class or Enum for the given object.
Args:
name: The name of the object to inspect.
e: The object to inspect, or None to fetch it from the underlying list.
Returns:
A Class object for the given type, or a list of function overloads.
"""
from .parser import make_class, make_functions
if e is None:
e = self.raw_objects[name]
if name not in self.raw_objects:
self.raw_objects[name] = e
if name not in self.objects:
if isinstance(e, type):
self.objects[name] = make_class(e, self)
elif callable(e):
self.objects[name] = make_functions(e)
return self.objects[name]
def get_object(self, name: str):
"""
Retrieve the object if the given name. Fails if no object with this
name exists (if `make_object(name, ...)` has never been called).
Args:
name: Name of the object to retrieve.
Returns:
The object with the given name.
"""
return self.objects[name]
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More