Files
Paul Guyot e554d9ea6c sentencepiece: update to 0.2.1
Update both the C++ library (textproc/sentencepiece) and the Python
wrapper (python/py-sentencepiece) together as they are closely coupled.
Also add py-sentencepiece subports for 3.13 and 3.14.

Signed-off-by: Paul Guyot <pguyot@kallisys.net>
2026-03-21 06:49:53 -04:00

62 lines
2.1 KiB
Tcl

# -*- coding: utf-8; mode: tcl; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- vim:fenc=utf-8:ft=tcl:et:sw=4:ts=4:sts=4
PortSystem 1.0
PortGroup github 1.0
PortGroup python 1.0
github.setup google sentencepiece 0.2.1 v
revision 0
checksums rmd160 98e5b47ff08d9b0c83f78b7b4bc38ad399343058 \
sha256 c1a59e9259c9653ad0ade653dadff074cd31f0a6ff2a11316f67bee4189a8f1b \
size 13485527
name py-${github.project}
categories-append textproc
license Apache-2
maintainers nomaintainer
description Python wrapper for ${github.project}
long_description SentencePiece is an unsupervised text tokenizer \
and detokenizer mainly for Neural Network-based \
text generation systems where the vocabulary size \
is predetermined prior to the neural model \
training. SentencePiece implements subword units \
(e.g., byte-pair-encoding (BPE) \[Sennrich et al.\]) \
and unigram language model \[Kudo.\]) with the \
extension of direct training from raw \
sentences. SentencePiece allows us to make a \
purely end-to-end system that does not depend on \
language-specific pre/postprocessing.
github.tarball_from archive
dist_subdir ${github.project}
python.versions 310 311 312 313 314
if {${name} ne ${subport}} {
# The source code deliberately breaks macOS builds by hardcoding
# the flag. Fix this.
patchfiles-append \
patch-fix-macos-flag.diff
post-patch {
reinplace "s|@MAC_OS_V@|${macosx_deployment_target}|" ${worksrcpath}/setup.py
}
depends_build-append \
port:pkgconfig \
port:py${python.version}-setuptools
depends_lib-append \
port:protobuf3-cpp \
port:sentencepiece
worksrcdir ${distname}/python
compiler.cxx_standard 2017
test.run yes
livecheck.type none
}