mirror of
https://github.com/m5stack/StackFlow.git
synced 2026-05-20 11:32:11 -07:00
[init] StackFlow
This commit is contained in:
dianjixz
@@ -0,0 +1,91 @@
|
||||
# Module-LLM
|
||||
|
||||
<div class="product_pic"><img class="pic" src="https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/4.webp"><img class="pic" src="https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/10.webp"><img class="pic" src="https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/15.webp"></div>
|
||||
|
||||
## Description
|
||||
|
||||
**Module LLM** is an integrated offline Large Language Model (LLM) inference module designed for terminal devices that require efficient and intelligent interaction. Whether for smart homes, voice assistants, or industrial control, Module LLM provides a smooth and natural AI experience without relying on the cloud, ensuring privacy and stability. Integrated with the **StackFlow** framework and **Arduino/UiFlow** libraries, smart features can be easily implemented with just a few lines of code.<br>
|
||||
Powered by the advanced **AX630C** SoC processor, it integrates a 3.2 TOPs high-efficiency NPU with native support for Transformer models, handling complex AI tasks with ease. Equipped with **4GB LPDDR4** memory (1GB available for user applications, 3GB dedicated to hardware acceleration) and **32GB eMMC** storage, it supports parallel loading and sequential inference of multiple models, ensuring smooth multitasking. The main chip's runtime power consumption of approximately 1.5W, making it highly efficient and suitable for long-term operation.<br>
|
||||
It features a built-in microphone, speaker, TF storage card, **USB OTG**, and RGB status light, meeting diverse application needs with support for voice interaction and data transfer. The module offers flexible expansion: the onboard SD card slot supports cold/hot firmware upgrades, and the **UART** communication interface simplifies connection and debugging, ensuring continuous optimization and expansion of module functionality. The USB port supports master-slave auto-switching, serving as both a debugging port and allowing connection to additional USB devices like cameras. Users can purchase the LLM debugging kit to add a 100 Mbps Ethernet port and kernel serial port, using it as an SBC.<br>
|
||||
The module is compatible with multiple models and comes pre-installed with the **Qwen2.5-0.5B** language model. It features **KWS** (wake word), **ASR** (speech recognition), **LLM** (large language model), and **TTS** (text-to-speech) functionalities, with support for standalone calls or **pipeline** automatic transfer for convenient development. Future support includes Qwen2.5-1.5B, Llama3.2-1B, and InternVL2-1B models, allowing hot model updates to keep up with community trends and accommodate various complex AI tasks. Vision recognition capabilities include support for CLIP, YoloWorld, and future updates for DepthAnything, SegmentAnything, and other advanced models to enhance intelligent recognition and analysis.<br>
|
||||
Plug and play with **M5 hosts**, Module LLM offers an easy-to-use AI interaction experience. Users can quickly integrate it into existing smart devices without complex settings, enabling smart functionality and improving device intelligence. This product is suitable for offline voice assistants, text-to-speech conversion, smart home control, interactive robots, and more.
|
||||
|
||||
|
||||
|
||||
## Product Features
|
||||
|
||||
- Offline inference, 3.2T@INT8 precision computing power
|
||||
- Integrated KWS (wake word), ASR (speech recognition), LLM (large language model), TTS (text-to-speech generation)
|
||||
- Multi-model parallel processing
|
||||
- Onboard 32GB eMMC storage and 4GB LPDDR4 memory
|
||||
- Onboard microphone and speaker
|
||||
- Serial communication
|
||||
- SD card firmware upgrade
|
||||
- Supports ADB debugging
|
||||
- RGB indicator light
|
||||
- Built-in Ubuntu system
|
||||
- Supports OTG functionality
|
||||
- Compatible with Arduino/UIFlow
|
||||
|
||||
>
|
||||
|
||||
## Applications
|
||||
|
||||
- Offline voice assistants
|
||||
- Text-to-speech conversion
|
||||
- Smart home control
|
||||
- Interactive robots
|
||||
|
||||
## Specifications
|
||||
|
||||
| Specification | Parameter |
|
||||
| ---------------- | ------------------------------------------------------------------------------------------- |
|
||||
| Processor SoC | AX630C@Dual Cortex A53 1.2 GHz <br> MAX.12.8 TOPS @INT4 and 3.2 TOPS @INT8 |
|
||||
| Memory | 4GB LPDDR4 (1GB system memory + 3GB dedicated for hardware acceleration) |
|
||||
| Storage | 32GB eMMC5.1 |
|
||||
| Communication | Serial communication default baud rate 115200@8N1 (adjustable) |
|
||||
| Microphone | MSM421A |
|
||||
| Audio Driver | AW8737 |
|
||||
| Speaker | 8Ω@1W, Size:2014 cavity speaker |
|
||||
| Built-in Units | KWS (wake word), ASR (speech recognition), LLM (large language model), TTS (text-to-speech) |
|
||||
| RGB Light | 3x RGB LED@2020 driven by LP5562 (status indication) |
|
||||
| Power | Idle: 5V@0.5W, Full load: 5V@1.5W |
|
||||
| Button | For entering download mode for firmware upgrade |
|
||||
| Upgrade Port | SD card / Type-C port |
|
||||
| Working Temp | 0-40°C |
|
||||
| Product Size | 54*54*13mm |
|
||||
| Packaging Size | 133*95*16mm |
|
||||
| Product Weight | 17.4g |
|
||||
| Packaging Weight | 32.0g |
|
||||
|
||||
<div class="product_pic"><img class="pic" src="https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/11.webp"><img class="pic" src="https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/5.webp"><img class="pic" src="https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/12.webp"></div>
|
||||
|
||||
<div class="product_pic"><img class="pic" src="https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/9.webp"><img class="pic" src="https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/14.webp"><img class="pic" src="https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/7.webp"></div>
|
||||
|
||||
## Related Links
|
||||
|
||||
- [AX630C](https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/AX630C.pdf)
|
||||
|
||||
## PinMap
|
||||
|
||||
| Module LLM | RXD | TXD |
|
||||
| ------------ | --- | --- |
|
||||
| Core (Basic) | G16 | G17 |
|
||||
| Core2 | G13 | G14 |
|
||||
| CoreS3 | G18 | G17 |
|
||||
|
||||
>LLM Module Pin Switching| LLM Module has reserved soldering pads for pin switching. In cases of pin multiplexing conflicts, the PCB trace can be cut and reconnected to other sets of pins.
|
||||
|
||||
<img alt="module size" src="https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/03.jpg" width="100%" />
|
||||
|
||||
> Taking `CoreS3` as an example, the first column (left green box) is the TX pin for serial communication, where users can choose one out of four options as needed (from top to bottom, the pins are G18, G7, G14, and G10). The default is set to IO18. To switch to a different pin, cut the connection on the solder pad (at the red line) — it’s recommended to use a blade for this — and then connect to one of the three remaining pins below. The second column (right green box) is for RX pin selection, and, as with the TX pin, it also allows a choice of one out of four options.
|
||||
|
||||
|
||||
|
||||
## Video
|
||||
|
||||
- Module LLM product introduction and example showcase [Module_LLM_Video.mp4](https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/Module_LLM_Video.mp4)
|
||||
|
||||
## AI Benchmark Comparison
|
||||
|
||||
<img alt="compare" src="https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/Benchmark%E5%AF%B9%E6%AF%94.png" width="100%" />
|
||||
+105
@@ -0,0 +1,105 @@
|
||||
# Module-LLM
|
||||
|
||||
<div class="product_pic"><img class="pic" src="https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/4.webp"><img class="pic" src="https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/10.webp"><img class="pic" src="https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/15.webp"></div>
|
||||
|
||||
## 描述
|
||||
|
||||
**Module LLM**是一款集成化的离线大语言模型 (LLM) 推理模块，专为需要高效，智能交互的终端设备设计，无论是在智能家居，语音助手，还是在工业控制中，Module LLM 都能为您带来流畅，自然的 AI 体验，无需依赖云端，确保隐私安全与稳定性。集成 **StackFlow** 框架，配合 **Arduino/UiFlow** 库，几行代码就可轻松实现端侧智能。<br>
|
||||
搭载爱芯 **AX630C** SoC 先进处理器，集成 3.2 TOPs 高能效 NPU，原生支持 Transformer 模型，轻松应对复杂 AI 任务。配备 **4GB LPDDR4** 内存（其中1GB供用户使用，3GB专用于硬件加速）和**32GB eMMC**存储，支持多模型并行加载与串联推理，确保多任务处理流畅无阻。运行功耗仅约 1.5W，远低于同类产品，节能高效，适合长时间运行。<br>
|
||||
集成麦克风，扬声器，TF存储卡，**USB OTG** 及 RGB状态灯，满足多样化应用需求，轻松实现语音交互与数据传输。灵活扩展：板载 SD 卡槽支持固件冷/热升级，**UART** 通信接口简化连接与调试，确保模块功能持续优化与扩展。USB 口支持主从自动切换，既可以做调试口，也可以外接更多 USB 设备如摄像头。搭配 LLM 调试套件，用于扩展百兆以太网口，及内核串口，作为 SBC 使用。<br>
|
||||
多模型兼容，出厂预装 **Qwen2.5-0.5B** 大语言模型，内置**KWS**（唤醒词），**ASR**（语音识别），**LLM**（大语言模型）及**TTS**（文本转语音）功能，且支持分开调用或 **pipeline** 自动流转，方便开发。后续将支持Qwen2.5-1.5B、Llama3.2-1B及InternVL2-1B等多种端侧LLM/VLM模型，支持热更新模型，紧跟社区潮流，适应不同复杂度的AI任务；视觉识别能力：支持CLIP、YoloWorld等Open world模型，未来将持续更新DepthAnything、SegmentAnything等先进模型，赋能智能识别与分析；<br>
|
||||
即插即用，搭配**M5主机**，Module LLM 实现即插即用的AI交互体验。用户无需繁琐设置，即可将其集成到现有智能设备中，快速启用智能化功能，提升设备智能水平。该产品适用于离线语音助手，文本语音转换，智能家居控制，互动机器人等领域。
|
||||
|
||||
|
||||
|
||||
|
||||
## 产品特性
|
||||
|
||||
- 离线推理,3.2T@INT8精度算力
|
||||
|
||||
- 集成KWS（唤醒词），ASR（语音识别），LLM（大语言模型），TTS（文本生成语音）
|
||||
|
||||
- 多模型并行
|
||||
|
||||
- 板载32GB eMMC存储和4GB LPDDR4内存
|
||||
|
||||
- 板载麦克风及扬声器
|
||||
|
||||
- 串口通信
|
||||
|
||||
- SD卡固件升级
|
||||
|
||||
- 支持ADB调试
|
||||
|
||||
- RGB提示灯
|
||||
|
||||
- 内置ubuntu系统
|
||||
|
||||
- 支持OTG功能
|
||||
|
||||
- 支持Arduino/UIFlow
|
||||
|
||||
|
||||
|
||||
|
||||
## 应用
|
||||
|
||||
- 离线语音助手
|
||||
- 文本语音转换
|
||||
- 智能家居控制
|
||||
- 互动机器人
|
||||
|
||||
## 规格参数
|
||||
|
||||
| 规格 | 参数 |
|
||||
| ------------ | --------------------------------------------------------------------------- |
|
||||
| 处理器SoC | AX630C@Dual Cortex A53 1.2 GHz <br> MAX.12.8 TOPS @INT4,and 3.2 TOPS @INT8 |
|
||||
| 内存 | 4GB LPDDR4(1GB系统内存 + 3GB 硬件加速专用内存) |
|
||||
| 存储 | 32GB eMMC5.1 |
|
||||
| 通信接口 | 串口通信 波特率默认 115200@8N1（可调） |
|
||||
| 麦克风 | MSM421A |
|
||||
| 音频驱动 | AW8737 |
|
||||
| 扬声器 | 8Ω@1W,尺寸:2014 腔体喇叭 |
|
||||
| 内置功能单元 | KWS（唤醒词），ASR（语音识别），LLM（大语言模型），TTS（文本生成语音） |
|
||||
| RGB灯 | 3x RGB LED@2020 由LP5562驱动 (状态指示) |
|
||||
| 功耗 | 空载：5V@0.5W，满载:5V@1.5W |
|
||||
| 按键 | 用于升级固件进入下载模式 |
|
||||
| 升级接口 | SD卡/Type C口 |
|
||||
| 工作温度 | 0-40°C |
|
||||
| 产品尺寸 | 54*54*13mm |
|
||||
| 包装尺寸 | 133*95*16mm |
|
||||
| 产品重量 | 17.4g |
|
||||
| 包装重量 | 32.0g |
|
||||
|
||||
<div class="product_pic"><img class="pic" src="https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/11.webp"><img class="pic" src="https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/5.webp"><img class="pic" src="https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/12.webp"></div>
|
||||
|
||||
<div class="product_pic"><img class="pic" src="https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/9.webp"><img class="pic" src="https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/14.webp"><img class="pic" src="https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/7.webp"></div>
|
||||
|
||||
|
||||
## 相关链接
|
||||
|
||||
- [AX630C](https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/AX630C.pdf)
|
||||
|
||||
## PinMap
|
||||
|
||||
| Module LLM | RXD | TXD |
|
||||
| ------------ | --- | --- |
|
||||
| Core (Basic) | G16 | G17 |
|
||||
| Core2 | G13 | G14 |
|
||||
| CoreS3 | G18 | G17 |
|
||||
|
||||
>LLM Module引脚切换| LLM Module预留了引脚切换焊盘, 一些出现引脚复用冲突的情况下, 可以通过切割PCB线路然后跳线连接至其他组引脚。
|
||||
|
||||
<img alt="module size" src="https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/03.jpg" width="100%" />
|
||||
|
||||
> 以`CoreS3`为例子，第一列（左绿色框）是串口通讯的TX引脚，用户根据需要四选一，（从上到下分别代表引脚G18 G7 G14 G10）,默认是IO18引脚，如需要切换其他引脚，切断焊盘连接线（红线处）（建议使用刀片切割），然后连接下面三个引脚之间的一个即可；第二列（右绿色框）是RX引脚切换，如上所述，也是四选一操作
|
||||
|
||||
|
||||
|
||||
## 视频
|
||||
|
||||
- Module LLM 产品介绍以及例子展示 [Module_LLM_Video.mp4](https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/Module_LLM_Video.mp4)
|
||||
|
||||
## AI Benchmark 对比
|
||||
|
||||
<img alt="compare" src="https://m5stack.oss-cn-shenzhen.aliyuncs.com/resource/docs/products/module/Module%20LLM/Benchmark%E5%AF%B9%E6%AF%94.png" width="100%" />
|
||||
Submodule
+1
Submodule SDK added at c767769d28
@@ -0,0 +1,418 @@
|
||||
# StackFlow
|
||||
StackFlow is a communication framework developed by M5STACK for AI accelerated computing, mainly running on embedded Linux platforms, with [zmq](https://zeromq.org/) providing the underlying communication services.
|
||||
StackFlow primarily offers three functions: first, remote RPC calls for function invocation between units; second, message communication, providing standard message stream services for better context integration; and third, resource allocation, mainly to avoid communication address conflicts between related units and temporary data storage.
|
||||
|
||||
|
||||
## pzmq
|
||||
The zmq API is repackaged to make the invocation of ZMQ_PUB, ZMQ_SUB, ZMQ_PUSH, and ZMQ_PULL simpler and more convenient, using asynchronous callback methods to provide message reception functionality.
|
||||
On the basis of ZMQ_REP and ZMQ_REQ, simple RPC functionality is encapsulated to provide RPC services.
|
||||
|
||||
### Related Examples
|
||||
1. ZMQ_PULL
|
||||
```c++
|
||||
/*
|
||||
* SPDX-FileCopyrightText: 2024 M5Stack Technology CO LTD
|
||||
*
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
#include <iostream>
|
||||
#include "pzmq.hpp"
|
||||
#include <string>
|
||||
using namespace StackFlows;
|
||||
void pull_msg(const std::string &raw_msg){
|
||||
std::cout << raw_msg << std::endl;
|
||||
}
|
||||
int main(int argc, char *argv[]) {
|
||||
pzmq zpull_("ipc:///tmp.5000.socket", ZMQ_PULL, pull_msg);
|
||||
while(1) {
|
||||
sleep(1);
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
2. ZMQ_PUSH
|
||||
```c++
|
||||
/*
|
||||
* SPDX-FileCopyrightText: 2024 M5Stack Technology CO LTD
|
||||
*
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
#include <iostream>
|
||||
#include "pzmq.hpp"
|
||||
#include <string>
|
||||
using namespace StackFlows;
|
||||
int main(int argc, char *argv[]) {
|
||||
pzmq zpush_("ipc:///tmp.5000.socket", ZMQ_PUSH);
|
||||
zpush_.send_data("nihao");
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
3. ZMQ_PUB
|
||||
```c++
|
||||
/*
|
||||
* SPDX-FileCopyrightText: 2024 M5Stack Technology CO LTD
|
||||
*
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
#include <iostream>
|
||||
#include "pzmq.hpp"
|
||||
#include <string>
|
||||
using namespace StackFlows;
|
||||
int main(int argc, char *argv[]) {
|
||||
pzmq zpush_("ipc:///tmp.5001.socket", ZMQ_PUB);
|
||||
zpush_.send_data("nihao");
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
4. ZMQ_SUB
|
||||
```c++
|
||||
/*
|
||||
* SPDX-FileCopyrightText: 2024 M5Stack Technology CO LTD
|
||||
*
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
#include <iostream>
|
||||
#include "pzmq.hpp"
|
||||
#include <string>
|
||||
using namespace StackFlows;
|
||||
void sub_msg(const std::string &raw_msg){
|
||||
std::cout << raw_msg << std::endl;
|
||||
}
|
||||
int main(int argc, char *argv[]) {
|
||||
pzmq zpull_("ipc:///tmp.5001.socket", ZMQ_SUB, sub_msg);
|
||||
while(1) {
|
||||
sleep(1);
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
4. ZMQ_RPC_FUN
|
||||
```c++
|
||||
/*
|
||||
* SPDX-FileCopyrightText: 2024 M5Stack Technology CO LTD
|
||||
*
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
#include <iostream>
|
||||
#include "pzmq.hpp"
|
||||
#include <string>
|
||||
using namespace StackFlows;
|
||||
std::string fun1_(const std::string &raw_msg){
|
||||
std::cout << raw_msg << std::endl;
|
||||
return std::string("nihao");
|
||||
}
|
||||
std::string fun2_(const std::string &raw_msg){
|
||||
std::cout << raw_msg << std::endl;
|
||||
return std::string("hello");
|
||||
}
|
||||
int main(int argc, char *argv[]) {
|
||||
pzmq _rpc("test");
|
||||
_rpc.register_rpc_action("fun1", fun1_);
|
||||
_rpc.register_rpc_action("fun2", fun2_);
|
||||
while(1) {
|
||||
sleep(1);
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
5. ZMQ_RPC_CALL
|
||||
```c++
|
||||
/*
|
||||
* SPDX-FileCopyrightText: 2024 M5Stack Technology CO LTD
|
||||
*
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
#include <iostream>
|
||||
#include "pzmq.hpp"
|
||||
#include <string>
|
||||
using namespace StackFlows;
|
||||
std::string fun1_(const std::string &raw_msg){
|
||||
return std::string("nihao");
|
||||
}
|
||||
std::string fun2_(const std::string &raw_msg){
|
||||
return std::string("hello");
|
||||
}
|
||||
int main(int argc, char *argv[]) {
|
||||
pzmq _rpc("test");
|
||||
_rpc.call_rpc_action("fun1", "call fun1_", [](const std::string &raw_msg){std::cout << raw_msg << std::endl;});
|
||||
_rpc.call_rpc_action("fun2", "call fun2_", [](const std::string &raw_msg){std::cout << raw_msg << std::endl;});
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
## StackFlow Main Body
|
||||
StackFlow encapsulates pzmq and eventpp, providing basic RPC functions, asynchronous processing, and channel establishment for accelerated units.
|
||||
StackFlow provides seven basic RPC functions for basic function calls of the StackFlow JSON protocol.
|
||||
- setup: Unit configuration function, a function that each unit must implement.
|
||||
- pause: Suspend unit function.
|
||||
- work: Unit start work function.
|
||||
- exit: Unit exit function, a function that each unit must implement.
|
||||
- link: Link to the upstream output function, used to build a message communication chain.
|
||||
- unlink: Unlink from the upstream output function, stop receiving messages from the upstream.
|
||||
- taskinfo: Get unit running information.
|
||||
|
||||
StackFlow provides convenient APIs for unit usage:
|
||||
- unit_call: Unit RPC call function, to call RPC functions of other units.
|
||||
- sys_sql_select: sys unit simple key-value database query function.
|
||||
- sys_sql_set: sys unit simple key-value database setting function.
|
||||
- sys_sql_unset: sys unit simple key-value database deletion function.
|
||||
- repeat_event: Asynchronous periodic execution function.
|
||||
- send: Send user message function.
|
||||
- sys_register_unit: Unit registration function, generally not needed to be called.
|
||||
- sys_release_unit: Unit release function, generally not needed to be called.
|
||||
|
||||
llm_channel_obj encapsulates the communication functions required by the unit, with one configuration corresponding to one llm_channel_obj object.
|
||||
llm_channel_obj provides convenient communication APIs for unit usage:
|
||||
- subscriber_work_id: Subscribe to the pub output of the upstream work_id unit.
|
||||
- stop_subscriber_work_id: Unsubscribe from work_id.
|
||||
- subscriber: Subscribe to the pub output of the zmq URL.
|
||||
- stop_subscriber: Unsubscribe from zmq_url.
|
||||
- send: Send messages of this unit through pub.
|
||||
|
||||
### Basic Usage Example:
|
||||
``` c++
|
||||
/*
|
||||
* SPDX-FileCopyrightText: 2024 M5Stack Technology CO LTD
|
||||
*
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
#include "StackFlow.h"
|
||||
#include <signal.h>
|
||||
#include <sys/stat.h>
|
||||
#include <sys/types.h>
|
||||
#include <unistd.h>
|
||||
#include <fstream>
|
||||
#include <stdexcept>
|
||||
#include <iostream>
|
||||
using namespace StackFlows;
|
||||
int main_exit_flage = 0;
|
||||
static void __sigint(int iSigNo) {
|
||||
main_exit_flage = 1;
|
||||
}
|
||||
typedef std::function<void(const std::string &data, bool finish)> task_callback_t;
|
||||
class llm_task {
|
||||
private:
|
||||
public:
|
||||
std::string model_;
|
||||
std::string response_format_;
|
||||
std::vector<std::string> inputs_;
|
||||
task_callback_t out_callback_;
|
||||
bool enoutput_;
|
||||
bool enstream_;
|
||||
|
||||
void set_output(task_callback_t out_callback) {
|
||||
out_callback_ = out_callback;
|
||||
}
|
||||
|
||||
bool parse_config(const nlohmann::json &config_body) {
|
||||
try {
|
||||
model_ = config_body.at("model");
|
||||
response_format_ = config_body.at("response_format");
|
||||
enoutput_ = config_body.at("enoutput");
|
||||
if (config_body.contains("input")) {
|
||||
if (config_body["input"].is_string()) {
|
||||
inputs_.push_back(config_body["input"].get<std::string>());
|
||||
} else if (config_body["input"].is_array()) {
|
||||
for (auto _in : config_body["input"]) {
|
||||
inputs_.push_back(_in.get<std::string>());
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (...) {
|
||||
return true;
|
||||
}
|
||||
enstream_ = (response_format_.find("stream") != std::string::npos);
|
||||
return false;
|
||||
}
|
||||
|
||||
int load_model(const nlohmann::json &config_body) {
|
||||
if (parse_config(config_body)) {
|
||||
return -1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
void inference(const std::string &msg) {
|
||||
std::cout << msg << std::endl;
|
||||
if (out_callback_) out_callback_(std::string("hello"), true);
|
||||
}
|
||||
|
||||
llm_task(const std::string &workid) {
|
||||
}
|
||||
|
||||
~llm_task() {
|
||||
}
|
||||
};
|
||||
|
||||
class llm_llm : public StackFlow {
|
||||
private:
|
||||
int task_count_;
|
||||
std::unordered_map<int, std::shared_ptr<llm_task>> llm_task_;
|
||||
|
||||
public:
|
||||
llm_llm() : StackFlow("test") {
|
||||
task_count_ = 1;
|
||||
}
|
||||
|
||||
void task_output(const std::shared_ptr<llm_task> llm_task_obj, const std::shared_ptr<llm_channel_obj> llm_channel,
|
||||
const std::string &data, bool finish) {
|
||||
if (llm_channel->enstream_) {
|
||||
static int count = 0;
|
||||
nlohmann::json data_body;
|
||||
data_body["index"] = count++;
|
||||
data_body["delta"] = data;
|
||||
if (!finish)
|
||||
data_body["delta"] = data;
|
||||
else
|
||||
data_body["delta"] = std::string("");
|
||||
data_body["finish"] = finish;
|
||||
if (finish) count = 0;
|
||||
llm_channel->send(llm_task_obj->response_format_, data_body, LLM_NO_ERROR);
|
||||
} else if (finish) {
|
||||
llm_channel->send(llm_task_obj->response_format_, data, LLM_NO_ERROR);
|
||||
}
|
||||
}
|
||||
|
||||
void task_user_data(const std::shared_ptr<llm_task> llm_task_obj,
|
||||
const std::shared_ptr<llm_channel_obj> llm_channel, const std::string &object,
|
||||
const std::string &data) {
|
||||
const std::string *next_data = &data;
|
||||
int ret;
|
||||
std::string tmp_msg1;
|
||||
if (object.find("stream") != std::string::npos) {
|
||||
static std::unordered_map<int, std::string> stream_buff;
|
||||
if (decode_stream(data, tmp_msg1, stream_buff)) return;
|
||||
next_data = &tmp_msg1;
|
||||
}
|
||||
std::string tmp_msg2;
|
||||
if (object.find("base64") != std::string::npos) {
|
||||
ret = decode_base64((*next_data), tmp_msg2);
|
||||
if (!ret) {
|
||||
return;
|
||||
}
|
||||
next_data = &tmp_msg2;
|
||||
}
|
||||
llm_task_obj->inference((*next_data));
|
||||
}
|
||||
|
||||
int setup(const std::string &work_id, const std::string &object, const std::string &data) override {
|
||||
nlohmann::json error_body;
|
||||
if ((llm_task_channel_.size() - 1) == task_count_) {
|
||||
error_body["code"] = -21;
|
||||
error_body["message"] = "task full";
|
||||
send("None", "None", error_body, unit_name_);
|
||||
return -1;
|
||||
}
|
||||
int work_id_num = sample_get_work_id_num(work_id);
|
||||
auto llm_channel = get_channel(work_id);
|
||||
auto llm_task_obj = std::make_shared<llm_task>(work_id);
|
||||
nlohmann::json config_body;
|
||||
try {
|
||||
config_body = nlohmann::json::parse(data);
|
||||
} catch (...) {
|
||||
error_body["code"] = -2;
|
||||
error_body["message"] = "json format error.";
|
||||
send("None", "None", error_body, unit_name_);
|
||||
return -2;
|
||||
}
|
||||
int ret = llm_task_obj->load_model(config_body);
|
||||
if (ret == 0) {
|
||||
llm_channel->set_output(llm_task_obj->enoutput_);
|
||||
llm_channel->set_stream(llm_task_obj->enstream_);
|
||||
llm_task_obj->set_output(std::bind(&llm_llm::task_output, this, llm_task_obj, llm_channel,
|
||||
std::placeholders::_1, std::placeholders::_2));
|
||||
llm_channel->subscriber_work_id("", std::bind(&llm_llm::task_user_data, this, llm_task_obj, llm_channel,
|
||||
std::placeholders::_1, std::placeholders::_2));
|
||||
llm_task_[work_id_num] = llm_task_obj;
|
||||
send("None", "None", LLM_NO_ERROR, work_id);
|
||||
return 0;
|
||||
} else {
|
||||
error_body["code"] = -5;
|
||||
error_body["message"] = "Model loading failed.";
|
||||
send("None", "None", error_body, unit_name_);
|
||||
return -1;
|
||||
}
|
||||
}
|
||||
|
||||
void taskinfo(const std::string &work_id, const std::string &object, const std::string &data) override {
|
||||
nlohmann::json req_body;
|
||||
int work_id_num = sample_get_work_id_num(work_id);
|
||||
if (WORK_ID_NONE == work_id_num) {
|
||||
std::vector<std::string> task_list;
|
||||
std::transform(llm_task_channel_.begin(), llm_task_channel_.end(), std::back_inserter(task_list),
|
||||
[](const auto task_channel) { return task_channel.second->work_id_; });
|
||||
req_body = task_list;
|
||||
send("llm.tasklist", req_body, LLM_NO_ERROR, work_id);
|
||||
} else {
|
||||
if (llm_task_.find(work_id_num) == llm_task_.end()) {
|
||||
req_body["code"] = -6;
|
||||
req_body["message"] = "Unit Does Not Exist";
|
||||
send("None", "None", req_body, work_id);
|
||||
return;
|
||||
}
|
||||
auto llm_task_obj = llm_task_[work_id_num];
|
||||
req_body["model"] = llm_task_obj->model_;
|
||||
req_body["response_format"] = llm_task_obj->response_format_;
|
||||
req_body["enoutput"] = llm_task_obj->enoutput_;
|
||||
req_body["inputs_"] = llm_task_obj->inputs_;
|
||||
send("llm.taskinfo", req_body, LLM_NO_ERROR, work_id);
|
||||
}
|
||||
}
|
||||
|
||||
int exit(const std::string &work_id, const std::string &object, const std::string &data) override {
|
||||
nlohmann::json error_body;
|
||||
int work_id_num = sample_get_work_id_num(work_id);
|
||||
if (llm_task_.find(work_id_num) == llm_task_.end()) {
|
||||
error_body["code"] = -6;
|
||||
error_body["message"] = "Unit Does Not Exist";
|
||||
send("None", "None", error_body, work_id);
|
||||
return -1;
|
||||
}
|
||||
auto llm_channel = get_channel(work_id_num);
|
||||
llm_channel->stop_subscriber("");
|
||||
llm_task_.erase(work_id_num);
|
||||
send("None", "None", LLM_NO_ERROR, work_id);
|
||||
return 0;
|
||||
}
|
||||
|
||||
~llm_llm() {
|
||||
while (1) {
|
||||
auto iteam = llm_task_.begin();
|
||||
if (iteam == llm_task_.end()) {
|
||||
break;
|
||||
}
|
||||
iteam->second.reset();
|
||||
llm_task_.erase(iteam->first);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
int main(int argc, char *argv[]) {
|
||||
signal(SIGTERM, __sigint);
|
||||
signal(SIGINT, __sigint);
|
||||
mkdir("/tmp/llm", 0777);
|
||||
llm_llm llm;
|
||||
while (!main_exit_flage) {
|
||||
sleep(1);
|
||||
}
|
||||
llm.llm_firework_exit();
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
## StackFlowUtil
|
||||
Provides a convenient API:
|
||||
- sample_json_str_get: Simple function to read key values inside a JSON, used to quickly read JSON keys without parsing the JSON object.
|
||||
- sample_get_work_id_num: Read the numeric index from the work_id string.
|
||||
- sample_get_work_id_name: Read the unit name from the work_id string.
|
||||
- sample_get_work_id: Used to synthesize the work_id string.
|
||||
- sample_escapeString: Simple encoding of escape characters in a string.
|
||||
- sample_unescapeString: Simple decoding of escape strings in a string.
|
||||
- decode_stream: Parse streaming data flow.
|
||||
- decode_base64: Decode base64.
|
||||
- encode_base64: Encode base64.
|
||||
@@ -0,0 +1,418 @@
|
||||
# StackFlow
|
||||
StackFlow 是 M5STACK 为 AI 加速计算开发的一个通信框架，主要运行在嵌入式 Linux 平台，由 [zmq](https://zeromq.org/) 提供底层通信服务。
|
||||
StackFlow 主要提供三种功能，一、远程 RPC 调用，承载单元之间的函数调用。二、消息通信，提供标准消息流服务，更好的串通上下文。三、资源分配，主要用于避免相关单元的通信地址冲突和临时数据储存。
|
||||
|
||||
|
||||
## pzmq
|
||||
重新包装了 zmq 的 api，让 ZMQ_PUB、ZMQ_SUB、ZMQ_PUSH、ZMQ_PULL 的调用更加简单便捷，采用异步式回调的方法提供接收消息功能。
|
||||
在 ZMQ_REP、ZMQ_REQ 的基础上封装了简单的 RPC 功能，提供 RPC 服务。
|
||||
|
||||
### 相关示例
|
||||
1、ZMQ_PULL
|
||||
```c++
|
||||
/*
|
||||
* SPDX-FileCopyrightText: 2024 M5Stack Technology CO LTD
|
||||
*
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
#include <iostream>
|
||||
#include "pzmq.hpp"
|
||||
#include <string>
|
||||
using namespace StackFlows;
|
||||
void pull_msg(const std::string &raw_msg){
|
||||
std::cout << raw_msg << std::endl;
|
||||
}
|
||||
int main(int argc, char *argv[]) {
|
||||
pzmq zpull_("ipc:///tmp.5000.socket", ZMQ_PULL, pull_msg);
|
||||
while(1) {
|
||||
sleep(1);
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
2、ZMQ_PUSH
|
||||
```c++
|
||||
/*
|
||||
* SPDX-FileCopyrightText: 2024 M5Stack Technology CO LTD
|
||||
*
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
#include <iostream>
|
||||
#include "pzmq.hpp"
|
||||
#include <string>
|
||||
using namespace StackFlows;
|
||||
int main(int argc, char *argv[]) {
|
||||
pzmq zpush_("ipc:///tmp.5000.socket", ZMQ_PUSH);
|
||||
zpush_.send_data("nihao");
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
3、ZMQ_PUB
|
||||
```c++
|
||||
/*
|
||||
* SPDX-FileCopyrightText: 2024 M5Stack Technology CO LTD
|
||||
*
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
#include <iostream>
|
||||
#include "pzmq.hpp"
|
||||
#include <string>
|
||||
using namespace StackFlows;
|
||||
int main(int argc, char *argv[]) {
|
||||
pzmq zpush_("ipc:///tmp.5001.socket", ZMQ_PUB);
|
||||
zpush_.send_data("nihao");
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
4、ZMQ_SUB
|
||||
```c++
|
||||
/*
|
||||
* SPDX-FileCopyrightText: 2024 M5Stack Technology CO LTD
|
||||
*
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
#include <iostream>
|
||||
#include "pzmq.hpp"
|
||||
#include <string>
|
||||
using namespace StackFlows;
|
||||
void sub_msg(const std::string &raw_msg){
|
||||
std::cout << raw_msg << std::endl;
|
||||
}
|
||||
int main(int argc, char *argv[]) {
|
||||
pzmq zpull_("ipc:///tmp.5001.socket", ZMQ_SUB, sub_msg);
|
||||
while(1) {
|
||||
sleep(1);
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
4、ZMQ_RPC_FUN
|
||||
```c++
|
||||
/*
|
||||
* SPDX-FileCopyrightText: 2024 M5Stack Technology CO LTD
|
||||
*
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
#include <iostream>
|
||||
#include "pzmq.hpp"
|
||||
#include <string>
|
||||
using namespace StackFlows;
|
||||
std::string fun1_(const std::string &raw_msg){
|
||||
std::cout << raw_msg << std::endl;
|
||||
return std::string("nihao");
|
||||
}
|
||||
std::string fun2_(const std::string &raw_msg){
|
||||
std::cout << raw_msg << std::endl;
|
||||
return std::string("hello");
|
||||
}
|
||||
int main(int argc, char *argv[]) {
|
||||
pzmq _rpc("test");
|
||||
_rpc.register_rpc_action("fun1", fun1_);
|
||||
_rpc.register_rpc_action("fun2", fun2_);
|
||||
while(1) {
|
||||
sleep(1);
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
5、ZMQ_RPC_CALL
|
||||
```c++
|
||||
/*
|
||||
* SPDX-FileCopyrightText: 2024 M5Stack Technology CO LTD
|
||||
*
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
#include <iostream>
|
||||
#include "pzmq.hpp"
|
||||
#include <string>
|
||||
using namespace StackFlows;
|
||||
std::string fun1_(const std::string &raw_msg){
|
||||
return std::string("nihao");
|
||||
}
|
||||
std::string fun2_(const std::string &raw_msg){
|
||||
return std::string("hello");
|
||||
}
|
||||
int main(int argc, char *argv[]) {
|
||||
pzmq _rpc("test");
|
||||
_rpc.call_rpc_action("fun1", "call fun1_", [](const std::string &raw_msg){std::cout << raw_msg << std::endl;});
|
||||
_rpc.call_rpc_action("fun2", "call fun2_", [](const std::string &raw_msg){std::cout << raw_msg << std::endl;});
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
## StackFlow 主体
|
||||
StackFlow 封装了 pzmq 和 eventpp，为加速单元提供基础的 RPC 函数、异步处理和信道建立。
|
||||
StackFlow 提供基本的七个 RPC 函数，用于 StackFlow json 协议的基础功能调用。
|
||||
- setup：单元配置函数，是每个单元必须实现的函数。
|
||||
- pause：暂停单元函数。
|
||||
- work：单元开始工作函数。
|
||||
- exit：单元退出函数，是每个单元必须实现的函数。
|
||||
- link：链接单元上级输出函数，用于构建消息通信链。
|
||||
- unlink：解除上级输出函数，不在接收上级的消息。
|
||||
- taskinfo：获取单元运行信息。
|
||||
|
||||
StackFlow 提供了简便 API 方便单元使用：
|
||||
- unit_call: 单元 RPC 调用函数，调用其他单元的 RPC 函数。
|
||||
- sys_sql_select: sys 单元的简单键值数据库查寻函数。
|
||||
- sys_sql_set: sys 单元的简单键值数据库键值设置函数。
|
||||
- sys_sql_unset: sys 单元的简单键值数据库删除键值函数。
|
||||
- repeat_event: 异步的定时重复执行函数。
|
||||
- send: 发送用户消息函数。
|
||||
- sys_register_unit: 单元注册函数，一般情况不需要调用。
|
||||
- sys_release_unit: 单元释放函数，一般情况不需要调用。
|
||||
|
||||
llm_channel_obj 封装了单元所需的通信函数，一份配置对应一个 llm_channel_obj 对象。
|
||||
llm_channel_obj 提供了通信简便 API 方便单元使用：
|
||||
- subscriber_work_id: 订阅上级 work_id 单元的 pub 输出。
|
||||
- stop_subscriber_work_id：取消订阅 work_id 。
|
||||
- subscriber： 订阅 zmq url 的 pub 输出。
|
||||
- stop_subscriber： 取消订阅 zmq_url 。
|
||||
- send：将本单元的消息通过 pub 发送出去。
|
||||
|
||||
### 基本使用示例：
|
||||
``` c++
|
||||
/*
|
||||
* SPDX-FileCopyrightText: 2024 M5Stack Technology CO LTD
|
||||
*
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
#include "StackFlow.h"
|
||||
#include <signal.h>
|
||||
#include <sys/stat.h>
|
||||
#include <sys/types.h>
|
||||
#include <unistd.h>
|
||||
#include <fstream>
|
||||
#include <stdexcept>
|
||||
#include <iostream>
|
||||
using namespace StackFlows;
|
||||
int main_exit_flage = 0;
|
||||
static void __sigint(int iSigNo) {
|
||||
main_exit_flage = 1;
|
||||
}
|
||||
typedef std::function<void(const std::string &data, bool finish)> task_callback_t;
|
||||
class llm_task {
|
||||
private:
|
||||
public:
|
||||
std::string model_;
|
||||
std::string response_format_;
|
||||
std::vector<std::string> inputs_;
|
||||
task_callback_t out_callback_;
|
||||
bool enoutput_;
|
||||
bool enstream_;
|
||||
|
||||
void set_output(task_callback_t out_callback) {
|
||||
out_callback_ = out_callback;
|
||||
}
|
||||
|
||||
bool parse_config(const nlohmann::json &config_body) {
|
||||
try {
|
||||
model_ = config_body.at("model");
|
||||
response_format_ = config_body.at("response_format");
|
||||
enoutput_ = config_body.at("enoutput");
|
||||
if (config_body.contains("input")) {
|
||||
if (config_body["input"].is_string()) {
|
||||
inputs_.push_back(config_body["input"].get<std::string>());
|
||||
} else if (config_body["input"].is_array()) {
|
||||
for (auto _in : config_body["input"]) {
|
||||
inputs_.push_back(_in.get<std::string>());
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (...) {
|
||||
return true;
|
||||
}
|
||||
enstream_ = (response_format_.find("stream") != std::string::npos);
|
||||
return false;
|
||||
}
|
||||
|
||||
int load_model(const nlohmann::json &config_body) {
|
||||
if (parse_config(config_body)) {
|
||||
return -1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
void inference(const std::string &msg) {
|
||||
std::cout << msg << std::endl;
|
||||
if (out_callback_) out_callback_(std::string("hello"), true);
|
||||
}
|
||||
|
||||
llm_task(const std::string &workid) {
|
||||
}
|
||||
|
||||
~llm_task() {
|
||||
}
|
||||
};
|
||||
|
||||
class llm_llm : public StackFlow {
|
||||
private:
|
||||
int task_count_;
|
||||
std::unordered_map<int, std::shared_ptr<llm_task>> llm_task_;
|
||||
|
||||
public:
|
||||
llm_llm() : StackFlow("test") {
|
||||
task_count_ = 1;
|
||||
}
|
||||
|
||||
void task_output(const std::shared_ptr<llm_task> llm_task_obj, const std::shared_ptr<llm_channel_obj> llm_channel,
|
||||
const std::string &data, bool finish) {
|
||||
if (llm_channel->enstream_) {
|
||||
static int count = 0;
|
||||
nlohmann::json data_body;
|
||||
data_body["index"] = count++;
|
||||
data_body["delta"] = data;
|
||||
if (!finish)
|
||||
data_body["delta"] = data;
|
||||
else
|
||||
data_body["delta"] = std::string("");
|
||||
data_body["finish"] = finish;
|
||||
if (finish) count = 0;
|
||||
llm_channel->send(llm_task_obj->response_format_, data_body, LLM_NO_ERROR);
|
||||
} else if (finish) {
|
||||
llm_channel->send(llm_task_obj->response_format_, data, LLM_NO_ERROR);
|
||||
}
|
||||
}
|
||||
|
||||
void task_user_data(const std::shared_ptr<llm_task> llm_task_obj,
|
||||
const std::shared_ptr<llm_channel_obj> llm_channel, const std::string &object,
|
||||
const std::string &data) {
|
||||
const std::string *next_data = &data;
|
||||
int ret;
|
||||
std::string tmp_msg1;
|
||||
if (object.find("stream") != std::string::npos) {
|
||||
static std::unordered_map<int, std::string> stream_buff;
|
||||
if (decode_stream(data, tmp_msg1, stream_buff)) return;
|
||||
next_data = &tmp_msg1;
|
||||
}
|
||||
std::string tmp_msg2;
|
||||
if (object.find("base64") != std::string::npos) {
|
||||
ret = decode_base64((*next_data), tmp_msg2);
|
||||
if (!ret) {
|
||||
return;
|
||||
}
|
||||
next_data = &tmp_msg2;
|
||||
}
|
||||
llm_task_obj->inference((*next_data));
|
||||
}
|
||||
|
||||
int setup(const std::string &work_id, const std::string &object, const std::string &data) override {
|
||||
nlohmann::json error_body;
|
||||
if ((llm_task_channel_.size() - 1) == task_count_) {
|
||||
error_body["code"] = -21;
|
||||
error_body["message"] = "task full";
|
||||
send("None", "None", error_body, unit_name_);
|
||||
return -1;
|
||||
}
|
||||
int work_id_num = sample_get_work_id_num(work_id);
|
||||
auto llm_channel = get_channel(work_id);
|
||||
auto llm_task_obj = std::make_shared<llm_task>(work_id);
|
||||
nlohmann::json config_body;
|
||||
try {
|
||||
config_body = nlohmann::json::parse(data);
|
||||
} catch (...) {
|
||||
error_body["code"] = -2;
|
||||
error_body["message"] = "json format error.";
|
||||
send("None", "None", error_body, unit_name_);
|
||||
return -2;
|
||||
}
|
||||
int ret = llm_task_obj->load_model(config_body);
|
||||
if (ret == 0) {
|
||||
llm_channel->set_output(llm_task_obj->enoutput_);
|
||||
llm_channel->set_stream(llm_task_obj->enstream_);
|
||||
llm_task_obj->set_output(std::bind(&llm_llm::task_output, this, llm_task_obj, llm_channel,
|
||||
std::placeholders::_1, std::placeholders::_2));
|
||||
llm_channel->subscriber_work_id("", std::bind(&llm_llm::task_user_data, this, llm_task_obj, llm_channel,
|
||||
std::placeholders::_1, std::placeholders::_2));
|
||||
llm_task_[work_id_num] = llm_task_obj;
|
||||
send("None", "None", LLM_NO_ERROR, work_id);
|
||||
return 0;
|
||||
} else {
|
||||
error_body["code"] = -5;
|
||||
error_body["message"] = "Model loading failed.";
|
||||
send("None", "None", error_body, unit_name_);
|
||||
return -1;
|
||||
}
|
||||
}
|
||||
|
||||
void taskinfo(const std::string &work_id, const std::string &object, const std::string &data) override {
|
||||
nlohmann::json req_body;
|
||||
int work_id_num = sample_get_work_id_num(work_id);
|
||||
if (WORK_ID_NONE == work_id_num) {
|
||||
std::vector<std::string> task_list;
|
||||
std::transform(llm_task_channel_.begin(), llm_task_channel_.end(), std::back_inserter(task_list),
|
||||
[](const auto task_channel) { return task_channel.second->work_id_; });
|
||||
req_body = task_list;
|
||||
send("llm.tasklist", req_body, LLM_NO_ERROR, work_id);
|
||||
} else {
|
||||
if (llm_task_.find(work_id_num) == llm_task_.end()) {
|
||||
req_body["code"] = -6;
|
||||
req_body["message"] = "Unit Does Not Exist";
|
||||
send("None", "None", req_body, work_id);
|
||||
return;
|
||||
}
|
||||
auto llm_task_obj = llm_task_[work_id_num];
|
||||
req_body["model"] = llm_task_obj->model_;
|
||||
req_body["response_format"] = llm_task_obj->response_format_;
|
||||
req_body["enoutput"] = llm_task_obj->enoutput_;
|
||||
req_body["inputs_"] = llm_task_obj->inputs_;
|
||||
send("llm.taskinfo", req_body, LLM_NO_ERROR, work_id);
|
||||
}
|
||||
}
|
||||
|
||||
int exit(const std::string &work_id, const std::string &object, const std::string &data) override {
|
||||
nlohmann::json error_body;
|
||||
int work_id_num = sample_get_work_id_num(work_id);
|
||||
if (llm_task_.find(work_id_num) == llm_task_.end()) {
|
||||
error_body["code"] = -6;
|
||||
error_body["message"] = "Unit Does Not Exist";
|
||||
send("None", "None", error_body, work_id);
|
||||
return -1;
|
||||
}
|
||||
auto llm_channel = get_channel(work_id_num);
|
||||
llm_channel->stop_subscriber("");
|
||||
llm_task_.erase(work_id_num);
|
||||
send("None", "None", LLM_NO_ERROR, work_id);
|
||||
return 0;
|
||||
}
|
||||
|
||||
~llm_llm() {
|
||||
while (1) {
|
||||
auto iteam = llm_task_.begin();
|
||||
if (iteam == llm_task_.end()) {
|
||||
break;
|
||||
}
|
||||
iteam->second.reset();
|
||||
llm_task_.erase(iteam->first);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
int main(int argc, char *argv[]) {
|
||||
signal(SIGTERM, __sigint);
|
||||
signal(SIGINT, __sigint);
|
||||
mkdir("/tmp/llm", 0777);
|
||||
llm_llm llm;
|
||||
while (!main_exit_flage) {
|
||||
sleep(1);
|
||||
}
|
||||
llm.llm_firework_exit();
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
## StackFlowUtil
|
||||
提供一个简便使用的 API：
|
||||
- sample_json_str_get: 简单的读取 json 内的键值函数，用于在不解析 json 对象的情况下快速读取 json 键值。
|
||||
- sample_get_work_id_num： 从 work_id 字符串中读取数字索引。
|
||||
- sample_get_work_id_name： 从 work_id 字符串中读取单元名。
|
||||
- sample_get_work_id： 用于合成 work_id 字符串。
|
||||
- sample_escapeString：简单的对字符串中的转义字符进行编码。
|
||||
- sample_unescapeString：简单的对字符串中的转义字符串进行解码。
|
||||
- decode_stream：解析流式数据流。
|
||||
- decode_base64：解码 base64
|
||||
- encode_base64：编码 base64
|
||||
@@ -0,0 +1,297 @@
|
||||
# llm-asr
|
||||
A speech-to-text module that provides speech-to-text services. It supports both Chinese and English models for converting speech to text in these languages.
|
||||
|
||||
## setup
|
||||
Configure the unit.
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "asr",
|
||||
"action": "setup",
|
||||
"object": "asr.setup",
|
||||
"data": {
|
||||
"model": "sherpa-ncnn-streaming-zipformer-zh-14M-2023-02-23",
|
||||
"response_format": "asr.utf-8.stream",
|
||||
"input": "sys.pcm",
|
||||
"enoutput": true,
|
||||
"endpoint_config.rule1.min_trailing_silence": 2.4,
|
||||
"endpoint_config.rule2.min_trailing_silence": 1.2,
|
||||
"endpoint_config.rule3.min_trailing_silence": 30.1,
|
||||
"endpoint_config.rule1.must_contain_nonsilence": true,
|
||||
"endpoint_config.rule2.must_contain_nonsilence": true,
|
||||
"endpoint_config.rule3.must_contain_nonsilence": true
|
||||
}
|
||||
}
|
||||
```
|
||||
- request_id: Reference basic data explanation.
|
||||
- work_id: For configuration units, it is `asr`.
|
||||
- action: The method to be called is `setup`.
|
||||
- object: The type of data being transmitted is `asr.setup`.
|
||||
- model: The model used is the Chinese model `sherpa-ncnn-streaming-zipformer-zh-14M-2023-02-23`.
|
||||
- response_format: The result format is `asr.utf-8.stream`, a UTF-8 stream output.
|
||||
- input: The input is `sys.pcm`, representing system audio.
|
||||
- enoutput: Whether to enable user result output.
|
||||
- endpoint_config.rule1.min_trailing_silence: A speech break occurs 2.4 seconds after wake-up.
|
||||
- endpoint_config.rule2.min_trailing_silence: A speech break occurs 1.2 seconds after a speech is recognized.
|
||||
- endpoint_config.rule3.min_trailing_silence: A speech break occurs after a maximum of 30.1 seconds of recognition.
|
||||
|
||||
Response JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"created": 1731488402,
|
||||
"data": "None",
|
||||
"error": {
|
||||
"code": 0,
|
||||
"message": ""
|
||||
},
|
||||
"object": "None",
|
||||
"request_id": "2",
|
||||
"work_id": "asr.1001"
|
||||
}
|
||||
```
|
||||
- created: Message creation time, Unix time.
|
||||
- work_id: The successfully created work_id unit.
|
||||
|
||||
## link
|
||||
Link the output of the upstream unit.
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "3",
|
||||
"work_id": "asr.1001",
|
||||
"action": "link",
|
||||
"object": "work_id",
|
||||
"data": "kws.1000"
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"created": 1731488402,
|
||||
"data": "None",
|
||||
"error": {
|
||||
"code": 0,
|
||||
"message": ""
|
||||
},
|
||||
"object": "None",
|
||||
"request_id": "3",
|
||||
"work_id": "asr.1001"
|
||||
}
|
||||
```
|
||||
error::code of 0 indicates successful execution.
|
||||
|
||||
Link the asr and kws units so that when kws sends wake-up data, the asr unit starts recognizing the user's speech, pauses automatically after recognition, and waits until the next wake-up.
|
||||
|
||||
> **Ensure that kws is already configured and working when linking. Linking can also be done during the setup phase.**
|
||||
|
||||
Example:
|
||||
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "asr",
|
||||
"action": "setup",
|
||||
"object": "asr.setup",
|
||||
"data": {
|
||||
"model": "sherpa-ncnn-streaming-zipformer-zh-14M-2023-02-23",
|
||||
"response_format": "asr.utf-8.stream",
|
||||
"input": ["sys.pcm", "kws.1000"],
|
||||
"enoutput": true,
|
||||
"endpoint_config.rule1.min_trailing_silence": 2.4,
|
||||
"endpoint_config.rule2.min_trailing_silence": 1.2,
|
||||
"endpoint_config.rule3.min_trailing_silence": 30.1,
|
||||
"endpoint_config.rule1.must_contain_nonsilence": false,
|
||||
"endpoint_config.rule2.must_contain_nonsilence": false,
|
||||
"endpoint_config.rule3.must_contain_nonsilence": false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## unlink
|
||||
Unlink
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "4",
|
||||
"work_id": "asr.1001",
|
||||
"action": "unlink",
|
||||
"object": "work_id",
|
||||
"data": "kws.1000"
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"created": 1731488402,
|
||||
"data": "None",
|
||||
"error": {
|
||||
"code": 0,
|
||||
"message": ""
|
||||
},
|
||||
"object": "None",
|
||||
"request_id": "4",
|
||||
"work_id": "asr.1001"
|
||||
}
|
||||
```
|
||||
error::code of 0 indicates successful execution.
|
||||
|
||||
## pause
|
||||
Pause the unit's work.
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "5",
|
||||
"work_id": "asr.1001",
|
||||
"action": "pause"
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"created": 1731488402,
|
||||
"data": "None",
|
||||
"error": {
|
||||
"code": 0,
|
||||
"message": ""
|
||||
},
|
||||
"object": "None",
|
||||
"request_id": "5",
|
||||
"work_id": "asr.1001"
|
||||
}
|
||||
```
|
||||
error::code of 0 indicates successful execution.
|
||||
|
||||
## work
|
||||
Resume the unit's work.
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "6",
|
||||
"work_id": "asr.1001",
|
||||
"action": "work"
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"created": 1731488402,
|
||||
"data": "None",
|
||||
"error": {
|
||||
"code": 0,
|
||||
"message": ""
|
||||
},
|
||||
"object": "None",
|
||||
"request_id": "6",
|
||||
"work_id": "asr.1001"
|
||||
}
|
||||
```
|
||||
error::code of 0 indicates successful execution.
|
||||
|
||||
## exit
|
||||
Exit the unit's work.
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "7",
|
||||
"work_id": "asr.1001",
|
||||
"action": "exit"
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"created": 1731488402,
|
||||
"data": "None",
|
||||
"error": {
|
||||
"code": 0,
|
||||
"message": ""
|
||||
},
|
||||
"object": "None",
|
||||
"request_id": "7",
|
||||
"work_id": "asr.1001"
|
||||
}
|
||||
```
|
||||
error::code of 0 indicates successful execution.
|
||||
|
||||
## Task Information
|
||||
|
||||
Get task list:
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "asr",
|
||||
"action": "taskinfo"
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
```json
|
||||
{
|
||||
"created":1731580350,
|
||||
"data":[
|
||||
"asr.1001"
|
||||
],
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"asr.tasklist",
|
||||
"request_id":"2",
|
||||
"work_id":"asr"
|
||||
}
|
||||
```
|
||||
|
||||
Get task runtime parameters:
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "asr.1001",
|
||||
"action": "taskinfo"
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
```json
|
||||
{
|
||||
"created":1731579679,
|
||||
"data":{
|
||||
"enoutput":false,
|
||||
"inputs_":[
|
||||
"sys.pcm"
|
||||
],
|
||||
"model":"sherpa-ncnn-streaming-zipformer-zh-14M-2023-02-23",
|
||||
"response_format":"asr.utf-8-stream"
|
||||
},
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"asr.taskinfo",
|
||||
"request_id":"2",
|
||||
"work_id":"asr.1001"
|
||||
}
|
||||
```
|
||||
|
||||
> **Note: The work_id increases according to the initialization registration order of the unit and is not a fixed index value.**
|
||||
@@ -0,0 +1,300 @@
|
||||
# llm-asr
|
||||
语音转文字单元，用于提供语音转文字服务，可选择中英文模型，用于提供中英文语音转文字服务。
|
||||
|
||||
## setup
|
||||
配置单元工作.
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "asr",
|
||||
"action": "setup",
|
||||
"object": "asr.setup",
|
||||
"data": {
|
||||
"model": "sherpa-ncnn-streaming-zipformer-zh-14M-2023-02-23",
|
||||
"response_format": "asr.utf-8.stream",
|
||||
"input": "sys.pcm",
|
||||
"enoutput": true,
|
||||
"endpoint_config.rule1.min_trailing_silence":2.4,
|
||||
"endpoint_config.rule2.min_trailing_silence":1.2,
|
||||
"endpoint_config.rule3.min_trailing_silence":30.1,
|
||||
"endpoint_config.rule1.must_contain_nonsilence":true,
|
||||
"endpoint_config.rule2.must_contain_nonsilence":true,
|
||||
"endpoint_config.rule3.must_contain_nonsilence":true
|
||||
}
|
||||
}
|
||||
```
|
||||
- request_id：参考基本数据解释。
|
||||
- work_id：配置单元时，为 `asr`。
|
||||
- action：调用的方法为 `setup`。
|
||||
- object：传输的数据类型为 `asr.setup`。
|
||||
- model：使用的模型为 `sherpa-ncnn-streaming-zipformer-zh-14M-2023-02-23` 中文模型。
|
||||
- response_format：返回结果为 `asr.utf-8.stream`, utf-8 的流式输出。
|
||||
- input：输入的为 `sys.pcm`,代表的是系统音频。
|
||||
- enoutput：是否起用用户结果输出。
|
||||
- endpoint_config.rule1.min_trailing_silence：唤醒后 2.4 s 后语音产生断点。
|
||||
- endpoint_config.rule2.min_trailing_silence：识别语音后停顿 1.2 s 后语音产生断点。
|
||||
- endpoint_config.rule3.min_trailing_silence：最长能够识别 30.1 s 后语音产生断点。
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"2",
|
||||
"work_id":"asr.1001"
|
||||
}
|
||||
```
|
||||
|
||||
- created：消息创建时间，unix 时间。
|
||||
- work_id：返回成功创建的 work_id 单元。
|
||||
|
||||
## link
|
||||
链接上级单元的输出。
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "3",
|
||||
"work_id": "asr.1001",
|
||||
"action": "link",
|
||||
"object":"work_id",
|
||||
"data":"kws.1000"
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"3",
|
||||
"work_id":"asr.1001"
|
||||
}
|
||||
```
|
||||
error::code 为 0 表示执行成功。
|
||||
|
||||
将 asr 和 kws 单元链接起来，当 kws 发出唤醒数据时，asr 单元开始识别用户的语音，识别完成后自动暂停，直到下一次的唤醒。
|
||||
|
||||
> **link时必须保证 kws 此时已经配置好进入工作状态。link 也可以在 setup 阶段进行。**
|
||||
|
||||
示例:
|
||||
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "asr",
|
||||
"action": "setup",
|
||||
"object": "asr.setup",
|
||||
"data": {
|
||||
"model": "sherpa-ncnn-streaming-zipformer-zh-14M-2023-02-23",
|
||||
"response_format": "asr.utf-8.stream",
|
||||
"input": ["sys.pcm","kws.1000"],
|
||||
"enoutput": true,
|
||||
"endpoint_config.rule1.min_trailing_silence":2.4,
|
||||
"endpoint_config.rule2.min_trailing_silence":1.2,
|
||||
"endpoint_config.rule3.min_trailing_silence":30.1,
|
||||
"endpoint_config.rule1.must_contain_nonsilence":false,
|
||||
"endpoint_config.rule2.must_contain_nonsilence":false,
|
||||
"endpoint_config.rule3.must_contain_nonsilence":false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## unlink
|
||||
取消链接
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "4",
|
||||
"work_id": "asr.1001",
|
||||
"action": "unlink",
|
||||
"object":"work_id",
|
||||
"data":"kws.1000"
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"4",
|
||||
"work_id":"asr.1001"
|
||||
}
|
||||
```
|
||||
|
||||
error::code 为 0 表示执行成功。
|
||||
|
||||
## pause
|
||||
暂停单元工作。
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "5",
|
||||
"work_id": "asr.1001",
|
||||
"action": "pause",
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"5",
|
||||
"work_id":"asr.1001"
|
||||
}
|
||||
```
|
||||
error::code 为 0 表示执行成功。
|
||||
|
||||
## work
|
||||
恢复单元工作.
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "6",
|
||||
"work_id": "asr.1001",
|
||||
"action": "work",
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"6",
|
||||
"work_id":"asr.1001"
|
||||
}
|
||||
```
|
||||
error::code 为 0 表示执行成功。
|
||||
|
||||
## exit
|
||||
恢复单元工作.
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "7",
|
||||
"work_id": "asr.1001",
|
||||
"action": "exit",
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"7",
|
||||
"work_id":"asr.1001"
|
||||
}
|
||||
```
|
||||
error::code 为 0 表示执行成功。
|
||||
|
||||
## taskinfo
|
||||
|
||||
获取任务列表：
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "asr",
|
||||
"action": "taskinfo"
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731580350,
|
||||
"data":[
|
||||
"asr.1001"
|
||||
],
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"asr.tasklist",
|
||||
"request_id":"2",
|
||||
"work_id":"asr"
|
||||
}
|
||||
```
|
||||
|
||||
获取任务运行参数：
|
||||
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "asr.1001",
|
||||
"action": "taskinfo"
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731579679,
|
||||
"data":{
|
||||
"enoutput":false,
|
||||
"inputs_":[
|
||||
"sys.pcm"
|
||||
],
|
||||
"model":"sherpa-ncnn-streaming-zipformer-zh-14M-2023-02-23",
|
||||
"response_format":"asr.utf-8-stream"
|
||||
},
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"asr.taskinfo",
|
||||
"request_id":"2",
|
||||
"work_id":"asr.1001"
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
> **注意：work_id 是按照单元的初始化注册顺序增加的，并不是固定的索引值。**
|
||||
@@ -0,0 +1,13 @@
|
||||
# llm-audio
|
||||
System audio unit for providing system audio playback and recording.
|
||||
|
||||
## API
|
||||
- play: Play audio. This call will interrupt any previously unfinished audio playback.
|
||||
- queue_play: Queue playback. Add audio to the playback queue.
|
||||
- play_stop: Stop playback.
|
||||
- queue_play_stop: Clear the playback queue.
|
||||
- audio_status: Get the current playback status.
|
||||
- cap: Start a recording task, can be called repeatedly.
|
||||
- cap_stop: Stop a recording task, can be called repeatedly. The last call will stop the data output of the recording channel.
|
||||
- cap_stop_all: Force stop all ongoing recording tasks.
|
||||
- setup: Configure the working parameters of the recording unit.
|
||||
@@ -0,0 +1,16 @@
|
||||
# llm-audio
|
||||
系统音频单元，用于提供系统音频播放和录音。
|
||||
|
||||
|
||||
## API
|
||||
- play: 播放音频，本次调用会打断上次未播放的完的音频。
|
||||
- queue_play：队列播放，将音频加入播放队列。
|
||||
- play_stop：停止播放。
|
||||
- queue_play_stop：清理播放队列。
|
||||
- audio_status：当前播放状态。
|
||||
- cap：开始一个录音任务，可重复调用。
|
||||
- cap_stop：停止一个录音任务，可重复调用，最后一个调用会停止录音信道的数据输出。
|
||||
- cap_stop_all：强制停止所有开启的录音任务。
|
||||
- setup: 配置录音单元的工作参数。
|
||||
|
||||
|
||||
@@ -0,0 +1,199 @@
|
||||
# llm-kws
|
||||
Voice wake-up unit, used to provide voice wake-up services. You can choose between Chinese and English models to provide wake-up services in either language.
|
||||
|
||||
## setup
|
||||
Configure the unit to work.
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "kws",
|
||||
"action": "setup",
|
||||
"object": "kws.setup",
|
||||
"data": {
|
||||
"model": "sherpa-onnx-kws-zipformer-wenetspeech-3.3M-2024-01-01",
|
||||
"response_format": "kws.bool",
|
||||
"input": "sys.pcm",
|
||||
"enoutput": true,
|
||||
"kws": "你好你好"
|
||||
}
|
||||
}
|
||||
```
|
||||
- request_id: Refer to basic data explanation.
|
||||
- work_id: When configuring the unit, it is `kws`.
|
||||
- action: The method called is `setup`.
|
||||
- object: The type of data being transmitted is `kws.setup`.
|
||||
- model: The model used is the Chinese model `sherpa-onnx-kws-zipformer-wenetspeech-3.3M-2024-01-01`.
|
||||
- response_format: The result returned is in `kws.bool` format.
|
||||
- input: The input is `sys.pcm`, representing system audio.
|
||||
- enoutput: Whether to enable user result output.
|
||||
- kws: The Chinese wake-up word is `"你好你好"`.
|
||||
|
||||
Response JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"2",
|
||||
"work_id":"kws.1000"
|
||||
}
|
||||
```
|
||||
- created: Message creation time in Unix time.
|
||||
- work_id: The successfully created work_id unit returned.
|
||||
|
||||
## pause
|
||||
Pause the unit's work.
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "3",
|
||||
"work_id": "kws.1000",
|
||||
"action": "pause",
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"3",
|
||||
"work_id":"kws.1000"
|
||||
}
|
||||
```
|
||||
error::code of 0 indicates successful execution.
|
||||
|
||||
## work
|
||||
Resume the unit's work.
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "4",
|
||||
"work_id": "kws.1000",
|
||||
"action": "work",
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"4",
|
||||
"work_id":"kws.1000"
|
||||
}
|
||||
```
|
||||
error::code of 0 indicates successful execution.
|
||||
|
||||
## exit
|
||||
Resume the unit's work.
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "5",
|
||||
"work_id": "kws.1000",
|
||||
"action": "exit",
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"5",
|
||||
"work_id":"kws.1000"
|
||||
}
|
||||
```
|
||||
error::code of 0 indicates successful execution.
|
||||
|
||||
## Task Information
|
||||
|
||||
Get task list:
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "kws",
|
||||
"action": "taskinfo"
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
```json
|
||||
{
|
||||
"created":1731580350,
|
||||
"data":[
|
||||
"kws.1000"
|
||||
],
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"kws.tasklist",
|
||||
"request_id":"2",
|
||||
"work_id":"kws"
|
||||
}
|
||||
```
|
||||
|
||||
Get task runtime parameters:
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "kws.1000",
|
||||
"action": "taskinfo"
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
```json
|
||||
{
|
||||
"created":1731652086,
|
||||
"data":{
|
||||
"enoutput":true,
|
||||
"inputs_":["sys.pcm"],
|
||||
"model":"sherpa-onnx-kws-zipformer-wenetspeech-3.3M-2024-01-01",
|
||||
"response_format":"kws.bool"
|
||||
},
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"kws.taskinfo",
|
||||
"request_id":"2",
|
||||
"work_id":"kws.1000"
|
||||
}
|
||||
```
|
||||
|
||||
> **Note: work_id increases according to the initialization registration order of the unit and is not a fixed index value.**
|
||||
@@ -0,0 +1,205 @@
|
||||
# llm-kws
|
||||
语音唤醒单元，用于提供语音唤醒服务，可选择中英文模型，用于提供中英文唤醒服务。
|
||||
|
||||
## setup
|
||||
配置单元工作.
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "kws",
|
||||
"action": "setup",
|
||||
"object": "kws.setup",
|
||||
"data": {
|
||||
"model": "sherpa-onnx-kws-zipformer-wenetspeech-3.3M-2024-01-01",
|
||||
"response_format": "kws.bool",
|
||||
"input": "sys.pcm",
|
||||
"enoutput": true,
|
||||
"kws": "你好你好"
|
||||
}
|
||||
}
|
||||
```
|
||||
- request_id：参考基本数据解释。
|
||||
- work_id：配置单元时，为 `kws`。
|
||||
- action：调用的方法为 `setup`。
|
||||
- object：传输的数据类型为 `kws.setup`。
|
||||
- model：使用的模型为 `sherpa-onnx-kws-zipformer-wenetspeech-3.3M-2024-01-01` 中文模型。
|
||||
- response_format：返回结果为 `kws.bool` 型。
|
||||
- input：输入的为 `sys.pcm`,代表的是系统音频。
|
||||
- enoutput：是否起用用户结果输出。
|
||||
- kws：中文唤醒词为 `"你好你好"`。
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"2",
|
||||
"work_id":"kws.1000"
|
||||
}
|
||||
```
|
||||
|
||||
- created：消息创建时间，unix 时间。
|
||||
- work_id：返回成功创建的 work_id 单元。
|
||||
|
||||
## pause
|
||||
暂停单元工作.
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "3",
|
||||
"work_id": "kws.1000",
|
||||
"action": "pause",
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"3",
|
||||
"work_id":"kws.1000"
|
||||
}
|
||||
```
|
||||
|
||||
error::code 为 0 表示执行成功。
|
||||
|
||||
## work
|
||||
恢复单元工作.
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "4",
|
||||
"work_id": "kws.1000",
|
||||
"action": "work",
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"4",
|
||||
"work_id":"kws.1000"
|
||||
}
|
||||
```
|
||||
|
||||
error::code 为 0 表示执行成功。
|
||||
|
||||
## exit
|
||||
恢复单元工作.
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "5",
|
||||
"work_id": "kws.1000",
|
||||
"action": "exit",
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"5",
|
||||
"work_id":"kws.1000"
|
||||
}
|
||||
```
|
||||
|
||||
error::code 为 0 表示执行成功。
|
||||
|
||||
## taskinfo
|
||||
|
||||
获取任务列表：
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "kws",
|
||||
"action": "taskinfo"
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731580350,
|
||||
"data":[
|
||||
"kws.1000"
|
||||
],
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"kws.tasklist",
|
||||
"request_id":"2",
|
||||
"work_id":"kws"
|
||||
}
|
||||
```
|
||||
|
||||
获取任务运行参数：
|
||||
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "kws.1000",
|
||||
"action": "taskinfo"
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731652086,
|
||||
"data":{
|
||||
"enoutput":true,
|
||||
"inputs_":["sys.pcm"],
|
||||
"model":"sherpa-onnx-kws-zipformer-wenetspeech-3.3M-2024-01-01",
|
||||
"response_format":"kws.bool"
|
||||
},
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"kws.taskinfo",
|
||||
"request_id":"2",
|
||||
"work_id":"kws.1000"
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
> **注意：work_id 是按照单元的初始化注册顺序增加的，并不是固定的索引值。**
|
||||
@@ -0,0 +1,284 @@
|
||||
# llm-llm
|
||||
Large Model Unit, used to provide large model inference services.
|
||||
|
||||
## setup
|
||||
Configure the unit.
|
||||
|
||||
Send json:
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "llm",
|
||||
"action": "setup",
|
||||
"object": "llm.setup",
|
||||
"data": {
|
||||
"model": "qwen2.5-0.5B-prefill-20e",
|
||||
"response_format": "llm.utf-8.stream",
|
||||
"input": "llm.utf-8",
|
||||
"enoutput": true,
|
||||
"max_token_len": 256,
|
||||
"prompt": "You are a knowledgeable assistant capable of answering various questions and providing information."
|
||||
}
|
||||
}
|
||||
```
|
||||
- request_id: Refer to the basic data explanation.
|
||||
- work_id: For configuration unit, it is `llm`.
|
||||
- action: The method called is `setup`.
|
||||
- object: The type of data transmitted is `llm.setup`.
|
||||
- model: The model used is the Chinese model `qwen2.5-0.5B-prefill-20e`.
|
||||
- response_format: The result returned is in `llm.utf-8.stream`, utf-8 streaming output.
|
||||
- input: The input is `llm.utf-8`, representing input from the user.
|
||||
- enoutput: Whether to enable user result output.
|
||||
- max_token_len: Maximum output token, this value is limited by the model's maximum limit.
|
||||
- prompt: The prompt for the model.
|
||||
|
||||
Response json:
|
||||
|
||||
```json
|
||||
{
|
||||
"created": 1731488402,
|
||||
"data": "None",
|
||||
"error": {
|
||||
"code": 0,
|
||||
"message": ""
|
||||
},
|
||||
"object": "None",
|
||||
"request_id": "2",
|
||||
"work_id": "llm.1002"
|
||||
}
|
||||
```
|
||||
- created: Message creation time, unix time.
|
||||
- work_id: The successfully created work_id unit.
|
||||
|
||||
## link
|
||||
Link the output of the upper unit.
|
||||
|
||||
Send json:
|
||||
```json
|
||||
{
|
||||
"request_id": "3",
|
||||
"work_id": "llm.1002",
|
||||
"action": "link",
|
||||
"object": "work_id",
|
||||
"data": "kws.1000"
|
||||
}
|
||||
```
|
||||
|
||||
Response json:
|
||||
|
||||
```json
|
||||
{
|
||||
"created": 1731488402,
|
||||
"data": "None",
|
||||
"error": {
|
||||
"code": 0,
|
||||
"message": ""
|
||||
},
|
||||
"object": "None",
|
||||
"request_id": "3",
|
||||
"work_id": "llm.1002"
|
||||
}
|
||||
```
|
||||
error::code of 0 indicates successful execution.
|
||||
|
||||
Link the llm and kws units so that when kws issues wake-up data, the llm unit stops the previous unfinished inference for repeated wake-up functionality.
|
||||
|
||||
> **When linking, ensure that kws has been configured and is in working status. Linking can also be done during the setup phase.**
|
||||
|
||||
Example:
|
||||
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "llm",
|
||||
"action": "setup",
|
||||
"object": "llm.setup",
|
||||
"data": {
|
||||
"model": "qwen2.5-0.5B-prefill-20e",
|
||||
"response_format": "llm.utf-8.stream",
|
||||
"input": ["llm.utf-8", "asr.1001", "kws.1000"],
|
||||
"enoutput": true,
|
||||
"max_token_len": 256,
|
||||
"prompt": "You are a knowledgeable assistant capable of answering various questions and providing information."
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## unlink
|
||||
Unlink
|
||||
|
||||
Send json:
|
||||
```json
|
||||
{
|
||||
"request_id": "4",
|
||||
"work_id": "llm.1002",
|
||||
"action": "unlink",
|
||||
"object": "work_id",
|
||||
"data": "kws.1000"
|
||||
}
|
||||
```
|
||||
|
||||
Response json:
|
||||
|
||||
```json
|
||||
{
|
||||
"created": 1731488402,
|
||||
"data": "None",
|
||||
"error": {
|
||||
"code": 0,
|
||||
"message": ""
|
||||
},
|
||||
"object": "None",
|
||||
"request_id": "4",
|
||||
"work_id": "llm.1002"
|
||||
}
|
||||
```
|
||||
error::code of 0 indicates successful execution.
|
||||
|
||||
## pause
|
||||
Pause the unit.
|
||||
|
||||
Send json:
|
||||
```json
|
||||
{
|
||||
"request_id": "5",
|
||||
"work_id": "llm.1002",
|
||||
"action": "pause"
|
||||
}
|
||||
```
|
||||
|
||||
Response json:
|
||||
|
||||
```json
|
||||
{
|
||||
"created": 1731488402,
|
||||
"data": "None",
|
||||
"error": {
|
||||
"code": 0,
|
||||
"message": ""
|
||||
},
|
||||
"object": "None",
|
||||
"request_id": "5",
|
||||
"work_id": "llm.1002"
|
||||
}
|
||||
```
|
||||
error::code of 0 indicates successful execution.
|
||||
|
||||
## work
|
||||
Resume the unit.
|
||||
|
||||
Send json:
|
||||
```json
|
||||
{
|
||||
"request_id": "6",
|
||||
"work_id": "llm.1002",
|
||||
"action": "work"
|
||||
}
|
||||
```
|
||||
|
||||
Response json:
|
||||
|
||||
```json
|
||||
{
|
||||
"created": 1731488402,
|
||||
"data": "None",
|
||||
"error": {
|
||||
"code": 0,
|
||||
"message": ""
|
||||
},
|
||||
"object": "None",
|
||||
"request_id": "6",
|
||||
"work_id": "llm.1002"
|
||||
}
|
||||
```
|
||||
error::code of 0 indicates successful execution.
|
||||
|
||||
## exit
|
||||
Exit the unit.
|
||||
|
||||
Send json:
|
||||
```json
|
||||
{
|
||||
"request_id": "7",
|
||||
"work_id": "llm.1002",
|
||||
"action": "exit"
|
||||
}
|
||||
```
|
||||
|
||||
Response json:
|
||||
|
||||
```json
|
||||
{
|
||||
"created": 1731488402,
|
||||
"data": "None",
|
||||
"error": {
|
||||
"code": 0,
|
||||
"message": ""
|
||||
},
|
||||
"object": "None",
|
||||
"request_id": "7",
|
||||
"work_id": "llm.1002"
|
||||
}
|
||||
```
|
||||
error::code of 0 indicates successful execution.
|
||||
|
||||
## Task Information
|
||||
|
||||
Get task list:
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "llm",
|
||||
"action": "taskinfo"
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
```json
|
||||
{
|
||||
"created":1731652149,
|
||||
"data":["llm.1002"],
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"llm.tasklist",
|
||||
"request_id":"2",
|
||||
"work_id":"llm"
|
||||
}
|
||||
```
|
||||
|
||||
Get task runtime parameters:
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "llm.1002",
|
||||
"action": "taskinfo"
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
```json
|
||||
{
|
||||
"created":1731652187,
|
||||
"data":{
|
||||
"enoutput":true,
|
||||
"inputs_":["llm.utf-8"],
|
||||
"model":"qwen2.5-0.5B-prefill-20e",
|
||||
"response_format":"llm.utf-8.stream"
|
||||
},
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"llm.taskinfo",
|
||||
"request_id":"2",
|
||||
"work_id":"llm.1002"
|
||||
}
|
||||
```
|
||||
|
||||
> **Note: work_id increases according to the initialization registration order of the unit and is not a fixed index value.**
|
||||
@@ -0,0 +1,287 @@
|
||||
# llm-llm
|
||||
大模型单元，用于提供大模型推理服务。
|
||||
|
||||
## setup
|
||||
配置单元工作.
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "llm",
|
||||
"action": "setup","object": "llm.setup",
|
||||
"data": {
|
||||
"model": "qwen2.5-0.5B-prefill-20e",
|
||||
"response_format": "llm.utf-8.stream",
|
||||
"input": "llm.utf-8",
|
||||
"enoutput": true,
|
||||
"max_token_len": 256,
|
||||
"prompt": "You are a knowledgeable assistant capable of answering various questions and providing information."
|
||||
}
|
||||
}
|
||||
```
|
||||
- request_id：参考基本数据解释。
|
||||
- work_id：配置单元时，为 `llm`。
|
||||
- action：调用的方法为 `setup`。
|
||||
- object：传输的数据类型为 `llm.setup`。
|
||||
- model：使用的模型为 `qwen2.5-0.5B-prefill-20e` 中文模型。
|
||||
- response_format：返回结果为 `llm.utf-8.stream`, utf-8 的流式输出。
|
||||
- input：输入的为 `llm.utf-8`,代表的是从用户输入。
|
||||
- enoutput：是否起用用户结果输出。
|
||||
- max_token_len：最大输出 token,该值的最大值受到模型的最大限制。
|
||||
- prompt：模型的提示词。
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"2",
|
||||
"work_id":"llm.1002"
|
||||
}
|
||||
```
|
||||
|
||||
- created：消息创建时间，unix 时间。
|
||||
- work_id：返回成功创建的 work_id 单元。
|
||||
|
||||
## link
|
||||
链接上级单元的输出。
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "3",
|
||||
"work_id": "llm.1002",
|
||||
"action": "link",
|
||||
"object":"work_id",
|
||||
"data":"kws.1000"
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"3",
|
||||
"work_id":"llm.1002"
|
||||
}
|
||||
```
|
||||
error::code 为 0 表示执行成功。
|
||||
|
||||
将 llm 和 kws 单元链接起来，当 kws 发出唤醒数据时，llm 单元停止上次未完成的推理，用于重复唤醒功能。
|
||||
|
||||
> **link时必须保证 kws 此时已经配置好进入工作状态。link 也可以在 setup 阶段进行。**
|
||||
|
||||
示例:
|
||||
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "llm",
|
||||
"action": "setup","object": "llm.setup",
|
||||
"data": {
|
||||
"model": "qwen2.5-0.5B-prefill-20e",
|
||||
"response_format": "llm.utf-8.stream",
|
||||
"input": ["llm.utf-8", "asr.1001", "kws.1000"],
|
||||
"enoutput": true,
|
||||
"max_token_len": 256,
|
||||
"prompt": "You are a knowledgeable assistant capable of answering various questions and providing information."
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## unlink
|
||||
取消链接
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "4",
|
||||
"work_id": "llm.1002",
|
||||
"action": "unlink",
|
||||
"object":"work_id",
|
||||
"data":"kws.1000"
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"4",
|
||||
"work_id":"llm.1002"
|
||||
}
|
||||
```
|
||||
|
||||
error::code 为 0 表示执行成功。
|
||||
|
||||
## pause
|
||||
暂停单元工作.
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "5",
|
||||
"work_id": "llm.1002",
|
||||
"action": "pause",
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"5",
|
||||
"work_id":"llm.1002"
|
||||
}
|
||||
```
|
||||
|
||||
error::code 为 0 表示执行成功。
|
||||
|
||||
## work
|
||||
恢复单元工作.
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "6",
|
||||
"work_id": "llm.1002",
|
||||
"action": "work",
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"6",
|
||||
"work_id":"llm.1002"
|
||||
}
|
||||
```
|
||||
|
||||
error::code 为 0 表示执行成功。
|
||||
|
||||
## exit
|
||||
恢复单元工作.
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "7",
|
||||
"work_id": "llm.1002",
|
||||
"action": "exit",
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"7",
|
||||
"work_id":"llm.1002"
|
||||
}
|
||||
```
|
||||
|
||||
error::code 为 0 表示执行成功。
|
||||
|
||||
## taskinfo
|
||||
|
||||
获取任务列表：
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "llm",
|
||||
"action": "taskinfo"
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731652149,
|
||||
"data":["llm.1002"],
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"llm.tasklist",
|
||||
"request_id":"2",
|
||||
"work_id":"llm"
|
||||
}
|
||||
```
|
||||
|
||||
获取任务运行参数：
|
||||
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "llm.1002",
|
||||
"action": "taskinfo"
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731652187,
|
||||
"data":{
|
||||
"enoutput":true,
|
||||
"inputs_":["llm.utf-8"],
|
||||
"model":"qwen2.5-0.5B-prefill-20e",
|
||||
"response_format":"llm.utf-8.stream"
|
||||
},
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"llm.taskinfo",
|
||||
"request_id":"2",
|
||||
"work_id":"llm.1002"
|
||||
}
|
||||
```
|
||||
|
||||
> **注意：work_id 是按照单元的初始化注册顺序增加的，并不是固定的索引值。**
|
||||
@@ -0,0 +1,272 @@
|
||||
# llm-melotts
|
||||
Text-to-speech unit accelerated by NPU, used to provide text-to-speech services. It supports both Chinese and English models for text-to-speech conversion.
|
||||
|
||||
## setup
|
||||
Configure the unit.
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "melotts",
|
||||
"action": "setup",
|
||||
"object": "melotts.setup",
|
||||
"data": {
|
||||
"model": "melotts_zh-cn",
|
||||
"response_format": "sys.pcm",
|
||||
"input": "tts.utf-8",
|
||||
"enoutput": false
|
||||
}
|
||||
}
|
||||
```
|
||||
- request_id: Refer to the basic data explanation.
|
||||
- work_id: For configuration, it is `melotts`.
|
||||
- action: The method to be called is `setup`.
|
||||
- object: The data type being transmitted is `melotts.setup`.
|
||||
- model: The model being used is the Chinese model `melotts_zh-cn`.
|
||||
- response_format: The result is returned as `sys.pcm`, system audio data, which is directly sent to the llm-audio module for playback.
|
||||
- input: The input is `tts.utf-8`, representing user input.
|
||||
- enoutput: Whether to enable user result output.
|
||||
|
||||
Response JSON:
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"2",
|
||||
"work_id":"melotts.1003"
|
||||
}
|
||||
```
|
||||
- created: Message creation time, UNIX time.
|
||||
- work_id: Successfully created work_id unit.
|
||||
|
||||
## link
|
||||
Link the output of the upper-level unit.
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "3",
|
||||
"work_id": "melotts.1003",
|
||||
"action": "link",
|
||||
"object":"work_id",
|
||||
"data":"kws.1000"
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"3",
|
||||
"work_id":"melotts.1003"
|
||||
}
|
||||
```
|
||||
error::code being 0 indicates success.
|
||||
|
||||
Link the llm and melotts units. When the kws melotts unit stops the unfinished inference from the last time, it is used for repeated wake-up functionality.
|
||||
|
||||
> **Ensure that kws is already configured and in working status during link. Link can also be performed during the setup stage.**
|
||||
|
||||
Example:
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "melotts",
|
||||
"action": "setup",
|
||||
"object": "melotts.setup",
|
||||
"data": {
|
||||
"model": "melotts_zh-cn",
|
||||
"response_format": "sys.pcm",
|
||||
"input": ["tts.utf-8", "llm.1002", "kws.1000"],
|
||||
"enoutput": false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## unlink
|
||||
Unlink
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "4",
|
||||
"work_id": "melotts.1003",
|
||||
"action": "unlink",
|
||||
"object":"work_id",
|
||||
"data":"kws.1000"
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"4",
|
||||
"work_id":"melotts.1003"
|
||||
}
|
||||
```
|
||||
error::code being 0 indicates success.
|
||||
|
||||
## pause
|
||||
Pause the unit.
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "5",
|
||||
"work_id": "llm.1003",
|
||||
"action": "pause",
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"5",
|
||||
"work_id":"llm.1003"
|
||||
}
|
||||
```
|
||||
error::code being 0 indicates success.
|
||||
|
||||
## work
|
||||
Resume the unit.
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "6",
|
||||
"work_id": "llm.1003",
|
||||
"action": "work",
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"6",
|
||||
"work_id":"llm.1003"
|
||||
}
|
||||
```
|
||||
error::code being 0 indicates success.
|
||||
|
||||
## exit
|
||||
Exit the unit.
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "7",
|
||||
"work_id": "llm.1003",
|
||||
"action": "exit",
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"7",
|
||||
"work_id":"llm.1003"
|
||||
}
|
||||
```
|
||||
error::code being 0 indicates success.
|
||||
|
||||
## Task Information
|
||||
|
||||
Get task list:
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "melotts",
|
||||
"action": "taskinfo"
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
```json
|
||||
{
|
||||
"created":1731652311,
|
||||
"data":["melotts.1003"],
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"melotts.tasklist",
|
||||
"request_id":"2",
|
||||
"work_id":"melotts"
|
||||
}
|
||||
```
|
||||
|
||||
Get task runtime parameters:
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "melotts.1003",
|
||||
"action": "taskinfo"
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
```json
|
||||
{
|
||||
"created":1731652344,
|
||||
"data":{
|
||||
"enoutput":false,
|
||||
"inputs_":["tts.utf-8"],
|
||||
"model":"melotts_zh-cn",
|
||||
"response_format":"sys.pcm"
|
||||
},
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"melotts.taskinfo",
|
||||
"request_id":"2",
|
||||
"work_id":"melotts.1003"
|
||||
}
|
||||
```
|
||||
|
||||
> **Note: work_id increases in the order of the unit's initialization registration and is not a fixed index value.**
|
||||
> **The same type of unit cannot configure multiple units to work simultaneously, or unknown errors may occur. For example, tts and melo tts cannot be activated to work at the same time.**
|
||||
@@ -0,0 +1,282 @@
|
||||
# llm-melotts
|
||||
使用 npu 加速的文字转语音单元，用于提供文字转语音服务，可选择中英文模型，用于提供中英文文字转语音服务。
|
||||
|
||||
## setup
|
||||
配置单元工作.
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "melotts",
|
||||
"action": "setup",
|
||||
"object": "melotts.setup",
|
||||
"data": {
|
||||
"model": "melotts_zh-cn",
|
||||
"response_format": "sys.pcm",
|
||||
"input": "tts.utf-8",
|
||||
"enoutput": false
|
||||
}
|
||||
}
|
||||
```
|
||||
- request_id：参考基本数据解释。
|
||||
- work_id：配置单元时，为 `melotts`。
|
||||
- action：调用的方法为 `setup`。
|
||||
- object：传输的数据类型为 `melotts.setup`。
|
||||
- model：使用的模型为 `melotts_zh-cn` 中文模型。
|
||||
- response_format：返回结果为 `sys.pcm`, 系统音频数据，并直接发送到 llm-audio 模块进行播放。
|
||||
- input：输入的为 `tts.utf-8`,代表的是从用户输入。
|
||||
- enoutput：是否起用用户结果输出。
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"2",
|
||||
"work_id":"melotts.1003"
|
||||
}
|
||||
```
|
||||
- created：消息创建时间，unix 时间。
|
||||
- work_id：返回成功创建的 work_id 单元。
|
||||
|
||||
## link
|
||||
链接上级单元的输出。
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "3",
|
||||
"work_id": "melotts.1003",
|
||||
"action": "link",
|
||||
"object":"work_id",
|
||||
"data":"kws.1000"
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"3",
|
||||
"work_id":"melotts.1003"
|
||||
}
|
||||
```
|
||||
error::code 为 0 表示执行成功。
|
||||
|
||||
将 llm 和 melotts 单元链接起来，当 kws melotts 单元停止上次未完成的推理，用于重复唤醒功能。
|
||||
|
||||
> **link时必须保证 kws 此时已经配置好进入工作状态。link 也可以在 setup 阶段进行。**
|
||||
|
||||
示例:
|
||||
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "melotts",
|
||||
"action": "setup",
|
||||
"object": "melotts.setup",
|
||||
"data": {
|
||||
"model": "melotts_zh-cn",
|
||||
"response_format": "sys.pcm",
|
||||
"input": ["tts.utf-8", "llm.1002", "kws.1000"],
|
||||
"enoutput": false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## unlink
|
||||
取消链接
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "4",
|
||||
"work_id": "melotts.1003",
|
||||
"action": "unlink",
|
||||
"object":"work_id",
|
||||
"data":"kws.1000"
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"4",
|
||||
"work_id":"melotts.1003"
|
||||
}
|
||||
```
|
||||
error::code 为 0 表示执行成功。
|
||||
|
||||
## pause
|
||||
暂停单元工作.
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "5",
|
||||
"work_id": "melotts.1003",
|
||||
"action": "pause",
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"5",
|
||||
"work_id":"melotts.1003"
|
||||
}
|
||||
```
|
||||
|
||||
error::code 为 0 表示执行成功。
|
||||
|
||||
## work
|
||||
恢复单元工作.
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "6",
|
||||
"work_id": "melotts.1003",
|
||||
"action": "work",
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"6",
|
||||
"work_id":"melotts.1003"
|
||||
}
|
||||
```
|
||||
|
||||
error::code 为 0 表示执行成功。
|
||||
|
||||
## exit
|
||||
恢复单元工作.
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "7",
|
||||
"work_id": "melotts.1003",
|
||||
"action": "exit",
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"7",
|
||||
"work_id":"melotts.1003"
|
||||
}
|
||||
```
|
||||
|
||||
error::code 为 0 表示执行成功。
|
||||
|
||||
## taskinfo
|
||||
|
||||
获取任务列表：
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "melotts",
|
||||
"action": "taskinfo"
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731652311,
|
||||
"data":["melotts.1003"],
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"melotts.tasklist",
|
||||
"request_id":"2",
|
||||
"work_id":"melotts"
|
||||
}
|
||||
```
|
||||
|
||||
获取任务运行参数：
|
||||
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "melotts.1003",
|
||||
"action": "taskinfo"
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731652344,
|
||||
"data":{
|
||||
"enoutput":false,
|
||||
"inputs_":["tts.utf-8"],
|
||||
"model":"melotts_zh-cn",
|
||||
"response_format":"sys.pcm"
|
||||
},
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"melotts.taskinfo",
|
||||
"request_id":"2",
|
||||
"work_id":"melotts.1003"
|
||||
}
|
||||
```
|
||||
|
||||
> **注意：work_id 是按照单元的初始化注册顺序增加的，并不是固定的索引值。**
|
||||
> **同类型单元不能配置多个单元同时工作，否则会产生未知错误。例如 tts 和 melo tts 不能同时拍起用工作。**
|
||||
@@ -0,0 +1,20 @@
|
||||
# llm-sys
|
||||
|
||||
The basic service unit of StackFlow, providing external serial and TCP channels and some system function services, while internally handling port resource allocation and a simple in-memory database.
|
||||
|
||||
## External API
|
||||
- sys.ping: Test if communication with the LLM is possible.
|
||||
- sys.lsmode: Models that have existed in the system in the past.
|
||||
- sys.bashexec: Execute bash commands.
|
||||
- sys.hwinfo: Retrieve onboard CPU, memory, and temperature parameters of the LLM.
|
||||
- sys.uartsetup: Set serial port parameters, effective for a single session.
|
||||
- sys.reset: Reset the entire LLM framework application.
|
||||
- sys.reboot: Restart the system.
|
||||
- sys.version: Get the version of the LLM framework program.
|
||||
|
||||
## Internal API
|
||||
- sql_select: Query key-value pairs in the small in-memory KV database.
|
||||
- register_unit: Register a unit.
|
||||
- release_unit: Release a unit.
|
||||
- sql_set: Set key-value pairs in the small in-memory KV database.
|
||||
- sql_unset: Delete key-value pairs in the small in-memory KV database.
|
||||
@@ -0,0 +1,20 @@
|
||||
# llm-sys
|
||||
StackFlow 的基本服务单元，对外提供串口和 TCP 外部信道和一部分系统功能服务，对内进行端口资源分配，和一个简单的内存数据库。
|
||||
|
||||
## 外部 API
|
||||
- sys.ping：测试是否能够和 LLM 进行通信。
|
||||
- sys.lsmode：过去系统中存在的模型。
|
||||
- sys.bashexec：执行 bash 命令。
|
||||
- sys.hwinfo：获取 LLM 板载 cpu、内存、温度参数。
|
||||
- sys.uartsetup：设置串口参数，单次生效。
|
||||
- sys.reset：复位整个 LLM 框架的应用程序。
|
||||
- sys.reboot：重启系统。
|
||||
- sys.version：获取 LLM 框架程序版本。
|
||||
|
||||
## 内部 API：
|
||||
- sql_select：查寻小型内存 KV 数据库键值。
|
||||
- register_unit：注册单元。
|
||||
- release_unit：释放单元。
|
||||
- sql_set：设定小型内存 KV 数据库键值。
|
||||
- sql_unset：删除小型内存 KV 数据库键值。
|
||||
|
||||
@@ -0,0 +1,279 @@
|
||||
# llm-tts
|
||||
Text-to-Speech unit, used to provide text-to-speech services, with options for Chinese and English models to provide text-to-speech services in both languages.
|
||||
|
||||
## setup
|
||||
Configure the unit.
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "tts",
|
||||
"action": "setup",
|
||||
"object": "tts.setup",
|
||||
"data": {
|
||||
"model": "single_speaker_fast",
|
||||
"response_format": "sys.pcm",
|
||||
"input": "tts.utf-8",
|
||||
"enoutput": false
|
||||
}
|
||||
}
|
||||
```
|
||||
- request_id: Refer to the basic data explanation.
|
||||
- work_id: For configuring the unit, it is `tts`.
|
||||
- action: The method to call is `setup`.
|
||||
- object: The type of data being transmitted is `tts.setup`.
|
||||
- model: The model used is the `single_speaker_fast` Chinese model.
|
||||
- response_format: The returned result is `sys.pcm`, system audio data, which is directly sent to the llm-audio module for playback.
|
||||
- input: Input is `tts.utf-8`, representing user input.
|
||||
- enoutput: Whether to enable user result output.
|
||||
Response JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"2",
|
||||
"work_id":"llm.1003"
|
||||
}
|
||||
```
|
||||
|
||||
- created: Message creation time, in Unix time.
|
||||
- work_id: The successfully created work_id unit.
|
||||
|
||||
## link
|
||||
Link the output of the upper unit.
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "3",
|
||||
"work_id": "tts.1003",
|
||||
"action": "link",
|
||||
"object":"work_id",
|
||||
"data":"kws.1000"
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"3",
|
||||
"work_id":"tts.1003"
|
||||
}
|
||||
```
|
||||
error::code 0 indicates successful execution.
|
||||
|
||||
Link the llm and tts units, so that when kws wakes up, the tts unit stops the previous unfinished inference, used for repeat wake-up functionality.
|
||||
|
||||
> **When linking, ensure that kws is already configured and in working state. Linking can also be done during the setup phase.**
|
||||
|
||||
Example:
|
||||
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "tts",
|
||||
"action": "setup",
|
||||
"object": "tts.setup",
|
||||
"data": {
|
||||
"model": "single_speaker_fast",
|
||||
"response_format": "sys.pcm",
|
||||
"input": ["tts.utf-8", "llm.1002", "kws.1000"],
|
||||
"enoutput": false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## unlink
|
||||
Unlink
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "4",
|
||||
"work_id": "tts.1003",
|
||||
"action": "unlink",
|
||||
"object":"work_id",
|
||||
"data":"kws.1000"
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"4",
|
||||
"work_id":"tts.1003"
|
||||
}
|
||||
```
|
||||
error::code 0 indicates successful execution.
|
||||
|
||||
## pause
|
||||
Pause unit work.
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "5",
|
||||
"work_id": "llm.1003",
|
||||
"action": "pause",
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"5",
|
||||
"work_id":"llm.1003"
|
||||
}
|
||||
```
|
||||
error::code 0 indicates successful execution.
|
||||
|
||||
## work
|
||||
Resume unit work.
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "6",
|
||||
"work_id": "llm.1003",
|
||||
"action": "work",
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"6",
|
||||
"work_id":"llm.1003"
|
||||
}
|
||||
```
|
||||
error::code 0 indicates successful execution.
|
||||
|
||||
## exit
|
||||
Exit unit work.
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "7",
|
||||
"work_id": "llm.1003",
|
||||
"action": "exit",
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"7",
|
||||
"work_id":"llm.1003"
|
||||
}
|
||||
```
|
||||
error::code 0 indicates successful execution.
|
||||
|
||||
## Task Information
|
||||
|
||||
Get task list:
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "tts",
|
||||
"action": "taskinfo"
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
```json
|
||||
{
|
||||
"created":1731652311,
|
||||
"data":["tts.1003"],
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"tts.tasklist",
|
||||
"request_id":"2",
|
||||
"work_id":"tts"
|
||||
}
|
||||
```
|
||||
|
||||
Get task runtime parameters:
|
||||
|
||||
Send JSON:
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "tts.1003",
|
||||
"action": "taskinfo"
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
```json
|
||||
{
|
||||
"created":1731652344,
|
||||
"data":{
|
||||
"enoutput":false,
|
||||
"inputs_":["tts.utf-8"],
|
||||
"model":"single_speaker_fast",
|
||||
"response_format":"sys.pcm"
|
||||
},
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"tts.taskinfo",
|
||||
"request_id":"2",
|
||||
"work_id":"tts.1003"
|
||||
}
|
||||
```
|
||||
|
||||
> **Note: work_id increases according to the order of unit initialization registration and is not a fixed index value.**
|
||||
> **The same type of unit cannot have multiple units working simultaneously, as it may cause unknown errors. For example, tts and melo tts cannot be enabled to work at the same time.**
|
||||
@@ -0,0 +1,282 @@
|
||||
# llm-tts
|
||||
文字转语音单元，用于提供文字转语音服务，可选择中英文模型，用于提供中英文文字转语音服务。
|
||||
|
||||
## setup
|
||||
配置单元工作.
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "tts",
|
||||
"action": "setup",
|
||||
"object": "tts.setup",
|
||||
"data": {
|
||||
"model": "single_speaker_fast",
|
||||
"response_format": "sys.pcm",
|
||||
"input": "tts.utf-8",
|
||||
"enoutput": false
|
||||
}
|
||||
}
|
||||
```
|
||||
- request_id：参考基本数据解释。
|
||||
- work_id：配置单元时，为 `tts`。
|
||||
- action：调用的方法为 `setup`。
|
||||
- object：传输的数据类型为 `tts.setup`。
|
||||
- model：使用的模型为 `single_speaker_fast` 中文模型。
|
||||
- response_format：返回结果为 `sys.pcm`, 系统音频数据，并直接发送到 llm-audio 模块进行播放。
|
||||
- input：输入的为 `tts.utf-8`,代表的是从用户输入。
|
||||
- enoutput：是否起用用户结果输出。
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"2",
|
||||
"work_id":"llm.1003"
|
||||
}
|
||||
```
|
||||
- created：消息创建时间，unix 时间。
|
||||
- work_id：返回成功创建的 work_id 单元。
|
||||
|
||||
## link
|
||||
链接上级单元的输出。
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "3",
|
||||
"work_id": "tts.1003",
|
||||
"action": "link",
|
||||
"object":"work_id",
|
||||
"data":"kws.1000"
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"3",
|
||||
"work_id":"tts.1003"
|
||||
}
|
||||
```
|
||||
error::code 为 0 表示执行成功。
|
||||
|
||||
将 llm 和 tts 单元链接起来，当 kws 唤醒时 tts 单元停止上次未完成的推理，用于重复唤醒功能。
|
||||
|
||||
> **link时必须保证 kws 此时已经配置好进入工作状态。link 也可以在 setup 阶段进行。**
|
||||
|
||||
示例:
|
||||
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "tts",
|
||||
"action": "setup",
|
||||
"object": "tts.setup",
|
||||
"data": {
|
||||
"model": "single_speaker_fast",
|
||||
"response_format": "sys.pcm",
|
||||
"input": ["tts.utf-8", "llm.1002", "kws.1000"],
|
||||
"enoutput": false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## unlink
|
||||
取消链接
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "4",
|
||||
"work_id": "tts.1003",
|
||||
"action": "unlink",
|
||||
"object":"work_id",
|
||||
"data":"kws.1000"
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"4",
|
||||
"work_id":"tts.1003"
|
||||
}
|
||||
```
|
||||
|
||||
error::code 为 0 表示执行成功。
|
||||
|
||||
## pause
|
||||
暂停单元工作.
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "5",
|
||||
"work_id": "tts.1003",
|
||||
"action": "pause",
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"5",
|
||||
"work_id":"tts.1003"
|
||||
}
|
||||
```
|
||||
error::code 为 0 表示执行成功。
|
||||
|
||||
## work
|
||||
恢复单元工作.
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "6",
|
||||
"work_id": "tts.1003",
|
||||
"action": "work",
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"6",
|
||||
"work_id":"tts.1003"
|
||||
}
|
||||
```
|
||||
error::code 为 0 表示执行成功。
|
||||
|
||||
## exit
|
||||
恢复单元工作.
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "7",
|
||||
"work_id": "tts.1003",
|
||||
"action": "exit",
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731488402,
|
||||
"data":"None",
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"None",
|
||||
"request_id":"7",
|
||||
"work_id":"tts.1003"
|
||||
}
|
||||
```
|
||||
|
||||
error::code 为 0 表示执行成功。
|
||||
|
||||
## taskinfo
|
||||
|
||||
获取任务列表：
|
||||
|
||||
发送 json：
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "tts",
|
||||
"action": "taskinfo"
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731652311,
|
||||
"data":["tts.1003"],
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"tts.tasklist",
|
||||
"request_id":"2",
|
||||
"work_id":"tts"
|
||||
}
|
||||
```
|
||||
|
||||
获取任务运行参数：
|
||||
|
||||
```json
|
||||
{
|
||||
"request_id": "2",
|
||||
"work_id": "tts.1003",
|
||||
"action": "taskinfo"
|
||||
}
|
||||
```
|
||||
|
||||
响应 json：
|
||||
|
||||
```json
|
||||
{
|
||||
"created":1731652344,
|
||||
"data":{
|
||||
"enoutput":false,
|
||||
"inputs_":["tts.utf-8"],
|
||||
"model":"single_speaker_fast",
|
||||
"response_format":"sys.pcm"
|
||||
},
|
||||
"error":{
|
||||
"code":0,
|
||||
"message":""
|
||||
},
|
||||
"object":"tts.taskinfo",
|
||||
"request_id":"2",
|
||||
"work_id":"tts.1003"
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
> **注意：work_id 是按照单元的初始化注册顺序增加的，并不是固定的索引值。**
|
||||
> **同类型单元不能配置多个单元同时工作，否则会产生未知错误。例如 tts 和 melo tts 不能同时拍起用工作。**
|
||||
@@ -0,0 +1,10 @@
|
||||
|
||||
menuconfig STACKFLOW_ENABLED
|
||||
bool "Enable stackflow"
|
||||
default n
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user