From 5793b649c659470ac102afbb16aab7680b20bd3c Mon Sep 17 00:00:00 2001 From: Knox Lively Date: Tue, 7 May 2024 00:33:01 -0600 Subject: [PATCH 1/4] re-woreded the ui config section --- features/supportedLLMs/BYOLLM.mdx | 2 +- features/supportedLLMs/llamacpp.mdx | 2 +- features/supportedLLMs/localai.mdx | 2 +- features/supportedLLMs/ollama.mdx | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/features/supportedLLMs/BYOLLM.mdx b/features/supportedLLMs/BYOLLM.mdx index 73554f7..044e24a 100644 --- a/features/supportedLLMs/BYOLLM.mdx +++ b/features/supportedLLMs/BYOLLM.mdx @@ -23,7 +23,7 @@ After installing and configuring your own LLM provider, you should be able to st **Note:** If the provided URL (e.g., `http://localhost:8080/v1/chat/completions`) doesn't work, try removing the `/chat/completions` directories from the end of the URL or using just the hostname and port (e.g., `http://localhost:8080`). This often resolves compatibility issues and allows Wave AI to communicate with your LLM provider successfully. ### Configuring via the UI -To configure your own LLM provider from the Wave AI user interface, navigate to the "Settings" menu and set the `AI Base URL` and `AI Model` parameters as described in the previous section. +To configure your own LLM provider from Wave's user interface, navigate to the "Settings" menu and set the `AI Base URL` and `AI Model` parameters as described in the previous section. ### Configuring via the CLI To configure your own LLM provider using the command line, set the `aibaseurl` and `aimodel` parameters using the [/client:set](/reference/slashcommands#client-set) command, as shown in the example below. diff --git a/features/supportedLLMs/llamacpp.mdx b/features/supportedLLMs/llamacpp.mdx index 4a8c7d5..7dd9490 100644 --- a/features/supportedLLMs/llamacpp.mdx +++ b/features/supportedLLMs/llamacpp.mdx @@ -26,7 +26,7 @@ After installing and configuring the llama.cpp server, you can start using it in - **AI Base URL:** Set this parameter to the base URL or endpoint that Wave AI should query. For llama.cpp servers running locally, use http://localhost:8080. Please note that the port number `8080` may be different depending on your specific installation. For remote llama.cpp server instances, replace `localhost` with the appropriate hostname or IP address of the server where llama.cpp is running. If the port number is different from the default `8080`, update it accordingly in the URL. ### Configuring via the UI -To configure llama.cpp from the Wave AI user interface, navigate to the "Settings" menu and set the `AI Base URL` parameter as described in the previous section. +To configure llama.cpp from Wave's user interface, navigate to the "Settings" menu and set the `AI Base URL` parameter as described in the previous section. ### Configuring via the CLI To configure llama.cpp using the command line, set the `aibaseurl` parameter using the [/client:set](/reference/slashcommands#client-set) command, as shown in the example below. diff --git a/features/supportedLLMs/localai.mdx b/features/supportedLLMs/localai.mdx index 6169937..f15f4f9 100644 --- a/features/supportedLLMs/localai.mdx +++ b/features/supportedLLMs/localai.mdx @@ -25,7 +25,7 @@ After installing and configuring LocalAI, you can start using it in Wave by sett Example: http://localhost:8080/v1/models. ### Configuring via the UI -To configure LocalAI from the Wave AI user interface, navigate to the "Settings" menu and set the `AI Base URL` and `AI Model` parameters as described in the previous section. +To configure LocalAI from Wave's user interface, navigate to the "Settings" menu and set the `AI Base URL` and `AI Model` parameters as described in the previous section. ### Configuring via the CLI To configure LocalAI using the command line, set the `aibaseurl` and `aimodel` parameters using the [/client:set](/reference/slashcommands#client-set) command, as shown in the example below. diff --git a/features/supportedLLMs/ollama.mdx b/features/supportedLLMs/ollama.mdx index 2c89726..42bc382 100644 --- a/features/supportedLLMs/ollama.mdx +++ b/features/supportedLLMs/ollama.mdx @@ -21,7 +21,7 @@ After installing and configuring Ollama, you can start using it in Wave by setti - **AI Model**: Specify the Ollama model you want to use. This can be any "pulled" model in Ollama and doesn't need to be actively running. To discover available models, use the `ollama list` command in your terminal. ### Configuring via the UI -To configure Ollama from the Wave AI user interface, navigate to the "Settings" menu and set the `AI Base URL` and `AI Model` parameters as described in the previous section. +To configure Ollama from Wave's user interface, navigate to the "Settings" menu and set the `AI Base URL` and `AI Model` parameters as described in the previous section. ### Configuring via the CLI To configure Ollama using the command line, set the `aibaseurl` and `aimodel` parameters using the [/client:set](/reference/slashcommands#client-set) command, as shown in the example below. From 45f4afc88d0f028f65a634d5a19222e6c692bd94 Mon Sep 17 00:00:00 2001 From: Knox Lively Date: Tue, 7 May 2024 00:41:51 -0600 Subject: [PATCH 2/4] added lmstudio to the third-party llm list in wave --- features/supportedLLMs/lmstudio.mdx | 66 +++++++++++++++++++++++++++++ features/waveAI.mdx | 1 + 2 files changed, 67 insertions(+) create mode 100644 features/supportedLLMs/lmstudio.mdx diff --git a/features/supportedLLMs/lmstudio.mdx b/features/supportedLLMs/lmstudio.mdx new file mode 100644 index 0000000..7609598 --- /dev/null +++ b/features/supportedLLMs/lmstudio.mdx @@ -0,0 +1,66 @@ +--- +title: 'LM Studio' +description: 'Get LM Studio up and running in Wave AI' +--- + +[LM Studio](https://lmstudio.ai/) is a desktop application that allows users to discover, download, and run Large Language Models (LLMs) offline on their personal computers. It supports ggml-compatible models from Hugging Face, including Llama, MPT, and StarCoder. LM Studio provides a user-friendly Chat UI for engaging with the models and an OpenAI-compatible local server for seamless integration with other tools and applications. + +Wave specifically supports the [LM Studio Server](https://lmstudio.ai/docs/local-server), which makes the models available locally to run from other applications like Wave. + + +To see a full list of supported LLM providers, please visit the [Third-Party LLM Support](/features/waveAI#third-party-llm-support) section in the Wave AI features page. + + +## Installation +Please visit LM Studio's [Website](https://lmstudio.ai/) to download and install LM Studio. + +## Configuration +After downloading and installing LM Studio, you need to configure and start the [LM Studio server](https://lmstudio.ai/docs/local-server#using-the-local-server). Once you've started the LM Studio server, you can start using it in Wave by setting a single parameter: `aibaseurl`. This parameter can be set either through the UI or from the command line, but please note that the parameter names are slightly different depending on the method you choose. + +### **Parameters** + +- **AI Base URL:** Set this parameter to the base URL or endpoint that Wave AI should query. If you've started the server using the default configuration, use http://localhost:1234/v1 as your base url. Otherwise, if you've changed the port, please update this example to use the appropriate port number. + +### Configuring via the UI +To configure LM Studio from Wave's user interface, navigate to the "Settings" menu and set the `AI Base URL` parameter as described in the previous section. + +### Configuring via the CLI +To configure LM Studio using the command line, set the `aibaseurl` parameter using the [/client:set](/reference/slashcommands#client-set) command, as shown in the example below. + +``` +/client:set aibaseurl= +``` + +## Usage +Once you have configured the LM Studio server for Wave you can begin using it. There are two primary ways to interact with your newly configured LLM: [Interactive Mode](/features/waveAI#interactive) and by using the [/chat](/features/waveAI#chat-command) command. + +- **Interactive Mode:** To enter Interactive Mode, click the "Wave AI" button in the command box or use the `ctrl + space` shortcut. This will open an interactive chat session where you can have a continuous conversation with the AI assistant powered by your LM Studio model. +- **/chat:** Alternatively, you can use the [/chat](/features/waveAI#chat-command) command followed by your question to get a quick answer from your LM Studio model directly in the terminal. + +## Troubleshooting +If you encounter issues while using LM Studio with Wave AI, consider the following troubleshooting steps: + +- **Connection failures:** If Wave AI fails to connect to your LM Studio server or returns an error message, [verify](https://lmstudio.ai/docs/local-server#make-an-inferencing-request-using-openais-chat-completions-format) that the LM Studio server is running is running and accessible from the system where Wave is installed. Check the LM Studio logs for any error messages or indications of why the connection might be failing. +- **Timeouts:** If you're unable to complete a query or incur frequent timeouts, try adjusting the `aitimeout` parameter to a higher value. This will give your LM Studio server more time to process and respond to your requests, especially if you are running it on a system with limited hardware resources. +- **Incorrect base URL or port:** Ensure that the `aibaseurl` parameter points to the correct URL and port number where the LM Studio server is running. If you have changed the default port or are running LM Studio on a remote server, update the URL accordingly. +- **Unexpected behavior or inconsistent results:** If you encounter unexpected behavior or inconsistent results when using the LM Studio server with Wave AI, try [resetting](#reset-wave-ai) the `aibaseurl` and `aimodel` parameters to their default values and reconfiguring LM Studio from scratch. This can help rule out any configuration issues that might be causing problems. + + +If you continue to face issues after trying these troubleshooting steps, please see the [Additional Resources](#additional-resources) section below for further assistance, or feel free to reach out to us on [Discord](https://discord.gg/XfvZ334gwU). + + +## Reset Wave AI +At any time if you find that you wish to return to the default Wave AI experience, you can reset the `aibaseurl` and `aimodel` parameters to their default state by using the following commands. + +``` +/client:set aibaseurl= +/client:set aimodel= +``` +**Note:** This can also be done in the UI just as described in previous steps. + +## Additional Resources +* [LM Studio Website](https://lmstudio.ai/) +* [LM Studio Docs](https://lmstudio.ai/docs/welcome) +* [LM Studio GitHub Page](https://github.com/lmstudio-ai) +* [LM Studio Discord Server](https://discord.com/invite/aPQfnNkxGC) +* [LM Studio Twitter](https://twitter.com/lmstudioai) \ No newline at end of file diff --git a/features/waveAI.mdx b/features/waveAI.mdx index 6ff4480..d33f900 100644 --- a/features/waveAI.mdx +++ b/features/waveAI.mdx @@ -50,6 +50,7 @@ To get started with a specific integration, simply click on the integration to a ### Supported LLM Providers: * [llamap.cpp](/features/supportedLLMs/llamacpp) +* [LMStudio](/features/supportedLLMs/lmstudio) * [LocalAI](/features/supportedLLMs/localai) * [Ollama](/features/supportedLLMs/ollama) * [BYOLLM (Bring Your Own LLM)](/features/supportedLLMs/BYOLLM) From 0c38f754ff2dd5534b58ec17d490e063d5300a6e Mon Sep 17 00:00:00 2001 From: Knox Lively Date: Tue, 7 May 2024 00:57:25 -0600 Subject: [PATCH 3/4] put providers in an accordian group --- features/waveAI.mdx | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/features/waveAI.mdx b/features/waveAI.mdx index d33f900..c19be72 100644 --- a/features/waveAI.mdx +++ b/features/waveAI.mdx @@ -49,11 +49,21 @@ Wave AI supports various third-party Large Language Model providers, allowing yo To get started with a specific integration, simply click on the integration to access the setup instructions and configuration details for that particular LLM provider. ### Supported LLM Providers: -* [llamap.cpp](/features/supportedLLMs/llamacpp) -* [LMStudio](/features/supportedLLMs/lmstudio) -* [LocalAI](/features/supportedLLMs/localai) -* [Ollama](/features/supportedLLMs/ollama) -* [BYOLLM (Bring Your Own LLM)](/features/supportedLLMs/BYOLLM) + + + - [llama.cpp](/features/supportedLLMs/llamacpp) + - [LM Studio](/features/supportedLLMs/lmstudio) + - [LocalAI](/features/supportedLLMs/localai) + - [Ollama](/features/supportedLLMs/ollama) + - [BYOLLM (Bring Your Own LLM)](/features/supportedLLMs/byollm) + + + + + + + + **Note:** Currently, Wave's third-party LLM integration supports language models that are compatible with [OpenAI's Chat Completions API](https://platform.openai.com/docs/api-reference/chat). We are actively working on expanding our integrations to support a wider range of LLM providers and API formats in the near future. From e672e64110ecab3f42d3ccb4fe2b59fa59dde1e3 Mon Sep 17 00:00:00 2001 From: Knox Lively Date: Tue, 7 May 2024 10:04:33 -0600 Subject: [PATCH 4/4] minor change to wording --- features/waveAI.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/features/waveAI.mdx b/features/waveAI.mdx index c19be72..b3edc11 100644 --- a/features/waveAI.mdx +++ b/features/waveAI.mdx @@ -44,7 +44,7 @@ Wave AI comes with default settings that work out of the box. By default, Wave w **Note:** In order to prevent abuse, [telemetry](/reference/telemetry) must be enabled to use the cloud servers. There is currently a rate limit of 200 requests per day. ## Third-Party LLM Support -Wave AI supports various third-party Large Language Model providers, allowing you to choose the model that best suits your needs and preferences. This section provides a comprehensive list of the third-party LLM providers that are compatible with Wave AI, enabling you to make an informed decision based on your specific needs and concerns. +Wave AI supports various third-party Large Language Model providers, allowing you to choose the model that best suits your needs and preferences. This section provides a comprehensive list of providers that are compatible with Wave AI, enabling you to make an informed decision based on your specific needs and concerns. To get started with a specific integration, simply click on the integration to access the setup instructions and configuration details for that particular LLM provider.