From 7cf6dbbc52052c3e89289a8c478c14506c08c5ab Mon Sep 17 00:00:00 2001 From: Knox Lively Date: Tue, 9 Apr 2024 23:39:02 -0600 Subject: [PATCH] added ollama support (#24) * added ollama support * added clarification around the customization paraemeters for Wave AI * revised openapitoken section to mention that API tokens are never sent to Waves servers --- features/supportedLLMs/ollama.mdx | 48 +++++++++++++++++++++++++++++++ features/waveAI.mdx | 27 +++++++++++------ 2 files changed, 67 insertions(+), 8 deletions(-) create mode 100644 features/supportedLLMs/ollama.mdx diff --git a/features/supportedLLMs/ollama.mdx b/features/supportedLLMs/ollama.mdx new file mode 100644 index 0000000..da89930 --- /dev/null +++ b/features/supportedLLMs/ollama.mdx @@ -0,0 +1,48 @@ +--- +title: 'Ollama' +description: 'Get Ollama up and running in Wave AI' +--- + +[Ollama](https://ollama.com/) is an open-source language model that offers a powerful and flexible alternative to proprietary LLMs, allowing you to run the model locally or on your own server infrastructure. Ollama provides high-quality language generation and understanding capabilities while giving you full control over your data and privacy. + +## Installation +Please visit Ollama's [GitHub](https://github.com/ollama/ollama?tab=readme-ov-file) page for instructions on downloading and installing Ollama, as well as a quickstart guide and a full list of supported models. + +## Configuration +After installing and configuring Ollama, you can start using it in Wave by setting two parameters: `openaibaseurl` and `openaimodel`. These parameters can be set either through the UI or from the command line, but please note that the parameter names are slightly different depending on the method you choose. + +### **Parameters** + +- **Base URL:** Set this parameter to the base URL or endpoint that Wave AI should query. For Ollama running locally, use http://localhost:11434/v1. Please note that the port number `11434` may be different depending on your specific installation. For remote Ollama instances, replace `localhost` with the appropriate hostname or IP address of the server where Ollama is running. If the port number is different from the default `11434`, update it accordingly in the URL. +- **AI Model**: Specify the Ollama model you want to use. This can be any "pulled" model in Ollama and doesn't need to be actively running. To discover available models, use the `ollama list` command in your terminal. + +### Configuring via the UI +To configure Ollama from the Wave AI user interface, navigate to the "Settings" menu and set the `AI Base URL` and `AI Model` parameters as described in the previous section. + +### Configuring via the CLI +To configure Ollama using the command line, set the `openaibaseurl` and `openaimodel` parameters using the [/client:set](/reference/slashcommands#client-set) command, as shown in the example below. Replace the values with the appropriate `Base URL` and `AI Model` for your Ollama installation. + +``` +/client:set openaibaseurl= +/client:set openaimodel= +``` + +## Usage +Once you have installed and configured Ollama, you can start using it in Wave. There are two primary ways to interact with your newly configured LLM: [Interactive Mode](/features/waveAI#interactive) and by using the [/chat](/features/waveAI#chat-command) command. + +- **Interactive Mode:** To enter Interactive Mode, click the "Wave AI" button in the command box or use the `ctrl + space` shortcut. This will open an interactive chat session where you can have a continuous conversation with the AI assistant powered by your Ollama model. +- **/chat:** Alternatively, you can use the [/chat](/features/waveAI#chat-command) command followed by your question to get a quick answer from your Ollama model directly in the terminal. + +## Reset Wave AI +At any time if you find that you wish to return to the default Wave AI experience, you can reset the `openaibaseurl` and `openaimodel` parameters to their default state by using the following commands. + +``` +/client:set openaibaseurl= +/client:set openaimodel= +``` +**Note:** This can also be done in the UI just as described in previous steps. + +## Additional Resources +* [Ollama Website](https://ollama.com/) +* [Ollama GitHub Page](https://github.com/ollama/ollama?tab=readme-ov-file) +* [Ollama Support (Discord)](https://discord.com/invite/ollama) \ No newline at end of file diff --git a/features/waveAI.mdx b/features/waveAI.mdx index d0cd65e..776ba07 100644 --- a/features/waveAI.mdx +++ b/features/waveAI.mdx @@ -6,6 +6,8 @@ icon: 'robot' Wave AI is our native ChatGPT integration that allows you to ask questions and receive answers directly from your terminal. This feature streamlines your workflow by providing AI assistance without the need to switch between multiple tools or applications. +Wave AI also supports the use of third party LLM's, allowing you to tailor your AI experience to your specific needs and concerns, whether they're related to privacy, ethics, or accessing the latest technologies. See the [Third-Party LLM Support](#third-party-llm-support) section for a full list of supported LLMs. + ## Using Wave AI ​There are currently two ways to use Wave AI: interactively, and via the "/chat" command. @@ -30,15 +32,24 @@ This method is ideal for one-off queries or when you need a straightforward answ ## Customization -Wave AI comes with default settings that work out of the box. However, you can customize your experience by modifying the following settings in the UI, or by using the [/client:set](/reference/slashcommands#client-set) command. +Wave AI comes with default settings that work out of the box. By default, Wave will proxy your requests through our cloud servers to OpenAI. However, you can customize your experience by modifying the following settings in the UI or by using the [/client:set](/reference/slashcommands#client-set) command: -- `openaiapitoken`: If you prefer to use your own OpenAI API key, you can set it up in the Wave settings. This allows you to have more control over your API usage and billing. -- `openaimaxtokens`: This setting allows you to control the maximum number of tokens (words or word pieces) that the AI model will generate in a single response. -- `openaimaxchoices`: This option determines the number of different response variations the AI model will generate for each query. Increasing this value will provide more diverse responses, while decreasing it will make the model's output more focused and consistent. -- `openaimodel`: Wave AI uses the ChatGPT *gpt-3.5-turbo* model by default, but you can choose a different model that suits your needs. Simply update the model settings in the Wave configuration. -- `openaibaseurl`: If you want to use other 3rd party APIs compatible with the OpenAI API, you can change the base URL using the following command: +- `openaiapitoken:` Set your own OpenAI API key if you prefer not to use Wave's default configuration. Note that your API token will never be sent to Wave's cloud servers. When you provide your own API key, all future requests will be sent directly to OpenAI or any other endpoint you specify using `openaibaseurl`. +- `openaimaxtokens:` This setting allows you to control the maximum number of tokens (words or word pieces) that the AI model will generate in a single response. +- `openaimaxchoices:` This option determines the number of different response variations the AI model will generate for each query. Increasing this value will provide more diverse responses, while decreasing it will make the model's output more focused and consistent. +- `openaimodel:` Wave AI uses the ChatGPT *gpt-3.5-turbo* model by default, but you can choose a different model that suits your needs. You will need to also set `openaiapitoken` if you choose to use another ChatGPT model. Also, when configuring other third-party services you will want to change this setting to the appropriate model. +- `openaibaseurl:` If you want to use other 3rd party APIs compatible with the OpenAI API, you can change the base URL. You will also want to set your `openaiapitoken` in conjunction with the base url to use a different service. + +**Note:** In order to prevent abuse, [telemetry](/reference/telemetry) must be enabled to use the cloud servers. There is currently a rate limit of 200 requests per day. + +## Third-Party LLM Support +Wave AI supports various third-party Large Language Models (LLMs), allowing you to choose the model that best suits your needs and preferences. This section provides a comprehensive list of the third-party LLMs compatible with Wave AI, enabling you to make an informed decision based on your specific needs and concerns. + +To get started with a specific integration, simply click on the integration to access the setup instructions and configuration details for that particular LLM. + +### Supported LLM's: +* [Ollama](/features/supportedLLMs/ollama) -In order to prevent abuse, [telemetry](/reference/telemetry) must be enabled to use the cloud servers. There is currently a rate limit of 200 requests per day. ## Future Plans -We're excited about the future of Wave AI and have plans to expand its capabilities. In the near future, we'll be introducing BYOLLM (Bring Your Own Large Language Model), which will enable users to connect Wave with any LLM they prefer. Possible integrations include Anthropic's Claude, Google's Gemini, or even local LLMs like OpenLLAMA. This reflects our commitment to user choice, allowing you to tailor your AI experience to your specific needs and concerns, whether they're related to privacy, ethics, or accessing the latest technology. \ No newline at end of file +We're excited about the future of Wave AI and have plans to expand its capabilities. In the near future, we'll be expanding our BYOLLM (Bring Your Own Large Language Model) offerings to include more integrations around popular cloud-based models like Anthropic's Claude, Google's Gemini, and more. \ No newline at end of file