# Local AI

Set up local models with [LocalAI](https://github.com/go-skynet/LocalAI) (LLama, GPT4All, Vicuna, Falcon, etc.)

## Preparation[​](https://doc.chathub.gg/custom-chatbots/openai#preparation) <a href="#preparation" id="preparation"></a>

Go to <https://github.com/go-skynet/LocalAI> and follow their instruction to run a model on your device.

For example, here is the command to setup LocalAI with Docker:

```
docker run -p 8080:8080 -ti --rm -v /Users/tonydinh/Desktop/models:/app/models quay.io/go-skynet/local-ai:latest --models-path /app/models --context-size 700 --threads 4 --cors true
```

{% hint style="info" %}
Note that we added the <mark style="color:red;">`--cors true`</mark> parameter to the command to make sure the local server is accessible from the browser. AIChatOne will send requests to the local model directly from the browser.
{% endhint %}

## Configuration <a href="#configuration" id="configuration"></a>

<figure><img src="/files/jJpAYbldS7hgUSApSzbt" alt=""><figcaption></figcaption></figure>

We're using the `gemma:2b` model in this example.

API Key is required but ignored, so you can input any string.

| Issue                    | Resolution                                                                                                                                                  |
| ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
| CORS related issues CORS | Make sure your server configuration allows the endpoint to be accessible from the browser. Open the Network tab in the browser console to see more details. |
| Long waiting time        | In the first request, your model can take a long time to respond. Check the terminal log of the Docker process to see if anything goes wrong.               |
| API Key Missing          | AIChatOne does not support API key authentication for custom model yet. Please reconfigurate your custom model to remove API key requirement.               |


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.aichatone.com/product-guides/custom-chatbots/local-llm-models/local-ai.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.