# Local AI

Set up local models with [LocalAI](https://github.com/go-skynet/LocalAI) (LLama, GPT4All, Vicuna, Falcon, etc.)

## Preparation[​](https://doc.chathub.gg/custom-chatbots/openai#preparation) <a href="#preparation" id="preparation"></a>

Go to <https://github.com/go-skynet/LocalAI> and follow their instruction to run a model on your device.

For example, here is the command to setup LocalAI with Docker:

```
docker run -p 8080:8080 -ti --rm -v /Users/tonydinh/Desktop/models:/app/models quay.io/go-skynet/local-ai:latest --models-path /app/models --context-size 700 --threads 4 --cors true
```

{% hint style="info" %}
Note that we added the <mark style="color:red;">`--cors true`</mark> parameter to the command to make sure the local server is accessible from the browser. AIChatOne will send requests to the local model directly from the browser.
{% endhint %}

## Configuration <a href="#configuration" id="configuration"></a>

<figure><img src="https://3481753452-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FzYU2cjsYvY3seDzkySsN%2Fuploads%2FndyzGmev7A4XUQV0HTzo%2Fimage.png?alt=media&#x26;token=33c7ee80-6854-4a4d-9899-6037198afae9" alt=""><figcaption></figcaption></figure>

We're using the `gemma:2b` model in this example.

API Key is required but ignored, so you can input any string.

| Issue                    | Resolution                                                                                                                                                  |
| ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
| CORS related issues CORS | Make sure your server configuration allows the endpoint to be accessible from the browser. Open the Network tab in the browser console to see more details. |
| Long waiting time        | In the first request, your model can take a long time to respond. Check the terminal log of the Docker process to see if anything goes wrong.               |
| API Key Missing          | AIChatOne does not support API key authentication for custom model yet. Please reconfigurate your custom model to remove API key requirement.               |