Local AI
Set up local models with LocalAI (LLama, GPT4All, Vicuna, Falcon, etc.)
Preparationβ
Go to https://github.com/go-skynet/LocalAI and follow their instruction to run a model on your device.
For example, here is the command to setup LocalAI with Docker:
docker run -p 8080:8080 -ti --rm -v /Users/tonydinh/Desktop/models:/app/models quay.io/go-skynet/local-ai:latest --models-path /app/models --context-size 700 --threads 4 --cors true
Configuration

We're using the gemma:2b
model in this example.
API Key is required but ignored, so you can input any string.
Issue
Resolution
CORS related issues CORS
Make sure your server configuration allows the endpoint to be accessible from the browser. Open the Network tab in the browser console to see more details.
Long waiting time
In the first request, your model can take a long time to respond. Check the terminal log of the Docker process to see if anything goes wrong.
API Key Missing
AIChatOne does not support API key authentication for custom model yet. Please reconfigurate your custom model to remove API key requirement.
Last updated
Was this helpful?