RIoT AI setup

This is an instruction on how to set up a mac from scratch.

Install Homebrew

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

echo >> /Users/riot/.zprofile
echo 'eval "$(/opt/homebrew/bin/brew shellenv zsh)"' >> /Users/riot/.zprofile
eval "$(/opt/homebrew/bin/brew shellenv zsh)"

Install Ollama

curl -fsSL https://ollama.com/install.sh | sh

Install Docker

brew install --cask docker-desktop

open -a Docker

Click through the install menu and if prompted, install software update.

Adjust resources

Docker -> Settings -> Resources:

CPU limit: MAX
Memory limi 20GB
Disk usage limit: MAX

Then hit Apply & restart

Install Onyx

curl -fsSL https://onyx.app/install_onyx.sh | bash

To acknowledge press Enter
Choose 2 for standard
Press Enter for edge

Onyx created a directory (~/onyx_data) in which you can find the docker compose to manually stop it or modify the images if needed. There is also a .env but we don't touch that since it is not necessary.

Disable sleeping for the mac mini sudo pmset -a sleep 0 disksleep 0

Set up ollama models

Clone this repo into home

cd ~
git clone https://github.com/cediackermann/riotsecure.git
cd ~/riotsecure

Create models from modelfiles
```
./updateModels.sh modelfiles
```
The needed model is pulled automatically, you just need to wait until it's done.
Navigate to the web-interface located on port 3000.
Make yourself an admin account
Go to Admin Panel -> Language Models
Select Ollama as a provider and give the provider a name.
Set the API base URL to http://host.docker.internal:11434
Refresh the models and select at least the three riot models and hit connect.
Go to Admin Panel -> Chat Preferences -> System prompt -> Modify Prompt. Delete the whole prompt and Save.

Upload RAG files

Upload as file

If needed, adjust the urls in riot-sources.txt

Run bash script to get json files

cd ~/riotsecure
./fetchContent.sh riot-sources.txt

Go to the web-interface -> Admine Panel -> Add Connector -> File
Give the connector a name and upload the files from ~/riotsecure/content
Give it a few seconds ~30 until it finished indexing.

Upload as URL

Go to the web-interface -> Admine Panel -> Add Connector -> Web
Add every single needed URL as a connector
For each URL choose the scrape method single
Give it a few seconds ~30 until it finished indexing.

2.5 KiB Raw Blame History