High-Performance Linux
for Local Autonomous AI.

Configure Device Target Visibility

For systems with multiple graphics cards, control device distribution using environment variables. By default, Ollama will load model segments across all visible devices. Force target cards using CUDA_VISIBLE_DEVICES (NVIDIA) or ROCR_VISIBLE_DEVICES (AMD):

# Limit Ollama target usage to GPU 0 and GPU 1
export CUDA_VISIBLE_DEVICES=0,1

# For AMD ROCm targets
export ROCR_VISIBLE_DEVICES=0,1

Tuning Context Sizes inside Custom Modelfiles

Ollama defaults to a 2,048 token context for many models. To leverage modern 32k or 128k context windows, build a custom model config mapping the desired context token size directly:

# Create a file named Modelfile:
FROM hermes3:8b

# Override context window parameter to 32k tokens
PARAMETER num_ctx 32768

# Customize generation temperature constraints
PARAMETER temperature 0.7

Compile the custom config with: ollama create hermes3-32k -f Modelfile

System Kernel Cache Optimizations

When running massive inference pipelines, keep memory throughput fluid. Prevent caching locks by tuning Arch Linux's kernel virtual memory dirty page controls:

# Maintain high system page availability for streaming tokens
vm.dirty_ratio = 10
vm.dirty_background_ratio = 5
vm.vfs_cache_pressure = 50

Apply settings immediately using: sudo sysctl --system

Install Virt-Manager Toolchain

Install the required virtualization packages, networking drivers, and the VirtIO drivers ISO for Windows systems:

sudo pacman -S qemu-full virt-manager libvirt dnsmasq iptables-nft virtio-win

Enable Service Daemon & Permissions

Add your user account to the libvirt group to grant permission to configure hypervisor configurations, then enable the startup daemon:

sudo usermod -aG libvirt $USER
sudo systemctl enable --now libvirtd.service

Virtual Machine Initialization Settings

Create a new VM in Virt-Manager. During setup, configure the hardware options below to maximize performance and meet Windows 11 system requirements:

• System Firmware: Select UEFI (OVMF) instead of legacy BIOS to enable Secure Boot.
• Virtual Disk Bus: Set the storage disk type to VirtIO instead of IDE/SATA.
• Network Device: Set the device model type to virtio to enable virtual 10Gbps connectivity.
• TPM Hardware: Add a TPM hardware device. Set model to TIS, type to Emulator, and version to 2.0.
• Mount Drivers ISO: Add a secondary CDROM hardware unit pointing to the VirtIO drivers file: /usr/share/virtio/virtio-win.iso.

Tuning PipeWire Clock Quantum

To reduce audio processing latency, configure PipeWire's default quantum parameters. Create a system override configuration mapping the buffers to lower latency values (e.g. 128 frames at 48kHz):

# Append to custom PipeWire daemon overrides
context.properties = {
    default.clock.quantum       = 128
    default.clock.min-quantum   = 64
    default.clock.max-quantum   = 512
    default.clock.rate          = 48000
}

Apply changes by restarting the user-space audio service: systemctl --user restart pipewire.service pipewire-pulse.service

Virtual Audio Loopback Setup

For AI voice agents that monitor or intercept application audio, set up a persistent virtual audio loopback sink. This allows routing desktop audio channels directly to the agent's voice parser:

# Route audio outputs through dynamic virtual pipelines
pactl load-module module-null-sink sink_name=Virtual_AI_Sink sink_properties=device.description=Virtual_AI_Loopback

# Map playback loopback channels
pw-loopback -m '[ FL FR ]' --capture Virtual_AI_Sink:monitor --playback FL,FR

Initialize CachyOS ISO Build Profiles

Install the Arch Linux iso builder toolchain and fetch the CachyOS custom profile structures from the official build repository:

# Install build package dependencies
sudo pacman -S archiso git --needed

# Clone CachyOS official iso profiles
git clone https://github.com/cachyos/cachyos-archiso.git
cd cachyos-archiso/cachyos-desktop

Inject Local AI Packages & Services

Append your target AI dev toolchain packages directly to the profile package manifest file. This guarantees they are baked into the ISO kernel:

# Append directly to cachyos-archiso/cachyos-desktop/packages.x86_64:
ollama-cuda
python-pip
python-virtualenv
docker
docker-compose
git-lfs

Enable Ollama on ISO Boot

To start the Ollama inference server automatically when booting the live installation media, add a systemd symlink trigger inside the profile customization hooks:

# Create custom execution symlinks inside the local configuration structure
ln -sf /usr/lib/systemd/system/ollama.service \
       cachyos-desktop/airootfs/etc/systemd/system/multi-user.target.wants/ollama.service

Compile the final optimized bootable ISO image by running: sudo ./build.sh -v